Transactionally Staged Job Drain Queues

Written
  • Author: Brandur Leach
  • Source: https://brandur.org/job-drain
  • For transactions that need to enqueue background jobs it's useful to not talk directly to the job queue, but to add your jobs to a table and have another process that periodically sweeps that table and enqueues the jobs in it.
  • This way you can do more work in SQL transactions, and the the job queueing can just be another part of the transaction.

Thanks for reading! If you have any questions or comments, please send me a note on Twitter. And if you enjoyed this, I also have a newsletter where I send out interesting things I read and the occasional nature photo.

You can check out a recent issue, or enter your email below to subscribe.