Sidekiq

Last updated
Sidekiq
Developer(s) Mike Perham
Initial releaseFebruary 5, 2012;13 years ago (2012-02-05) [1]
Stable release
8.0.3 [2]   OOjs UI icon edit-ltr-progressive.svg / 24 April 2025;34 days ago (24 April 2025)
Repository
Written in Ruby
Operating system Cross-platform
Available inEnglish
Type Working queue
License LGPLv3
Website sidekiq.org OOjs UI icon edit-ltr-progressive.svg

Sidekiq is an open source background job framework written in Ruby. [3]

Contents

Architecture

Sidekiq uses Redis for its persistent data store. Each job is stored as a map of key/value pairs, serialized using JSON. Developers can use any programming language to create jobs by constructing the necessary JSON and pushing it into the queue in Redis. A Sidekiq process reads jobs from that Redis queue, using the First In First Out (FIFO) model, and executes the corresponding Ruby code. Job processing is asynchronous, allowing a web-serving thread to continue serving new requests rather than be blocked processing slower tasks.

Sidekiq can be used standalone, or integrated with a Ruby on Rails web application. Sidekiq is multithreaded so multiple jobs can execute concurrently within one process. A large scale application may have dozens or hundreds of Sidekiq processes executing thousands of jobs per second.

Sidekiq comes with a graphical web interface for inspecting and managing job data.

Business model

Sidekiq uses an Open Core business model to provide sustainability for the open source project. [4] The company behind Sidekiq, Contributed Systems, sells closed-source commercial versions, Sidekiq Pro and Sidekiq Enterprise, which contain additional features not included in the open source version.

Reception and use

Sidekiq is described as “well-known queue processing software”. [5]

It's used by Ruby applications like Mastodon, Diaspora, [6] GitLab and Discourse, that need to run tasks in the background, without making web requests wait. Sidekiq is also used to submit threads to the PHASTER phage search tool. [7]

References

  1. v0.5.0
  2. "Release 8.0.3". 24 April 2025. Retrieved 1 May 2025.
  3. https://sidekiq.org
  4. https://codecodeship.com/blog/2023-04-14-mike-perham
  5. Cukier, Daniel (2013). DevOps patterns to scale web applications using cloud services . SPLASH '13. doi:10.1145/2508075.2508432.
  6. Diaspora Project (19 May 2013). "diaspora* 0.1.0.0". GitHub . Retrieved 20 January 2014.
  7. Arndt, David; Grant, Jason R.; Marcu, Ana; Sajed, Tanvir; Pon, Allison; Liang, Yongjie; Wishart, David S. (8 July 2016). "PHASTER: a better, faster version of the PHAST phage search tool". Nucleic Acids Research. 44 (W1): W16 –W21. doi:10.1093/nar/gkw387. ISSN   0305-1048. PMC   4987931 . PMID   27141966.