Implement a background task processing structure
It's a pre-requisite in some cases, a nice to have in others, but it also gives you better bang for your buck when it comes to hosting.
In the early phases, everything runs on the main thread of the application.
But processing on the main thread blocks it and means you cannot serve requests in the meantime. It quickly starts causing a poor user experience, and you’ll start implementing a background processing structure.
Emails are often the first thing to be moved to background processing.
Quite a few things, however, can be processed in the background. In fact, anything that doesn’t change the current user’s perception of the output can.
When the user places an order, for example, anything to do with the supplier can be processed in the background.
It’s wise to implement this pattern at the start. If it’s easy to use and robust enough, considering whether to use it would become a regular part of development.
Implementation is very similar across different tools. Most popular frameworks have extensions for background processing. For Rails, that means running Sidekiq or ActiveJob, among others.
The result is a structure where code can be executed outside of the main thread with minimal effort on the developers’ part.
Having a dashboard that shows the status, queue, errors, and retries is also very helpful.
Sidekiq for Rails looks like this for instance:
Things that must take place in the background thread include:
- Collecting stats
- Email sending
- Any notifications such as push or SMS
- Most API calls - add a
statusfield that indicates progress.
- Processing files - converting formats, image resizing, optimization
- Large exports - exports that contain many entries should be generated in the background and once complete, can be emailed
- Running scheduled (cron) jobs
But things that can be done in the background, but might not be:
- Calculations or changes that affect the other party - such as calculating supplier earnings when an order is placed
- Pre-emptively populating fields based on defaults - such as stock information, or delivery timing
- Creating related entries - such as audit log entries
Previous The more documentation you can generate automatically, the better
Suggest an improvement to this page (firstname.lastname@example.org)