I'm wondering, is there is way to implement background taks, maybe with workers pool. Can you show me direction, i'm thinking about writing package for this?
Before thinking about writing a package for anything, first look if there are existing packages that do what you need. In the Meteor world, this means looking on Atmosphere for "job/queue/task/worker management/scheduling" packages, then on npm for the same search terms. You also need to define your requirements more precisely:
Meteor has been able to use npm packages directly for several years now, so this question amounts to finding job/worker/queue management packages on NPM. If you don't care about persistence:
each...) as well as some common patterns for asynchronous control flow (
If you do want persistence, since Meteor uses MongoDB already, it may be advantageous to use a job scheduling package with persistence to MongoDb. The most powerful and popular seems to be Agenda, but unfortunately it hasn't been maintained in months, and it has a significant backlog of issues.
If you're willing to add a dependency backed by redis to your project, there are more choices:
Like MongoDB, Redis can also provide high-availability (via Redis Sentinel), and if you want to distribute jobs among multiple worker machines, you can point them all at the same Redis server.
There is a package based on Cron jobs which can be used to schedule tasks on certain intervals or dates. Here is the package: https://atmosphere.meteor.com/package/cron
And if you happen to look into the source of that package, you'll notice they're simply using:
Meteor.setInterval( ... , delay );
So if you save your tasks in a database, then load them into intervals during startup, then you'll probably be on the right track.
I'm guessing proper support is on their roadmap, but in the meantime I've managed to get it going in a hacky way via
setInterval. See the cron-tick package.
If you are looking for something that is specific to Meteor, I am happy to share there is a new package called Steve Jobs. It makes running background jobs as easy as calling a Method.
It has all the standard features you would expect, such as running a job only once, retrying failed jobs, and so on. You can learn more about it on GitHub:
©2020 All rights reserved.