Background tasks in Meteor

I'm wondering, is there is way to implement background taks, maybe with workers pool. Can you show me direction, i'm thinking about writing package for this?

Answers:

Answer

2019 update

Before thinking about writing a package for anything, first look if there are existing packages that do what you need. In the Meteor world, this means looking on Atmosphere for "job/queue/task/worker management/scheduling" packages, then on npm for the same search terms. You also need to define your requirements more precisely:

  • do you want persistence, or would an in-memory solution work?
  • do you want to be able to distribute jobs to different machines?

Meteor-specific

  • job-collection - reliable (I used it in 2014 in production at a startup), but currently in maintenance mode. Lets you schedule persistent jobs to be run anywhere (servers, clients).
  • SteveJobs - actively maintained by Max Savin, the author of several powerful Meteor tools
  • littledata:synced-cron - "A simple cron system for Meteor. It supports syncronizing jobs between multiple processes."

Abandoned packages:

Npm packages

Meteor has been able to use npm packages directly for several years now, so this question amounts to finding job/worker/queue management packages on NPM. If you don't care about persistence:

  • Async "provides around 70 functions that include the usual 'functional' suspects (map, reduce, filter, each...) as well as some common patterns for asynchronous control flow (parallel, series, waterfall...)"
  • d3-queue - minimalistic, written by D3 author Mike Bostock

If you do want persistence, since Meteor uses MongoDB already, it may be advantageous to use a job scheduling package with persistence to MongoDb. The most powerful and popular seems to be Agenda, but unfortunately it hasn't been maintained in months, and it has a significant backlog of issues.

If you're willing to add a dependency backed by redis to your project, there are more choices:

Like MongoDB, Redis can also provide high-availability (via Redis Sentinel), and if you want to distribute jobs among multiple worker machines, you can point them all at the same Redis server.

Answer

There is a package based on Cron jobs which can be used to schedule tasks on certain intervals or dates. Here is the package: https://atmosphere.meteor.com/package/cron

And if you happen to look into the source of that package, you'll notice they're simply using:

Meteor.setInterval( ... , delay );

So if you save your tasks in a database, then load them into intervals during startup, then you'll probably be on the right track.

Answer

I'm guessing proper support is on their roadmap, but in the meantime I've managed to get it going in a hacky way via setInterval. See the cron-tick package.

Answer

If you are looking for something that is specific to Meteor, I am happy to share there is a new package called Steve Jobs. It makes running background jobs as easy as calling a Method.

It has all the standard features you would expect, such as running a job only once, retrying failed jobs, and so on. You can learn more about it on GitHub:

http://github.com/msavin/stevejobs

Tags

Recent Questions

Top Questions

Home Tags Terms of Service Privacy Policy DMCA Contact Us

©2020 All rights reserved.