Hey guys so I'm building a Meteor application that is using a mongo database.
I have a collection that could potentially have 1000s of documents that need to be updated at different times...
Do I run setTimeouts on creation or a cron job that runs every second and loops over every document?
what are the pros and cons of doing each?
to put this into context:
I'm building an online tournament system. I can have 100s of tournaments running which means I could have 1000s of matches.
Each match needs to absolutely end at a specific time, and can end earlier under a condition.
Using an OS-level cron job won't work because you can only check with a 60-second resolution. So by "cron job", I think you mean a single
setTimeout (or synced-cron). Here are some thoughts:
strategy: Every second wake up and check a large number of matches, updating those which are complete. If you have multiple servers, you can prevent all but one of them from doing the check via synced-cron.
The advantage of this strategy is that it's straightforward to implement. The disadvantages are:
I'd recommend this strategy if you are confident that the runtime can be controlled. For example, if you can index your matches on an
endTime so only a few matches need to be checked in each cycle.
strategy: Add a
setTimeout for each match on creation or when the sever starts. As each timeout expires, update the corresponding match.
The advantage of this strategy is that it potentially removes a considerable amount of unnecessary database traffic. The disadvantages are:
I'd recommend this strategy if you think you will use a single server for the foreseeable future.
Those are the trade-offs which occurred to me given the choices you presented. A more robust solution would probably involve technology outside of the meteor/mongo stack. For example, storing match times in redis and then listening for keyspace notifications.
This is all a matter of preference, to be honest with you.
I'm a big fan of writing small, independent programs, that each do one thing, and do it well. If you're also like this, it's probably better to write separate programs to run periodically via cron.
This way you get guaranteed OS-controlled precision for the time, and small, simple programs that are easy to debug outside the context of your webapp.
This is just a preference though.
©2020 All rights reserved.