Skip to main content

Running cron scripts every minute is a waste of system resources

If you've ever done this in a crontab file to handle a user-initiated event that rarely happens:
* * * * * /path/to/script
Then you are definitely doing it wrong. I know I'm guilty of doing that but only because until recently I didn't have a viable solution for the problem. That type of cron line is lazy and running a script every single minute for a process that only happens on rare occasions is pretty wasteful of system resources. A much better solution is to use either Cloud Storage Server /scripts or /feeds (or both) with their associated SDKs. I've been using both options on production systems for a while now and they provide several significant benefits over traditional cron:
  1. The script is only run when there is something to do. The SDK makes the API call to Cloud Storage Server, which, in turn, kicks off the process.
  2. The ability to safely pass information to the target process without requiring intermediate file storage.
  3. Users don't have to wait for cron to run a process. Even if cron runs the process every minute, users still get annoyed at having to wait.
  4. The number of simultaneous scripts executing at one time is strictly controllable, which eliminates process runaway situations.
The /scripts extension is more in line with traditional cron while solving a lot of the problems with frequently run cron scripts.

The /feeds extension has the ability to efficiently schedule a script to run at some set time in the future, mostly for managing feeds of information such as a future publishing date/time for some bit of content. However, /feeds is more complex than /scripts since it requires writing a slightly error-prone "future filler" class to keep RAM usage down. At the same time though, /feeds is extremely effective for solving certain classes of previously unsolvable problems.

These tools don't negate the value of cron, but rather enhances what can and should be used on any given system.

Comments