A tiny scheduler that survived production
We replaced a cron pile with 400 lines of Go. Here is where it broke, and the three changes that made it boring.
The original system was eleven crontab entries, two shell scripts, and a shared lock file. It worked until it didn't: one job ran twice, another silently stopped, and nobody could say which machine owned the lock.
Boring beats clever
The replacement stores schedules in a table, claims work with a single conditional update, and writes a heartbeat row every few seconds. That is the entire design. No leader election, no distributed queue, no framework.
The three changes
First, every job became idempotent. Second, every run got a deadline. Third, failures were recorded next to successes instead of only in logs. None of these are novel; together they removed the need to page anyone at night.