LearnAI home

Keeping It Working ยท Lesson 5

What to Do When Automation Goes Wrong

How to notice a broken routine before it quietly costs you.

This is the lesson people skip

You have set something up, it works, and the temptation is to stop thinking about it. That is the entire point of automating, after all. Skip this lesson and the ending writes itself: in about four months you will discover that a routine you trusted has been wrong since March.

Automation does not usually fail with a bang. It fails politely, in a way designed not to attract your attention.

In plain English

Silent failure:
The routine stops working, or starts working on the wrong assumptions, and nothing tells you.
Stale assumption:
An instruction that was true when you wrote it and quietly stopped being true afterwards.
Human review step:
A deliberate pause where a person reads the output before anyone acts on it.
Off switch:
Knowing exactly how to stop a routine, before you urgently need to.

The three ways it goes quiet

It just stops. A file gets renamed, a permission changes, a workspace gets tidied up by someone helpful. The routine fails and you notice nothing, because the absence of a Monday summary looks identical to a Monday when you were too busy to read it. Missing things are almost impossible to spot. Your brain does not raise an alarm about an email that did not arrive.

It keeps running on stale assumptions. Worse, and far more common. Your weekly chase list still references a spreadsheet that was superseded in April. Your client summary still describes a contact who left. Nothing errors. It produces a confident, well formatted, entirely obsolete document every single week, and you skim it and nod.

It drifts past its usefulness. Your job changed, the project ended, the client left. The routine still runs, still lands, and is now pure noise. Noise trains you to ignore the channel, which means the day something genuinely important appears there, you will scroll past it.

โŒ Weak prompt

Prompt

Every Monday, summarise the project tracker and send me the highlights.

Output

Highlights: onboarding progressing well, audit on track, three items pending.

Never says what it read or when it changed. Two months after the tracker is retired it produces the same reassuring paragraph.

โœ… Good prompt

Prompt

Every Monday, summarise the project tracker. Start with a HEALTH line: the file name you used, the date it was last modified, and how many items it contains. If it has not changed since last week, say NO CHANGES SINCE and give the date. If you cannot find it, reply with SOURCE MISSING and nothing else.

Output

HEALTH: delivery-tracker, last modified 14 March, 22 items. NO CHANGES SINCE 14 March...

The header does the noticing for you. A stale date or a missing source is visible in a two second glance.

Make the routine report on itself

You will not remember to audit your automations. Nobody does. So build the audit into the output, where you cannot miss it.

Prompt you can copy: add a health line to any routine

Add this to the top of every run, before anything else.

HEALTH LINE

  • Which sources you actually used, by name.
  • When each was last updated.
  • How many items or how many pages you read.
  • Whether anything changed since the previous run.

Then these rules. If a source is missing or unreadable, reply with SOURCE MISSING plus the name, and produce nothing else. If a source has not changed in 30 days, add STALE and the date. If you had to guess or infer anything, list it under ASSUMED. Never fill a gap silently.

Three lines at the top, and silent failure becomes loud failure. That is the single highest value change in this whole course.

Checkpoint

Silent failure is the real risk. Make every routine state what it read and when that source last changed, so a broken one is obvious at a glance.

Keep a human in the loop where it counts

Not everywhere. A weekly summary you read yourself already has a human step, which is you reading it. The place to be strict is anything that leaves your desk or that someone else will act on.

The rule of thumb: if the output goes to another person, is used to make a decision, or touches money, dates, commitments or a named individual, a human reads it first. That human is you, it takes thirty seconds, and it is the cheapest insurance available.

Be especially careful with anything that sends. A routine that drafts is recoverable. A routine that emails a client automatically will, on the day something goes wrong, do so with total confidence and perfect formatting. Draft by default, send by hand.

The monthly ten minutes

Once a month, on a date you will actually keep, look at your routines. Not the output, the routines themselves.

Prompt you can copy: the monthly automation review

Here are the recurring tasks I have set up, with what each produces. Take me through them one at a time and ask me these four questions.

  1. Did this actually run and land every time this month? Yes or no.
  2. Did I read it, or skim and ignore it? Be honest.
  3. Is every source it depends on still current, correctly named and still mine to access?
  4. Is the reason I set this up still true?

Then tell me which ones to keep as they are, which to rewrite, and which to switch off entirely. Recommend switching something off if I have admitted to ignoring it.

MY ROUTINES: [list them]

Question two is uncomfortable and important. A routine you skim is a routine you have already stopped trusting. Fix it or kill it, but do not let it keep landing.

Know the off switch before you need it

The last piece, and it takes two minutes. When you set up any routine, find out how to pause it, edit it and delete it, and try each one once while the stakes are zero. Wherever your scheduled or recurring tasks are listed you should be able to see what exists and stop any of it, though the exact wording and location vary and do move about.

The reason to rehearse it is that the day you need the off switch is a day when something is going wrong and you are already flustered. That is a bad moment to be searching a menu.

Prompt you can copy: the rollback note

For each routine I have set up, write me a short note I can keep.

WHAT IT DOES: one line. WHAT IT TOUCHES: the files, folders or people involved. WHEN IT RUNS: day and time. HOW TO STOP IT: where I go to pause or delete it. WHAT BREAKS IF IT STOPS: what I would have to do by hand instead. WHO ELSE SEES THE OUTPUT: names, or NOBODY.

Keep it under six lines each. This is a note for a bad Tuesday, not documentation.

Store that note in the same document as your routines themselves. One page, kept somewhere you actually open. If it lives in a folder you visit twice a year, it does not exist.

๐Ÿ“ Quiz

Question 1 of 4

What is the biggest risk with a routine you have set and forgotten?

Found this useful? Pass it on.