How to Find the Weakest Link in a Multi-Step AI Agent Chain Before One Broken Step Silently Ships Garbage
Table of Contents
A few months ago, in Automated AI Content Doesn't Have to Be Junk: Three Quality Gates in Practice, I held up this self-review-and-rewrite quality gate as the signature mechanism that took our output from unreadable to better than what I write myself. This post goes back and exposes it: once that gate's parser broke, it silently let through the very draft it had just judged bad, and published a 14-word junk post.
Chapter 1 of Agentic Design Patterns covers Prompt Chaining. The core idea in one sentence: instead of handing a complex task to the model to do all at once, split it into a chain of sub-steps, where the output of one step feeds the next. One step researches, one generates, one self-reviews, one publishes, each with its own job, and the whole chain looks clean and reliable.
Our Moltbook auto-posting is exactly this kind of chain: first research (pulling industry news, Hacker News front-page items and the performance of our own past posts as material), then generate a post, then self-review and score it, and finally publish. All four steps were in place and we were pleased with it. Then we ran an adversarial audit on ourselves. The truth of this chapter: a chain that looks complete is not necessarily reliable. It is only as strong as the error handling in its weakest step. And our weakest step silently published a 14-word junk post.
Step three broke, and the garbage still ran the rest of the chain
The self-review step (we call it the quality gate) works like this: the generated draft goes back to the model to be scored from 1 to 10. A score of 7 or above is APPROVED and passes; below 7, the model replies with REWRITE plus a rewritten version. It sounds complete, a step that corrects itself.
The problem was in parsing that rewritten version. The code pulls a new title and new body out of the model's reply. If it finds them, it swaps them in. If it does not, it falls into the else branch:
if [ -n "$NEW_TITLE" ] && [ -n "$NEW_CONTENT" ]; then
TITLE="$NEW_TITLE"; CONTENT="$NEW_CONTENT" # use the rewritten version
else
log "Quality gate: REWRITE requested but failed to parse, using original"
fi
Whenever the model's rewrite drifts from the format (not following TITLE: plus ---), this block fails to parse and falls back to the original draft. And the original draft that gets a REWRITE verdict is, by definition, the one that was not good enough. The self-review step found the draft bad, asked for a rewrite, the rewrite failed to parse, and so it passed the draft it had just judged bad to the next step, untouched. Worse, the next step (publishing) had no length check at all at the time. So a post of only 14 words went all the way through self-review, all the way through publishing, and quietly went live on a public platform.
The self-review step's stated job was to stop bad drafts. Its error path was "if you cannot stop it, let the original through". The chain looked self-correcting, but the error handling in its weakest step was a silent pass.
First spend a few minutes checking your own chain
01 Is each step's output validated before the next step uses it Draw your chain and find every point that parses the previous step's output (pulling JSON, splitting a string, extracting a marked section). Every parse point should be followed immediately by a check: is the parsed result empty, is the format right. Using it downstream without a check means treating something unvalidated as reliable input. Empty values are not the only thing to catch: when an LLM call fails it returns an error string, and our research chain once wrote LLM error messages into its notes as analysis.
# List every parse point in the chain and see whether a check follows it
grep -nE "sed .*1,/\^---/d|grep \"\^TITLE:|jq -r|解析|parse" your-chain.sh
Red flag: the parsed value goes straight into the next step, with no if in between checking whether it is empty or correctly formatted.
02 When a step fails to parse, does it stop or silently pass
Go through the failure branch of every parse point, one by one. If the failure handling is "carry on with the previous value", "fall back to the original" or "swallow it with || true", garbage is being passed downstream without anyone calling a halt. We got caught by a single using original: it is not a fix, it is a pass. The first step of the same chain fell the same way: 2>/dev/null || true swallowed the error, and the research step silently came back empty for months, waved through every time as "no news today".
# Find silent fallbacks of the "revert to the original value on failure / swallow the error and continue" kind
grep -niE "using original|fallback|failed to parse|回原|退回|\|\| true" your-chain.sh
Red flag: the failure handling is "continue with the original value", and nobody guarantees that original value is any good.
03 Is there a hard min-length or quality gate before publishing
However loose the earlier steps are, there should be one final deterministic gate before anything goes out. Find the actual publish action (a curl POST, an API write, a schedule) and check whether a "stop if too short or too thin" gate stands in front of it.
# Is there a length or quality gate before the publish action
grep -nE "wc -w|wc -m|MIN_WORDS|-lt .*MIN|too short|too thin|太短|太薄" your-chain.sh
Red flag: between generation and actually sending, you cannot find a single "exit if it fails" gate.
How we actually fixed it
The fix on 2026-07-04 went in front of publishing: a min-length gate. If the body has fewer than 40 words, the run stops, logs too thin and does exit 0, and nothing is posted. Skipping one run does not hurt, since the next scheduled run generates a new post. Publishing a 14-word throwaway post to a public platform is what is actually embarrassing.
To be clear: we did not pretend to fix that upstream parser to perfection. The rewrite still occasionally drifts from the format and fails to parse. What we did was accept that the weakest step in the chain will fail, then guard the last gate before the end so that even when it fails, it cannot send garbage out. That is the real point of prompt chaining: you cannot stop every step from ever failing, but you can guarantee that a failure does not silently flow to the end.
Incidentally, our other posting chain, content-cascade, got this right: every generated post is validated once before it is scheduled. If the model replies with a cop-out line like "Please provide the article content", the post is dropped, and anything under 80 characters is dropped too. That is the current gate, stricter than the version in the post that first introduced this chain, which only dropped posts under 50 characters. Both are chains. The only difference is whether someone is guarding the error handling in the weakest step.
Prompt chain checkup checklist
Run this against your own multi-step chain:
- Is each step's output validated (non-empty, correct format) before the next step uses it
- Is every point that parses the previous step's output followed immediately by a check
- When a step fails to parse, does it stop and call a halt, or silently fall back to the original value and pass it downstream
- Is there a deterministic min-length or quality gate standing in front of publishing
- When that gate blocks something, does it skip safely (regenerate next run), or force a half-finished piece out anyway
- Can your logs tell apart "this run was blocked by the gate" from "this run posted normally"
Four takeaways from this chapter
- A chain is only as strong as the error handling in its weakest step. Having all four steps in place does not mean all four hold.
- The most dangerous thing is not a step throwing an error. It is a step that fails to parse, silently falls back to the original value, and passes garbage downstream as if nothing happened.
- Every point that parses the previous step's output needs a check right after it. Using it downstream without validation means treating something unvalidated as reliable input.
- You cannot stop every step from ever failing, but you can put a deterministic gate before the end and guarantee that a failure does not silently flow onto a public platform.
Source locations: ~/.openclaw/scripts/moltbook-autopost.sh (the research → generate → self-review → publish chain; the self-review quality gate is in the REVIEW_PROMPT section, using original is the silent fallback; the min-length gate MIN_WORDS=40 added on 2026-07-04 sits in front of publishing), content-cascade-multi.sh (clean_post plus the quality/length validation before each post is scheduled, the control group where someone is guarding the weakest step).
This is part of the Agentic Design Patterns × Lobster Fleet series. Following the book, we systematise a solo company's AI agent fleet, then run an adversarial audit on ourselves. Every chapter we claim to have implemented gets verified again, and the investigation and the fix are written up as steps you can run. The credibility of this series comes from our willingness to publish our own failures.
FAQ
What is prompt chaining?
It is the pattern covered in Chapter 1 of Agentic Design Patterns: instead of handing a complex task to the model to do all at once, split it into a chain of sub-steps, where the output of one step feeds the next. For example, one step researches, one generates, one self-reviews and one publishes, each with its own job.
How do you find the weakest link in a multi-step AI agent chain?
Draw your chain and find every point that parses the previous step's output (pulling JSON, splitting a string, extracting a marked section). Every parse point should be followed immediately by a check: is the parsed result empty, is the format right. Then go through the failure branch of every parse point and see whether it stops or silently passes.
Why is falling back to the original draft on a parse failure dangerous?
Because the original draft that gets a REWRITE verdict is, by definition, the one that was not good enough. Our self-review step found a draft bad, asked for a rewrite, the rewrite failed to parse, and it passed the draft it had just judged bad to the next step, untouched, which published a 14-word junk post. Carrying on with the previous value, falling back to the original or swallowing the error with || true all pass garbage downstream without calling a halt.
How do you stop an AI auto-posting pipeline from publishing posts that are too short?
Put one deterministic gate in front of the actual publish action (a curl POST, an API write, a schedule). Ours stops the run if the body has fewer than 40 words, logs too thin and does exit 0, so nothing is posted. Skipping one run does not hurt, since the next scheduled run generates a new post.
What if you cannot fix every step in the chain to perfection?
Accept that the weakest step in the chain will fail, then guard the last gate before the end. You cannot stop every step from ever failing, but you can put a deterministic gate before the end and guarantee that a failure does not silently flow onto a public platform.