There’s a lot of noise about AI-assisted development right now. Not much of it comes from people actually building production systems with it. Here’s what a year of using Claude day-to-day on real client work has taught us: the wins, the near-misses, and the parts we’ve learned to be paranoid about.

Where AI actually saves us time

The clearest wins are on scaffolding: the boilerplate that has to be right but isn’t the interesting part of the problem.

I set up a multi-agent orchestration pipeline: dozens of agents wired across parallel groups, dynamic routing, and looped child workflows, each with its own timeout, retry budget, and failure handling. Getting that structure right from scratch would have been two days of reading Temporal docs and debugging replay behaviour. Claude got the scaffolding right quickly, and I spent my time tuning the agent logic instead.

The activity options, retry policies, child workflow wiring: that would have taken most of the time, and it’s the least interesting part of the problem. If Claude did nothing else valuable, this alone would justify it.

The near-miss that changed how I work

Not every suggestion is right. And the wrong ones sound just as confident as the good ones.

The example I keep coming back to: Claude suggested reading from in-memory pipeline state (accumulated prior-agent outputs) for a value that should have come from the database instead. Looks fine on the surface.

But two agents running in parallel share that in-memory state. Agent B’s output can bleed into agent A’s context mid-run, and agent A produces its output based on the contaminated input. The result persisted and labelled as agent A’s, which is accurate to what agent A said, but wrong, because what agent A saw was already corrupted. Nothing downstream would catch it unless you knew to check for cross-agent bleed.

I caught it. Corrected it. And immediately wrote it into the project instructions file so the next session started already knowing the rule. That’s the real workflow. Not “Claude wrote it, I merged it”. It’s more like reviewing code from a very fast, very confident junior engineer who has no memory of any previous project decisions.

How we handle the fact that AI forgets

The model doesn’t carry learned mistakes across sessions. That’s a real problem when you’re building anything non-trivial with it.

So we maintain a CLAUDE.md file checked into the repo. Every rule I’ve had to correct twice gets written in there as a permanent constraint. A couple of examples that are in there right now:

• Always read from the database for audit inputs, never from pipeline state.
• Parallel activities must opt out of shared context or they’ll contaminate each other.

It’s basically converting conversation corrections into project documentation. The model reads it at the start of every session so I don’t have to repeat myself.

Do I write more code now, or less?

More, but faster.

The system I’m currently building orchestrated dozens of AI agents, Snowflake integration, a Temporal workflow layer, and a FastAPI layer. I wouldn’t have attempted that scope in this timeframe. What changed is where the time goes: less on boilerplate, more on whether the design is actually right.

The real risk is that AI makes ambitious scope feel cheap. The first 80% goes fast. It’s easy to commit to something large because the initial velocity is intoxicating. But the last 20% is still all on you, and if you’ve overcommitted, that’s when things fall over.

Knowing if an AI output is actually correct, not just plausible

Building AI-powered systems is a different problem from using AI as a development tool. It’s not just “did Claude give me good code”. It’s “is Claude giving good outputs in production, to a real user?”

The trick we’ve settled on: put constraints in code, not in prompts.

Structured output schemas reject anything that doesn’t match, like a form that won’t submit unless every field is valid and the right type. For some agents, the list of valid answers is pulled from the live database at runtime, so the model literally cannot hallucinate an option that doesn’t exist without triggering an automatic retry.

But there’s a ceiling. Schema validation catches structural errors, not factual ones. If the model confidently picks the wrong valid answer, that’s harder to catch. We’ve designed an evaluator agent as the intended last line of defence for exactly this.

How far bad outputs travel before something catches them

A sequential agent in the chain receives all prior outputs in a shared context. A bad label from agent 1 flows into agent 2, then 3. Parallel agents are deliberately excluded from that shared context (that’s the fix from the near-miss above), so they can’t contaminate each other, but they also don’t get the benefit of seeing upstream corrections.

The evaluator agent is designed to catch it at the end. So if the evaluator misses it, the bad output ships.

In practice, the most reliable catch is earlier: the output schemas doing form-validation-style rejection. If a model returns something structurally wrong (wrong field, invalid value), it gets re-prompted automatically up to four times before the schema rejects it entirely.

What we still don’t give AI

Data contract decisions. Which table, which column, what constitutes the source of truth. The AI doesn’t have live system context and it’ll confidently pick a real, plausible-looking table that’s simply the wrong one for the job. Nothing in the code stops it, because the query still runs and returns an answer.

Anything where being wrong is hard to reverse, and the model can’t see the actual system state it’s making decisions about — those decisions stay with humans.

If you’re new to this: one habit to build, one to watch

Build: put constraints in code, not just in prompts. A structured output schema forces a retry when the model gets it wrong. A prompt instruction can be quietly ignored with no signal that it happened.

Watch out for: don’t let AI make architecture decisions for systems it can’t see. It’ll give you a confident answer based on what’s plausible in general, not what’s true about your specific setup.

Thinking about AI in your development workflow?

If you’re building AI-powered systems or figuring out how AI-assisted development actually fits into your team’s workflow, we’re happy to compare notes. Get in touch.