What infrastructure drift, frontend validation, and shared engineering patterns have in common.*
Somewhere in your infrastructure right now, a resource doesn't match what your Terraform state file claims.
I'd put money on it.
Maybe it's a firewall rule someone added through the portal during an incident at 2 a.m. and never ticketed. Maybe it's a tag that got renamed by a policy remediation task outside the pipeline.
Doesn't matter which.
The shape of the bug is always the same:
There is a declared source of truth, and there is an easier path around it. Eventually, someone takes the easier path.
I spent last month staring at the exact same bug in a codebase that has nothing to do with infrastructure.
A teammate and I built a generic field-validation engine for the React forms in one of our applications.
The idea was straightforward: a rule registry.
Each form declares a small configuration object:
We ended up with around ten rule types covering almost everything we needed:
A hook wires everything together:
Any page that needs field validation imports the hook, defines its rule configuration, and it works.
It's a good pattern.
I'd build it again.
The whole point was to stop every page from reinventing its own bespoke regex checks scattered throughout the codebase — each slightly different, each with its own bugs.
Except there is one problem.
The escape hatch that makes the engine flexible is the same door that lets it drift.
Nothing stops a developer, six months from now, in the middle of a sprint, from writing a raw regex check directly inside a field's onChange handler instead of routing it through the rule registry.
It compiles.
It works.
For that one field, in that one moment.
Nobody's build breaks.
QA doesn't flag it because the field still validates correctly from the user's point of view.
But something important has happened.
There is now a validation rule living outside the registry.
The registry doesn't know about it.
The next developer won't find it when they look at the shared configuration to understand what's enforced and where.
It won't automatically get the live-sanitization behavior or the shared error-display wiring.
The declared source of truth — "these are the validation rules for this application" — quietly stopped being true for one field.
Nothing announced it.
That's drift.
Not a compile error.
Not a test failure.
A fact that becomes false without anyone deciding it should.
Terraform drift works the same way, mechanically, even though the domain couldn't be more different.
You declare desired state in .tf files.
Someone with portal access — an on-call engineer under pressure, or someone doing "just this one quick thing" — changes a resource directly.
Your infrastructure has now diverged from its declared configuration.
In a clean environment, terraform plan should show zero unexpected differences.
The moment it doesn't, your declared infrastructure state no longer completely describes what is actually running.
And just like the validation engine, the manual portal edit usually works fine in isolation.
Nothing breaks that day.
It breaks three months later.
Someone runs an apply based on stale assumptions, and the plan output doesn't match what's actually running.
Or worse, the apply "corrects" a change someone made deliberately — and nobody remembers why that change was made anymore.
That's the dangerous part of drift.
The original change wasn't necessarily wrong.
The problem is that the system lost track of the fact that the change existed.
Here's the part I think most teams get wrong about drift, in both the infrastructure sense and the code sense:
They try to solve it with process.
Code review.
A wiki page that says:
"Please use the shared validation hook."
A runbook that says:
"Please don't edit resources manually. Always go through the pipeline."
But review didn't catch the inline regex.
It looked like a normal, working field check.
Nobody reviewing that PR was necessarily thinking about the registry's completeness as a system property.
The runbook doesn't stop the 2 a.m. portal edit either.
In the moment, going through a slower deployment pipeline felt like the wrong call to the person holding the pager.
That's understandable.
And that's exactly why the process fails.
Process asks people to consistently make the right choice under pressure.
Engineering should make the right choice easier — and the wrong choice visible.
Process catches drift only when someone happens to be looking at the right diff at the right time.
That's luck with a policy document stapled to it.
What actually works is either removing the choice altogether or, at minimum, making drift loud and automatic instead of quiet and manual.
For infrastructure, that means scheduled drift detection.
For example:
1terraform plan -detailed-exitcodeRun it on a schedule against every managed environment, not just before a deliberate deployment.
The important part isn't simply running the command.
The important part is what happens when it detects a difference.
A non-zero drift result shouldn't become a dashboard tile that nobody opens.
It should trigger an actionable workflow.
Depending on the environment, that might mean:
The goal isn't to prevent humans from ever making mistakes.
The goal is to reduce the time between drift occurring and someone knowing about it.
For our validation engine, the fix was considerably less glamorous.
A lint rule.
We flag an onChange handler in a form component when it contains a raw regex literal.
The reasoning is simple:
If you're writing pattern-matching logic directly inside a form component, you've probably reinvented something the shared validation registry already does.
That doesn't mean every regex inside a component is inherently wrong.
It means bypassing the shared mechanism should be deliberate.
The lint rule makes the bypass visible.
And that's the important part.
Neither fix is complete.
The lint rule can be silenced with an inline comment.
A developer can still find ways around infrastructure controls.
Terraform itself has mechanisms such as targeted operations that can be used to bypass parts of a normal workflow.
But that's not necessarily a failure.
The escape hatch exists for a reason.
Sometimes you genuinely need it.
The mistake is making the escape hatch completely frictionless.
If bypassing the shared path costs nothing, people will eventually bypass it whenever the local problem feels easier to solve that way.
The useful middle ground is:
Make friction proportional to how much a bypass should hurt to justify.
Total lockdown kills the flexibility that made the registry or pipeline worth building in the first place.
Total permissiveness turns "declared state" into a suggestion.
The engineering sweet spot is somewhere between the two.
This is the part I think is worth taking beyond Terraform.
If your team runs drift detection on infrastructure but nowhere else that has this shape:
A declared shared source of truth + a legitimate escape hatch
then you're only watching for the bug in the one place where you've already been burned by it.
Look around your systems.
Is there a central feature-flag system, but developers can quietly introduce local configuration switches?
Is configuration supposed to come from a centralized registry, but individual services maintain their own hidden defaults?
Is there a central permissions model, but individual endpoints implement additional authorization logic outside it?
Is there a shared validation framework, but components can bypass it with custom checks?
Is there a canonical schema, but services add undocumented assumptions that aren't represented in it?
Is the pipeline the source of truth, but someone occasionally changes production configuration manually?
These are all variations of the same problem.
The technology changes.
The failure mode doesn't.
The pattern is worth naming because once you recognize it, you start seeing it everywhere.
You have:
1. A declared source of truth
Something says:
"This is how the system is supposed to behave."
2. A convenient escape hatch
Something allows someone to say:
"I'll just change this one thing directly."
3. No automatic reconciliation
Nothing checks whether reality still matches the declaration.
4. Delayed consequences
The change works today.
The problem appears weeks or months later.
That's drift.
And drift isn't fundamentally a Terraform problem.
Terraform just happens to make the problem particularly visible.
The strongest systems aren't necessarily the ones that prevent every deviation.
They're the ones that make deviations observable.
A developer might still need a custom validation rule.
An operator might still need to make an emergency infrastructure change.
A production incident might still require bypassing the normal deployment path.
That's reality.
The architectural question is:
What happens after the bypass?
Does the system know it happened?
Does someone get notified?
Does the change become part of the declared source of truth?
Is there an explicit reconciliation step?
Or does the system simply move on and hope everyone remembers?
That's the difference between a controlled escape hatch and silent drift.
Here's a simple test you can apply to your architecture.
Find a system with a central source of truth.
Then ask:
"What happens if someone bypasses it at 2 a.m.?"
If the answer is:
"They shouldn't do that."
you don't have a control.
You have a policy.
If the answer is:
"The system will detect the deviation, make it visible, and force reconciliation,"
you have an engineering control.
That's a much stronger place to be.
Because people will take the fast path.
Especially when they're under pressure.
The goal isn't to pretend they won't.
The goal is to make sure the fast path doesn't quietly become the new source of truth.
Go find the other places in your system where declared state can quietly become fiction.
Before they find you at 2 a.m.