Core scenario
Situation: Latency rises on a live service.
How to apply: Check metrics and recent changes, then test one reversible mitigation.
Outcome: Watch errors and latency before expanding or rolling back.
Operational thinking
Learning guide and study notes
How will this work reliably now? Use it for live execution and feedback.
What you will learn
- Separate observed signals from the interpretation of an incident.
- Define a safe next action and the signal for stopping it.
What is it?
Operational thinking turns goals into reliable live execution under real-time constraints—separating raw telemetry from interpretation, bounding blast radius, and verifying feedback loops before scaling an action.
Operational thinking turns a goal into observable action under current constraints. The OODA loop—Observe, Orient, Decide, Act—is one useful cycle, not a command to act without understanding.
Tools and techniques
Key techniques
- OODA cycle (Observe–Orient–Decide–Act)Separate raw telemetry from situational orientation before executing a bounded live change.
- Blast-radius guardrail and rollbackDefine scope limits, single owner, and a pre-tested rollback condition before acting.
- Closed-loop feedback checkVerify whether the mitigation moved live error and latency signals before scaling the response.
Procedure: think it through step by step
Work through these steps in order, applying the tools above. Write down the answer to each step before moving on.
- Observe and orient
Gather current signals and separate observations from interpretations. [1] Source: Air University — Skill Development Using the OODA Loop (opens in a new tab)
- Decide and act
Choose a bounded response with an owner, safety boundary and rollback path. [1] Source: Air University — Skill Development Using the OODA Loop (opens in a new tab)
- Review the feedback
Measure the effect and repeat the cycle as conditions change. [1] Source: Air University — Skill Development Using the OODA Loop (opens in a new tab)
Try it now
For one live issue, write observed facts, your interpretation, a bounded next action and the signal for stopping it.
Visual guide
Read the workflow as text
- SituationLatency rises on a live service.
- Step 1Observe and orient
- Step 2Decide and act
- Step 3Review the feedback
- OutputWatch errors and latency before expanding or rolling back.
Read the mind map as text
Operational thinking
Current signals
Operating constraints
Owner and response
Measured feedback
Detailed examples
02 / Draining a degraded availability zone during peak load
Situation: Packet loss spikes in one zone during peak traffic, and engineers consider shifting 100% of traffic away immediately.
How to apply: Check remaining capacity in healthy zones first, then shift traffic in 20% increments while watching saturation metrics.
Outcome: Restore user success rates without overloading the healthy zones into a total blackout.
03 / Running a high-risk database maintenance change
Situation: A production table requires an index rebuild that could lock writes if run unthrottled.
How to apply: Define a single command owner, set lock-wait timeouts, test abort behavior on a replica, and watch live lock queues during execution.
Outcome: Complete the maintenance online with zero customer-facing write timeouts.
Illustrative examples showing how the lens changes decisions; not measured outcomes.
Practice and keep learning
Watch out: Speed without orientation can turn a wrong diagnosis into a faster outage.
Getting better at it
- 01 · Learn the foundationNever execute a live change without answering three questions first: “How will we know it worked, how will we know it hurt, and how do we undo it in 60 seconds?”
- 02 · Build daily fluencyDuring incidents, keep a shared log that explicitly separates observed facts (“error rate 8% in zone B”) from hypotheses (“maybe storage is full”).
- 03 · Master under pressureChange one variable at a time under clear ownership, wait for telemetry confirmation, and halt immediately if the rollback threshold is crossed.
Combine this lens with others rather than relying on it alone. Browse all thinkings
Review and make it your own
The idea to remember
Reliable execution is a feedback cycle: observe conditions, choose a bounded action, and check its effect.
An outline for your notes
Use a problem of your own. Write a short answer to each prompt, then return after trying the method.
- Current state: record facts, impact, and the working explanation.
- Action: name an owner, boundary, and rollback path.
- Feedback: record the expected signal and when to reassess.
Check your understanding
Answer from memory before opening the suggested answer.
Why can immediate action make an incident worse?
Compare with a suggested answer
Without orientation, the response may target the wrong mechanism or introduce new changes. Use current evidence and bounded, reversible interventions.
What closes the execution loop?
Compare with a suggested answer
Observing the actual effect and comparing it with the intended effect, then deciding whether to continue, adjust, or roll back.
Sources
These sources support the method; the examples and exercises are original teaching illustrations.
- Air University — Skill Development Using the OODA LoopObserve, orient, decide and act as a repeating cycle.
- NIST — Risk Management FrameworkControl, monitoring and reassessment context.