Diagnosing Bugs - A Systematic Diagnostic Process for Difficult Bugs and Performance Regressions

Skill Overview

Diagnosing Bugs is a diagnostic discipline for difficult bugs and performance regressions: it requires first establishing a feedback loop that can reliably reproduce the issue and clearly flag failure, and only then proposing hypotheses, adding instrumentation for validation, and implementing fixes—thereby avoiding attempts to guess the cause by reading code alone.

Applicable Scenarios

1. The user says “help me diagnose this” or “debug this,” or reports an error, exception, failure, or slowdown: The skill first requires establishing a command that can flag this specific symptom before proceeding to analysis.
2. Intermittent production bugs that cannot be reproduced locally: The skill provides methods for increasing reproducibility—repeat the triggering action hundreds of times, parallelize it, add load, narrow the time window, and inject sleeps, turning a 1% intermittent occurrence into a 50% reproduction rate that can be debugged.
3. Locating performance regressions: These issues are not solved through logs. First establish baseline measurements (timing scripts, profilers, query plans), then perform a binary search; measure before fixing.

Core Functions

1. Build and tighten the feedback loop: Provide ten reproduction methods in priority order—failing tests, curl/HTTP scripts, CLI calls with fixed inputs, headless browser scripts, trace replay, a minimal standalone runnable environment, property-based/fuzz testing, binary search, old-versus-new version differentials, and, as a last resort, human-in-the-loop scripts. After establishing the loop, continue tightening it: make it faster, more precise in its signal, and more deterministic.
2. Reproduce, minimize, and prioritize hypotheses: Confirm that the loop flags the symptom described by the user rather than an unrelated failure, then minimize the reproduction by removing one element at a time. Before testing any hypothesis, list 3–5 falsifiable hypotheses and provide predictions for each, then show the prioritized results to the user for review.
3. Fix, regression-test, and wrap up: Write the regression test before changing the code, but only when there is a “correct testing seam”—a seam that can reproduce the real call path. If only a shallow seam exists, the skill explicitly requires reporting “lack of a correct testing seam” as a conclusion rather than writing a test that creates false confidence. The wrap-up checklist includes: the original reproduction scenario no longer occurs, the regression test passes, all debugging instrumentation has been removed, temporary prototypes have been deleted, and the final validated hypothesis has been recorded in the commit message.

Common Questions

Why must the feedback loop be built first instead of reading the code directly to find the cause?

The skill treats this as a hard gate: if there is a tight pass/fail signal that flags the bug, binary search, hypothesis validation, and instrumentation become mechanical execution; without one, no amount of staring at the code will help. Therefore, “without a command that can flag failure, do not proceed to the next stage.”

What should I do when a bug cannot be reproduced at all?

The skill requires stopping and stating this explicitly rather than guessing. List the methods already attempted, then ask the user for one of three things: access to an environment where the issue can be reproduced, a sanitized artifact from the incident (HAR file, logs, core dump, or timestamped screen recording), or permission to add temporary instrumentation in production.

Are logs and credentials safe during debugging?

The skill mandates sanitization: all secrets in commands, output, and incident artifacts must be replaced with <REDACTED>. Credentials must be passed through environment variables so they remain in the environment rather than appearing in displayed content. Captured artifacts often contain authentication headers; quote only the lines that carry relevant signals. If the sanitized information is insufficient to locate the issue, the skill requires stating this directly and asking the user for more information.

Does it support intermittent (nondeterministic) bugs?

Yes, but the goal is different: it does not seek one clean reproduction, but rather a higher reproduction rate. Repeat the trigger 100 times within a loop, parallelize it, add load, and narrow the timing window until the reproduction rate is high enough to debug before proceeding.

What should I do if I cannot find a suitable regression-testing seam?

When the seam is too shallow—for example, the bug requires multiple callers but only a single-caller unit test exists—the resulting test will create false confidence. The skill requires recording and reporting “lack of a correct testing seam” as a finding from the diagnosis. This itself indicates that the code architecture is obstructing efforts to lock down the bug.

How does this differ from an ordinary debugging process?

The difference lies in the order and discipline: spend a disproportionate amount of time building the loop first, then reproduce and minimize, and only afterward are hypotheses allowed. Each hypothesis must first be written as a falsifiable prediction (“If X is the cause, then changing Y will make the bug disappear”), with only one variable changed at a time. Instrumentation logs must include a unique prefix (such as [DEBUG-a4f2]) so they can all be removed cleanly with a single grep during wrap-up.