Performance Profiling Guide

Optimization without measurement is superstition with a diff attached. This skill runs a disciplined performance investigation: reproduce the slowness, measure a baseline, profile to find where the time actually goes, change one thing, and prove the win with the same measurement — keeping receipts at every step so the conclusion survives skeptical review.

When to use this skill

A user-facing operation is slow and nobody has measured why
A batch job, build, or test suite has crept from minutes into tens of minutes
Someone proposes an optimization and you need to know whether it is worth the complexity
After an optimization lands, to verify it helped anywhere other than the author's machine
Capacity planning: finding the ceiling before traffic finds it for you

Workflow

Define the metric and the target. "Slow" becomes "p95 of this endpoint is 2.4s; target is under 800ms," or "the build takes 14 minutes; target 5." No target, no finish line.
Reproduce on demand. Build a harness that triggers the slow path with realistic data volume and shape. Tiny synthetic inputs hide superlinear behavior — if production sorts 100k rows, the harness sorts 100k rows.
Measure the baseline honestly: several runs, warm and cold noted, median and spread recorded. One run is an anecdote, not a baseline.
Profile before hypothesizing. Use the runtime's sampling profiler, the database's query analyzer, or distributed traces to get an attribution of where time goes. Read it top-down: the top three frames or spans usually account for most of the wall time.
Identify which budget is being spent — CPU, I/O wait, lock contention, or allocation and collector pressure — because the fix for each is different, and a CPU fix applied to an I/O problem is pure risk with no reward.
Form one hypothesis and predict its magnitude before touching code: "the query inside the loop costs ~1.4s; batching it should halve p95." A hypothesis without a predicted size is unfalsifiable and will always feel confirmed.
Change one thing. Re-run the same harness — same data, same run count — and compare against baseline. Keep the change if the prediction held; revert it and write down what you learned if it did not. Reverted experiments are progress too.
Repeat until the target is met, then stop. The next hotspot after target is somebody's future project, not today's scope creep.
Guard the win: wire the harness number into CI or a dashboard with an alert threshold, so the regression arrives with a name attached instead of a mystery.

Output format

## Perf investigation: <operation>

Baseline: <median, p95, n runs, environment>
Profile: <top 3 time sinks with %>
Hypothesis: <cause> → predicted saving <x>
Change: <one line + reference to the diff>
Result: <same metrics after> (delta: <y>)
Verdict: kept | reverted — <one sentence>
Regression guard: <check added>

One block per iteration; the chain of blocks is the report.

Guardrails

Never optimize unprofiled code, however guilty it looks — the obvious suspect is innocent about half the time
Do not benchmark on a loaded laptop and call it science; pin the environment, or compare only within one session
Micro-benchmarks lie about caches, JIT warmup, and I/O; the end-to-end harness delivers the verdict
Reject any optimization that trades a measured second for unmeasured correctness risk without a test covering the risky path
Report medians and tails, never bare means — a flat average can hide a bimodal disaster
Check algorithmic complexity and data growth before shaving constant factors; big-O beats micro-tuning at any realistic scale

Quick triage table

Symptom	First suspect	First tool
Slow only at scale	superlinear algorithm, per-row I/O	complexity check, query log
Slow always, CPU pegged	hot loop, serialization	sampling profiler
Slow always, CPU idle	blocking I/O, lock waits	trace view, lock statistics
Slow after running a while	leak, collector pressure, cache eviction	heap profile over time
Fast locally, slow deployed	data volume, network hops, cold caches	measure in the real environment