Findings · verified by execution

Same rows. Faster.
Or the doctor won't sign.

Dropping an unused LEFT JOIN made our hero query 2.11× faster on Zetaris across 10 paired runs — unless the joined key has duplicates; then the same edit quietly changes the count. Query Doctor executes both versions on the engine before anyone merges and signs only when the rows match. Rewrites shipped with changed rows: 0.

Source on GitHub Watch the demo (2:50) Pitch deck See the recorded run
docker build -t querydoc . && docker run --rm -p 8000:8000 -e SAMPLE_MODE=true querydoc

runs keyless in sample modeexecutes both queriesa model never decides

Case report hero-r1

recorded live · Zetaris

Specimen · the rewrite drops the struck line

SELECT COUNT(1) AS n
FROM   TPCDS_DB.store_sales ss
LEFT JOIN TPCDS_DB.promotion p ON ss.ss_promo_sk = p.p_promo_sk
WHERE  ss.ss_customer_sk IS NULL

The key is unique, so the join cannot change the count.

Results vs reference · per-side medians of 3 alternating pairs

2.1×live verdict · median pair ratio · 3/3 pairs faster · saves 16.9 s (median pair saving)

original
H
29.3 s rewrite
13.2 s
original87cd00c5rewrite87cd00c5
ROW-IDENTICAL · 87CD00C5
Verified by Verifier — code, no model · recorded live on Zetaris

recorded/live/hero-r1/case.json

2.11×

hero speedup on Zetaris, bench

median of 10 per-pair ratios, 10/10 faster, same row (p50 ÷ p50 = 1.97×); the live verdict was 2.1× over 3 pairs · DEMO.md (bench), recorded/live/hero-r1 (live verdict)

0

rewrites shipped with changed rows

an ACCEPT needs an identical row fingerprint · evals/results.md

6 / 15

baseline rewrites that changed rows

one plain model call, same Verifier, DuckDB sample pack · evals/results.md

447

tests, no keys needed

34 files · npm test

Report · hero-r1, shown twice

The report a reviewer signs, and the trace behind it.

Left, the app's Report screen replaying the recorded live run. Right, the same run's agent trace, line by line: the Verifier runs the original twice, the Diagnoser reads the plan and the key, the Rewriter proposes, the Verifier times three alternating pairs and signs.

scroll the screen sideways →

localhost:8000/#/recorded/hero-r1
Query Doctor Report screen for the recorded live run hero-r1: Case report — Accepted, 2.1× median over 3 alternating pairs, saves 16.9 s, a red runtime bar of 29.3 s above a green bar of 13.2 s, a green stamp reading ROW-IDENTICAL · 87CD00C5, and the specimen SQL.
Real screen · npm start → #/recorded/hero-r1 · evidence from the recorded run, not a new execution

logs/live-hero-r1.jsonl · 21 events · 12:58:44 → 13:03:09 UTC
re-set from the recorded JSONL · real timestamps · plays about 50× faster

  1. 12:58:44Orchestratorstart_caseengine zetaris; llm nim; attempt cap 3
  2. 12:59:09Verifierexecute1 row(s) in 24460.9 mszetaris:run_sql
  3. 12:59:46Verifierexecute1 row(s) in 37452.7 mszetaris:run_sql
  4. 12:59:46Verifiercritiquedeterministic, 1 row(s), multiset compare, fingerprint rows 87cd00c5
  5. 12:59:56Orchestratorexplain12 plan node(s)zetaris:explain
  6. 13:00:05Orchestratorprofile_keysrows 300, non-null 300, distinct 300zetaris:run_sql
  7. 13:00:05Diagnoserdetect_patternunused_outer_join; 1 hit(s); parser ok; cites plan nodes 7
  8. 13:00:38Diagnoseranalyse_plan3 LLM hypothesis(es), 0 dropped; ranked: h1 > h2 > h3 > h4nim:chat_completions
  9. 13:00:38Orchestratorplaybook_open12 entries; pattern unused_outer_join; schema tpcds_dbhmem:open
  10. 13:00:38Orchestratorplaybook_querymem-12 refused (0.262), mem-3 refused (0.256), mem-2 refused (0.254), mem-11 refused (0.24), mem-1 refused (0.237)hmem:query
  11. 13:00:41RewriterrewriteRemoved unnecessary LEFT JOIN to promotion since no promotion columns are referenced and p_promo_sk is unique, so the join cannot change the | sql SELECT COUNT(1) AS n FROM TPCDS_DB.store_sales ss WHERE ss.ss_customer_sk IS NULLnim:chat_completions
  12. 13:00:41Orchestratorhandoffto Verifiermarkovian:chunk
  13. 13:00:59Verifierexecute1 row(s) in 18157.7 mszetaris:run_sql
  14. 13:01:27Verifierexecute1 row(s) in 27703.2 mszetaris:run_sql
  15. 13:01:40Verifierexecute1 row(s) in 13228.6 mszetaris:run_sql
  16. 13:01:56Verifierexecute1 row(s) in 15470.7 mszetaris:run_sql
  17. 13:02:28Verifierexecute1 row(s) in 32414.6 mszetaris:run_sql
  18. 13:02:58Verifierexecute1 row(s) in 29309.4 mszetaris:run_sql
  19. 13:03:09Verifierexecute1 row(s) in 11562.9 mszetaris:run_sql
  20. 13:03:09VerifiercritiqueACCEPT: same 1 row(s), 2.1× median over 3 pairs
  21. 13:03:09Orchestratorcase_verdictACCEPTED

Method · three agents and an orchestrator, one rule

A model proposes. Code decides.

Nemotron writes the diagnosis and the rewrite. The verdict, every number and every timing come from code that executes both queries on Zetaris.

code · no model

Verifier

It runs the original and the rewrite on the engine and refuses anything it cannot measure.

  1. determinism pre-check — the original runs twicesame rows
  2. schema check — column names and typesequal
  3. row diff, or an engine-side row hash above the cap0 rows differ
  4. alternating paired timing — ≥ 1.2×, most pairs wonsaving ≥ 1,782.9 ms (Zetaris noise floor)
xxhash64 · EXPLAIN FORMATTED · run_sql
hero-r1 · the three timed pairs in the order they ran — the order swaps each pair
  1. pair 1
    original 27.7 s
    rewrite 13.2 s
  2. pair 2
    rewrite 15.5 s
    original 32.4 s
  3. pair 3
    original 29.3 s
    rewrite 11.6 s

per-side medians 29.3 s → 13.2 s · 2.1× = median of the three pair ratios · saves 16.9 s = median pair saving · recorded/live/hero-r1

model

Diagnoser

Reads the Spark plan, a pattern detector and SQL-aggregate key facts.

hero: unused_outer_join at plan node 7 · key 300 distinct of 300

nemotron-3-ultra-550b-a55b

model

Rewriter

One rewrite per hypothesis. It sees every earlier refusal as a reason code, and the latest one with the Verifier's detail and the differing row.

nemotron-3-ultra-550b-a55b

code

Playbook

Refusals, and outcomes a human approves, go into the playbook. A candidate already refused for the same original result is answered from the ledger, not run again.

Meterless H-MEM

code

Orchestrator

Runs the retry chain: one attempt is one chunk. "Correct but no time saved" (no_gain) goes back to Diagnoser.revise; at most three attempts.

Meterless Markovian

Comments · the same edit on two keys

Three wrong rewrites, kept on the page.

Same query, a different join key: p_item_sk, which repeats. Dropping the join now changes the count. The Rewriter tried three times; the Verifier refused two by execution and the third from its ledger, and wrote why in the margin.

hero-r1 · ON p_promo_sk

300 distinct

of 300 non-null values · fingerprints 87cd00c5 = 87cd00c5

ROW-IDENTICAL · 87CD00C5

hero-trap-r1 · ON p_item_sk

291 distinct

of 293 non-null values · attempt 1 fingerprints e51f9da7 ≠ 87cd00c5

REFUSED · ROW_DIFF
original fingerprinte51f9da7
attempt 1 · drop the joinROW_DIFF · row 129752
attempt 2 · LEFT SEMI JOINROW_DIFF · row 2853
attempt 3 · drop the join againrefused from the ledger
verdictNO_SAFE_REWRITE

Without the Verifier

Sample pack on DuckDB (the hero, three traps, a control): one plain model call per query, no diagnosis, no memory, one attempt — scored by the same Verifier.

ΔΔΔΔΔΔ=====

■ changed rows 6□ identical, not faster 5⬚ guard / error 4identical and faster 0

With it

On the same trap queries the baseline changed rows in 3 of 9 runs, and Query Doctor's own first rewrite in 9 of 15. Every one of those 9 was refused.

ΔΔΔΔΔΔΔΔΔ

▨ changed rows, refused 9□ other first attempts 6shipped with changed rows 0

scroll the screen sideways →

The Attempts section of the hero-trap-r1 report: three attempts, each Rewriter note struck through in red, with margin notes 'rows differ: 1 vs 1 — first differing row ["129752"] (rewrite only)', 'rows differ: 1 vs 1 — first differing row ["2853"] (rewrite only)', and 'refusal reused from ledger mem-11 — same original-result fingerprint; not re-run'.
Real screen · #/recorded/hero-trap-r1 · Attempts

Designed queries, small N — the shares are not a rate on real workloads (evals/results.md).

Four sponsors, four jobs.

Zetaris

The engine both queries run on, over JDBC to the local platform: run_sql, EXPLAIN FORMATTED, engine-side xxhash64 fingerprints. The local bundle has no MCP endpoint, so no MCP tools are used.

11calls in hero-r1: 10 run_sql + 1 EXPLAIN

Meterless

H-MEM is the playbook, Markovian is the retry chain. Nightshift is the default model provider in code, tested against a mock runtime only.

3calls in hero-r1: 2 H-MEM + 1 Markovian · Nightshift live: 0

NVIDIA

Nemotron 3 Ultra serves the Diagnoser and the Rewriter. The recorded runs called it directly through NIM, the declared fallback.

2model calls in hero-r1: diagnose + rewrite

Cursor

Cursor's agent wrote the scaffold commit (d3e823c). Claude Code wrote the rest; every commit says which.

1commit, the scaffold

Honest by design · from the README

What it can't do yet.

COMMENT 1

“The retry loop does not reliably learn from a refusal: on the duplicated-key hero trap the first two attempts changed the rows when executed (one dropped the join, one swapped it for a LEFT SEMI JOIN) and the third repeated the first and was refused from the ledger without running, although the prompt carried the measured fact that the key is not unique.”

— README · Limitations
COMMENT 2

“Nightshift was never run live; there is no hosted CI run to cite (GitHub Actions did not start on the private repo).”

— README · Limitations
COMMENT 3

“We expected a 5–12 s hero; it runs about 35 s, so one live verification takes minutes, not seconds.”

— README · What we got wrong

Questions a judge would ask

Before you sign off.

Does a model decide whether a rewrite ships?

No. The Verifier is deterministic code: it executes both queries, compares the rows and times alternating pairs. The model's text is labelled as model text in every report and never decides anything.

Is the 2.11× measured or estimated?

Measured: wall clock on the local Zetaris platform over JDBC, 10 alternating pairs, rows verified identical first. The p50s are 32,442.6 ms and 16,428 ms. It is one laptop and says nothing about a hosted deployment.

Can I run it without keys?

Yes. Sample mode runs DuckDB on a committed NYC TLC subset with recorded model answers, and all 447 tests run keyless. The sample pack has no ACCEPT: the accepted speedup is from live Zetaris only.

What is not shown live?

Meterless Nightshift. It is the default provider in code with 16 mock-runtime tests, but it was never run against a live runtime; every recorded run used direct NVIDIA NIM, and every recorded trace with a model call names nim.

Will it change my database?

No. The guard admits one read-only statement per query; a rewrite cannot write or run a second statement. An accepted case is a candidate; when a human approves it, a patch file is written for them to review and apply.

Is "same rows" a proof?

No, and the report says so: the rewrite is identical on the current data and verified by execution — it is not a proof for all future data.