Calibration
The pool concurrency defaults agb ships, where they came from, and how agb probe measures new ones.
agb caps how many agy requests run at once in each quota pool. The caps are measured, not guessed: agb probe fires sentinel prompts at increasing widths and records which widths a pool sustains. The results are committed as reports under docs/calibration/.
Current defaults
These are the defaults in lib/pools.mjs at v1.0.0 (DEFAULT_CAPS):
| Pool | Concurrent requests | Basis (probes-2026-07-14) |
|---|---|---|
gemini-flash | 8 | 8 of 8 succeeded at width 8. |
gemini-pro | 4 | 4 of 4 at width 4. |
claude | 4 | 4 of 4 at width 4. |
gpt-oss | 2 | 2 of 2 at width 2, only 2 of 4 at width 4. |
A plan's caps object is merged over these, per pool (PoolSet). At runtime the caps can shrink further when agy's quota telemetry reports less headroom. The pools are independent: saturating one does not slow another, which is why the router spreads tickets across pools rather than queueing on one. See Pools and tiers.
Probe reports
| Report | Status |
|---|---|
| probes-2026-07-14.md | Current. The numbers behind the defaults above. |
| probes-2026-06-11.md (superseded) | Measured with agy v1.0.7 against the Gemini 3.5 model generation using single samples; the 2026-07-14 report re-measured every pool on agy 1.1.1 and is what the shipped caps follow. |
The 2026-06-11 report is kept unchanged in the repository as a historical record; only this page marks it superseded.
agb probe writes its output as probes-<date>.md, the same naming as the committed reports (v1.0.0 wrote the singular probe-<date>.md: C7), and the comment above DEFAULT_CAPS cites docs/calibration/probes-2026-07-14.md.
Running a probe
agb probe [widths] [model]
# defaults: widths 2,4,8 and model "Gemini 3.8 Flash (Low)"
agb probe 2,4,8 "Claude Sonnet 5.5 (Medium)"For each width the probe sends that many concurrent Reply with exactly: PONG-<i> prompts with a 120 s timeout; a request counts as ok only if agy succeeds and the output contains its own sentinel (probe loop). Each row is printed to stdout as JSON: model, width, ok, wall_ms, median_ms, max_ms.
If at least one request succeeded, the rows are appended as a Markdown table to the day's calibration file. If every request failed, nothing is written, so garbage latencies never become calibration data (write guard). A write failure (for example a read-only install prefix) is reported but does not change the exit code.
The report goes to AGB_CALIBRATION_DIR if it is set, otherwise to docs/calibration/ under the current working directory, so running agb probe from your repository root puts the report in that repository.
When to re-probe
- After an agy upgrade, since server-side limits and retry behavior change between releases.
- When a pool starts producing timeouts or throttling at the default cap.
- When new models appear in
agy models.
To apply new numbers without a release, set caps in your plan. To change the shipped defaults, update DEFAULT_CAPS and commit the probe report that justifies them in the same change.
Statistical evaluation & benchmarks (npm run eval)
Grounded in the empirical calibration curves in docs/calibration/probes-2026-06-11.md and docs/calibration/probes-2026-07-14.md, the repository ships a deterministic, zero-dependency statistical evaluation suite under evals/ (evals/stats.mjs, evals/scenarios.mjs, evals/runner.mjs) and its generated report at docs/benchmarks/marketplace-eval-report.md. For the full Pareto frontier analysis, 80/20 root-cause Pareto charts, and per-class distributions, see Benchmarks & Statistical Evals.
Run the benchmark suite locally at any time:
npm run eval
# or emit machine-readable JSON:
node evals/runner.mjs --jsonAcross N = 100 paired Monte Carlo trials per scenario (seed 20261030), agb demonstrates statistically significant (p < 0.0001) improvements over both naive Straight Antigravity (agy) and Raw agy with Best Practices (/plan, /teamwork, invoke_subagent(Workspace: 'share', Model: 'flash' | 'pro'), AGENTS.md, and adversarial-reviewer in a fresh subagent context):
| Benchmark Scenario | Straight agy (Naive) | Raw agy (Best Practices) | Antigravity + Booster (agb) | agb Gain vs. Best Practices (and vs. Naive) | Statistical Significance (vs. Best Practices) |
|---|---|---|---|---|---|
| 8-Ticket Feature DAG Latency (Mean / P95) | 576.1s / 647.0s (Serial)194.8s / 265.0s (1-Pool) | 184.4s / 216.2s | 138.5s / 160.4s | 1.33x faster vs. Best Practices (d = 2.84)4.16x faster vs. Serial | p < 0.0001 (Mann-Whitney U) |
| 16-Ticket Scale Refactor Latency (Mean / P95) | 312.5s / 354.9s | 274.2s / 366.3s | 132.0s / 173.7s | 2.08x faster vs. Best Practices (-52.6% P95)2.37x faster vs. Naive | p < 0.0001 (Mann-Whitney U) |
| Pool Rate-Limit Timeout Rate (1,600 Dispatches) | 24.1% ([22.1%, 26.3%]) | 9.3% ([8.0%, 10.8%]) | 1.2% ([0.8%, 1.8%]) | -87.2% timeouts vs. Best Practices-95.1% vs. Naive | z = 10.30, p < 0.0001 |
| 5h Frontier Pool Quota Consumed (8-Ticket DAG) | 18.9% (Serial) / 15.6% (1-Pool) | 10.5% | 4.8% | -54.2% frontier quota vs. Best Practices-74.6% vs. Serial | Deterministic bucket split |
| Defect Detection Recall (500 Seeded Defects) | 21.8% (No review)52.0% (Same-model) | 73.6% (adversarial-reviewer subagent) | 92.4% (Cross-family prosecutor + 4-lens fleet) | +18.8 pp recall vs. Best Practices+40.4 pp vs. Same-Model | z = 7.91, p < 0.0001 |
| Post-Review Defect Escape Rate | 78.2% (No review)48.0% (Same-model) | 26.4% | 7.6% | -71.2% fewer escaped defects vs. Best Practices-84.2% vs. Same-Model | z = 7.91, p < 0.0001 |
| Safety Hazard Prevention (Rails, Scope, Mutants) | 56.3% blocked (Prompt-only) | 80.3% blocked (AGENTS.md + review) | 98.0% blocked (100% on rails & scope) | -89.8% hazard escapes vs. Best Practices-95.4% vs. Naive | z = 6.96, p < 0.0001 |
Exit and error codes
Every agb exit code per command, and the error classes and kinds users see.
Benchmarks & Statistical Evals
Empirically calibrated statistical benchmarks, Pareto frontier analysis, and 80/20 root-cause breakdowns comparing Antigravity Booster (agb) against Naive Straight agy and Raw agy (Best Practices).