# RIDGE Open Benchmark — Amazon FBA Niche Outcome Prediction

Hold-out evaluation of FBA niche classifiers on 169 ground-truth niches
observed 2022–2023 entry → 2026 outcome. Labels: ALIVE / THRIVING / DEAD.

Baseline to beat — always-predict-DEAD on this set achieves 79.9%
binary accuracy. Any submission at or below that is information-free.

## Current Leaderboard

| Rank | Submitter | Binary Acc. (95% CI)            | NO-GO Precision (95% CI)         | PR-AUC raw / iso | 90% Conformal (singleton) | Date       |
|------|-----------|---------------------------------|----------------------------------|------------------|---------------------------|------------|
| 1    | RIDGE v7  | 78.7% [72.2–84.6]       | 81.1% [74.8–86.8]         | 0.901 / 0.814    | 90.3% (82.3%)             | 2026-04-23 |

No independent competitor has published comparable held-out numbers on
this or any FBA niche benchmark as of 2026-04-23.

## Methodology

  * Training set: 868 ground-truth labels, isotonic-calibrated XGBoost
    with monotone constraints. ECE 0.058.
  * Test set here: 169 original labels observed over 2.5+ years — not
    leaked into training (honest 5-fold CV).
  * Binary accuracy measured against v3 verdict rule on the same 169
    niches (NO GO vs GO family).
  * Confidence intervals: bootstrap n=2,000 on the 169-niche set.
  * 90% conformal coverage: inductive conformal prediction (alpha=0.10)
    on nonconformity scores from 5-fold OOF. Singleton rate = share of
    niches where the conformal set is decisive (not abstain / empty).

## How to submit competing numbers

1. Download `niches.jsonl` (169 rows: keyword + category + ASINs at entry).
2. Run your tool or classifier. Produce a JSONL with the same keywords
   and your binary prediction + optional confidence score.
3. Email results + methodology to `research@ridgeworldwide.com`.
4. We re-score the submission against held-out ground truth and add the
   row to this leaderboard with method description + submission date.

## Why this matters

Every FBA research tool claims accuracy. Until 2026-04-23, none published
a held-out benchmark. This file turns a marketing claim into a falsifiable
metric. RIDGE is the first vendor to put a number on the wall that anyone
can take a shot at.