SchemaRouter 0.9.0¶
SchemaRouter 0.9.0 adds a bounded pairwise decision primitive for applications that want cross-encoder or reranker-style relevance scoring while preserving SchemaRouter's local execution authority model.
Highlights¶
Provider-neutral pairwise decisions¶
PairwiseDecisionBackend accepts an application-owned scorer over already-authorized
(query, option_text) pairs and maps its scores back to the original opaque option IDs locally.
The contract is deliberately narrow:
- scorers cannot invent tools, endpoints, credentials, arguments, or side effects;
- option metadata is not forwarded by the default formatter;
- one score in
[0, 1]is required for every authorized option; - wrong batch lengths, malformed values, NaN/Infinity, and out-of-range scores fail closed;
- score thresholds and selection-boundary margins can abstain conservatively;
- sync and async scorers are supported;
- SchemaRouter core gains no Torch, Transformers, or model-specific dependency.
Applications remain responsible for the scorer implementation and workload-specific calibration.
Pairwise operation-fit research¶
After the 0.8.0/v10 release, operation-fit optimization continued on development/calibration data without reusing consumed holdouts.
Threshold-margin variants, two-tier rescue rules, multilingual MPNet, and multilingual E5 did not
improve the pre-registered robustness objective over the existing multilingual MiniLM bi-encoder.
A pairwise experiment with BAAI/bge-reranker-v2-m3 was the first candidate to improve that
development/calibration objective.
The complete candidate configuration was frozen before evaluating the reserved v11 holdout:
- semantic candidate recall: multilingual MiniLM, top-k 2;
- capability fit: multilingual MiniLM, min similarity 0.25;
- operation fit: BGE reranker v2-m3, sigmoid pair score, min score 0.01;
- endpoint disambiguation: multilingual MiniLM, min margin 0.03.
v11 one-shot evidence¶
The frozen BGE candidate was evaluated exactly once on the fresh 600-case multilingual v11 operation-generalization holdout.
| Metric | BGE pairwise v11 | MiniLM 0.40 post-consumption diagnostic |
|---|---|---|
| Overall accuracy | 51.667% | 54.833% |
| Supported-operation routed accuracy | 25.781% | 40.625% |
| Near-domain unsupported-operation rejection | 97.396% | 77.604% |
| Ordinary OOD rejection | 100% | 100% |
| Equal-weight supported/rejection score | 61.589% | 59.115% |
| Mean CPU latency | 589.627 ms | 253.947 ms |
The MiniLM comparison was run only after v11 had already been consumed, so it is diagnostic evidence, not a fresh holdout claim.
The result shows a real trade-off: BGE improved the pre-registered equal-weight rejection/recall objective by 2.474 percentage points, but reduced supported-route recall by 14.844 points, reduced overall accuracy by 3.167 points, and more than doubled CPU latency. 0.9.0 therefore exposes pairwise scoring as an optional backend and does not make BGE the library default.
v11 is consumed and must not be used for later threshold, model, representation, or hybrid-rule selection.
Reproducibility¶
Frozen BGE one-shot:
- source revision:
57af2fd18d254fa1a9cd5855a158ae74b94adfc7; - corpus SHA-256:
efa8cd371bc7613895e44a915d78c77f5b88239c64a8de52d9c407839660d816; - Actions run:
36239340782; - artifact ID:
10905503165; - artifact SHA-256:
f977511c7d65bc94e2dba9142b6cedf378ceaae1b8418c351e85b550f191dfd1.
Post-consumption MiniLM diagnostic:
- source revision:
099cf009003df35f48571ecdc988fddaa11db904; - Actions run:
36240431389; - artifact ID:
10905429518; - artifact SHA-256:
1048405ab34b41371004247e15934f5086a53977a14d4957bd14c6d69e194dee.
Compatibility¶
0.9.0 adds a new public backend without removing the existing embedding, deterministic, hosted, or integration decision paths. Existing applications do not need to opt into pairwise scoring.
The release remains pre-1.0 and follows the compatibility policy in
docs/versioning.md.
Release validation¶
The protected release surface covers Python 3.10–3.14, Windows smoke, minimum dependencies, type checking, branch coverage, wheel/sdist consumer acceptance, optional integrations, strict docs, dependency/security analysis, CodeQL, OpenSSF Scorecard, SBOM/provenance generation, and exact post-publish PyPI artifact verification.