Skip to content

SchemaRouter 0.8.0

SchemaRouter 0.8.0 strengthens bounded semantic routing and multi-provider evidence handling while keeping the same execution-boundary philosophy: models may assist selection among locally declared capabilities, but they do not gain authority to invent tools, schemas, credentials, or side effects.

Highlights

Bounded semantic routing pipeline

0.8 separates several decisions that were previously easier to conflate:

  • optional multilingual semantic candidate recall with an explicit top-k bound;
  • a capability-fit gate that may abstain without choosing an execution route;
  • same-tool endpoint disambiguation that can only reorder sibling endpoints;
  • an operation-fit gate scoped to the currently leading tool domain;
  • trusted EndpointSpec.operation_aliases that are declared locally and fingerprinted.

These stages remain subordinate to registered schemas, policy, fingerprints, and execution validation. A decision backend can suppress or prioritize an already declared option; it cannot create execution authority.

Field evidence and coverage

Planning now makes heterogeneous information requirements more explicit:

  • PlanRequest.field_evidence can require provenance, license, source type, or unit evidence for one semantic field without imposing the same requirement on unrelated fields;
  • ToolCall.required_evidence and route-available evidence are revalidated again at execution;
  • ExecutionPlan.coverage reports required, covered, and uncovered semantic-field requirements;
  • explicit multi-call planning prefers complementary field coverage and can stop before max_calls once the matched field set is complete.

Unknown active field-evidence identifiers and forged evidence overclaims fail closed.

Multi-provider corroboration and aggregation

PlanRequest.retrieval_mode="corroborate" can deliberately request independent providers when the application wants corroborating evidence rather than minimum-call coverage.

Returned provider records can then be normalized with SourceRecord and aggregated through aggregate_records():

  • documents use trusted identifiers such as DOI/PMID/PMCID/arXiv for canonical grouping;
  • material and chemical identity requires explicit structural/canonical identifiers rather than formula-only or name/title similarity;
  • material and chemical fields preserve independent observations, units, qualifiers, provider, and provenance by default;
  • disagreement is surfaced instead of silently selecting one scientific value as truth.

SchemaRouter intentionally leaves fuzzy entity resolution and domain truth adjudication to the application layer.

Reproducible multilingual routing evaluation

0.8 adds a larger benchmark/evaluation discipline around the bounded routing stack:

  • 1,200-case multilingual/adversarial v2 stress corpus;
  • separate capability-fit, endpoint-disambiguation, operation calibration, and post-change holdouts;
  • split- and language-level metrics, Wilson intervals, failure taxonomy, JSON/CSV/HTML reports, and source/corpus reproducibility metadata;
  • explicit holdout-consumption rules so a measured test set is not reused as a tuning set.

The selected operation-fit representation and threshold were frozen using v5 development/calibration data only. The final frozen configuration was then run exactly once on a fresh 600-case v10 operation-generalization holdout.

Metric v9 consumed holdout v10 fresh one-shot
Overall accuracy 51.167% 55.667%
Supported-operation routed accuracy 38.281% 42.188%
Near-domain unsupported-operation rejection 70.833% 77.083%
Ordinary OOD rejection 100% 100%

The v10 failure taxonomy was:

  • 334 correct;
  • 44 false routes;
  • 199 missed routes;
  • 23 wrong endpoints;
  • 0 wrong tools;
  • 0 execution errors;
  • 0 invalid plans.

Of the 199 missed supported routes, 162 were attributed to operation-fit and 37 to capability-fit.

Current limitation

The result is an improvement over v9, but it is not presented as solved routing quality. Supported-operation routed accuracy is still 42.188% on v10, and operation-fit is the dominant remaining recall bottleneck.

v10 is therefore now consumed regression evidence. It must not be used for iterative threshold or representation tuning. Any post-0.8 optimization cycle needs a newly reserved untouched holdout before implementation choices are evaluated.

Compatibility and scope

0.8.0 intentionally removes no documented public API from 0.7.0. New routing stages are optional and bounded; multi-provider corroboration is explicit; aggregation does not bypass execution policy or schema validation.

SchemaRouter remains pre-1.0 and remains deliberately narrower than a general agent framework. Conversation memory, autonomous tool loops, workflow/DAG ownership, and model-provider orchestration remain outside core.

Release validation

The protected release surface covers:

  • Python 3.10–3.14 plus Windows smoke and Python preview signals;
  • minimum dependencies, Pyright, branch coverage, and warnings-as-errors;
  • wheel/sdist build and clean consumer acceptance;
  • LangChain, LangGraph, LlamaIndex, Jev, Laya, MCP, and OpenTelemetry integrations;
  • strict documentation build;
  • dependency audit, CodeQL, OpenSSF Scorecard, immutable Action pins;
  • release provenance, SPDX SBOM, and exact post-publish public-PyPI artifact verification.

External follow-up tracks

These remain deliberately separate from the runtime release:

  • LangChain/LlamaIndex ecosystem listing or packaging guidance; outreach is already sent and no maintainer reply had arrived as of 2026-09-26;
  • broader live hosted-provider/Jev decision evidence with dated runtime/token/cost metadata;
  • GitHub About metadata and protected-main freshness settings;
  • constraining the PyPI Trusted Publisher to the dedicated pypi GitHub environment.

The last two are repository/package-administration settings rather than missing runtime code.