SemVer-Trust

Steelman Analysis — SemVer-Trust Specification & Codification Strategy

Date: 2026-07-04 · Status: findings dispositioned (ADR-017, ADR-018, ADR-019; spec v0.2 queue; design record r4) Method: charitable reconstruction → load-bearing decomposition → adversarial testing → refinement. Conducted pre-implementation, at maintainer direction, before codification momentum could outrun stress-testing.

1. Steelman reconstruction

The specification’s claim, in its strongest form. When the marginal cost of producing plausible code approaches zero, every trust proxy in software distribution silently decalibrates. Version numbers, changelogs, commit histories, review badges, and maintainer reputation were evidentiary signals calibrated for a world where code was expensive to produce — rate-limited by accountable human attention. Agents break the calibration without breaking the signals: a plausible history, a plausible release, a plausible PATCH bump now imply nothing about whether any human ever attended to the change. The signals persist; their evidence value collapses. SemVer-Trust is not “labeling AI code” — it recalibrates the release-trust signal by making the evidence behind a compatibility claim explicit, cryptographically verifiable, and consumable through the one channel every dependency resolver already parses: version precedence. Under-evidenced releases become opt-in by construction, with zero consumer-side tooling change — an adoption property almost no proposed standard has. It measures the only thing cryptography can honestly measure (attested human accountability, not keystrokes), declining an unwinnable authorship-detection war. It completes a stack: SLSA answers “how was this built,” sigstore answers “who signed this,” and nothing answers “what human evidence stands behind this release’s claims.” Its incentive gradients point the right way: derivation proofs make spec-first architecture pay; transitive propagation makes shared-library review the highest-ROI human attention. And the timing claim: the convention is only cheap to establish before unlabeled agent code becomes the normalized default — the window is open now and will not reopen.

The codification strategy’s claim. A conformance suite as the sync contract is the only mechanism proven to keep multi-implementation standards coherent (toml-test, JSON-Schema-Test-Suite); building it before the implementation makes the reference implementation the first conformer rather than the de facto spec; Go-first maximizes evidence-provider richness and maintainer velocity during the phase where maintainer velocity is the project.

2. Load-bearing decomposition

Empirical claims

# Claim Status
E1 Agent-authored code is a large, growing share of production changes Effectively established
E2 Trust levels correlate with outcome risk (defects, vulns, incidents) Unverified. The keystone.
E3 Provenance capture deploys at acceptable friction (signing, trailers, merge attestations) Verified by existing tooling
E4 Pre-release precedence yields opt-in-by-construction across resolvers Mechanically verified (Go/npm/Cargo; PyPI lossy, known)
E5 Consumers will act on the signal (resolvers, policy engines, procurement) Unverified; zero demand-side artifacts existed at analysis time
E6 Attested accountability ≈ actual accountability at scale (laundering residual stays small) Unverified; auditable in principle
E7 Weakest-link flooring yields non-degenerate level distributions on real repositories Unverified; testable retroactively

Value judgments

V1 — accountability is the right thing to measure (vs. purely outcome-based verification). V2 — weakest-link over weighted aggregation (security posture over signal richness). V3 — honesty over coverage (measure less, truthfully). V4 — zero-consumer-change adoption over signal expressiveness. V5 — the version string is a legitimate carrier. V6 — obscurity, not misuse, is the near-term risk (ADR-014).

Keystone

E2, buffered by V1. If trust levels predict nothing about outcomes, the apparatus is ceremony — unless accountability has value independent of defect rates (someone to answer; liability attachment; incentive alignment), which is V1’s claim. The true collapse condition is a conjunction: E2 false and the market ceases to value accountability per se. The spec half-knows this (§1.2 disclaims correctness guarantees; §12.1 flags T1 efficacy) but had never stated the fallback plainly. E5 is the other near-fatal claim, but E5 is fixable by us (build consumers) where E2 is only testable by us.

3. Adversarial testing

3.1 If the steelman is 100% right, current strategy has three exposures

  1. Sequencing optimizes the wrong variable. The roadmap (domain → schemas → conformance → implementation → dogfood) is thoroughness-ordered; a closing calibration window demands visible-adoption ordering — and no roadmap item built a consumer. The signal’s value is realized entirely at consumption; a verify badge (“SemVer-Trust: T2 ✓”) is the precedented zero-friction wedge (coverage badges, goreportcard) and was absent.
  2. The keystone was uninstrumented. Retrospective trust profiling — computing would-have-been profiles from existing history (signatures and platform review data already exist) — both tests E2 against CVE/incident data and gives every repository a zero-commitment onboarding moment. Highest-leverage unplanned artifact.
  3. T2 is the volume tier, so attestation ergonomics (.gitmessage, --trailer, merge-attestation workflows) are product surface, not hygiene.

3.2 Boundary conditions (severity-ordered)

  1. Security-patch velocity vs. demote — the sharpest break, and internal: an under-evidenced security fix lands in the channel default resolvers don’t select, slowing patch propagation exactly when speed matters; any expedite carve-out is the door an attacker labels “security fix.” Unsolved. Owned publicly via spec §12 (Appendix A.3).
  2. Capability parity (E2 erosion). As agent review approaches median human quality, a high-evidence T1 release may empirically beat a rubber-stamped T3. Read as risk-ordering, levels are falsified by parity; read as accountability-ordering they remain true. → P6 (ADR-019).
  3. Goodhart equilibrium. Procurement-mandated T2+ breeds signature-as-insurance markets and review theater. The floor holds — a named accountable human with liability exposure is not nothing (the SOC2 lesson) — but the mature framing is accountability infrastructure first, quality signal second, adopted deliberately (standing prediction, §5).
  4. Mixed-authorship decay. Within years, Provenance: mixed plausibly describes nearly every commit; the authorship axis degenerates and levels converge to counting independent human reviewers. Scheme survives (the review axis carries accountability); trailer taxonomy becomes partly vestigial. Standing prediction, §5.
  5. Runtime-confinement boundary. Where execution is sandboxed/capability-scoped (wasm components, curated stores), release provenance adds little; the honest domain is code running with ambient authority. Scope observation; no spec change (§1.2 already gestures).
  6. Conformance fixtures age (codification-specific). Suite-as-sync-contract is strongest for the timeless core (levels, precedence, aggregation) and weakest where the scheme is hardest: fixtures embedding real signatures rot, because keyless certificates are short-lived by design and historical verification leans on transparency-log inclusion. Unaddressed, the suite tests the easy 20%. → ADR-018.

4. Dispositions

Finding Disposition
No demand-side artifact (E5) ADR-017: verify GitHub Action + badge, ranked with the CLI
Keystone uninstrumented (E2) ADR-017: retrospective trust profiling
Attestation UX is product ADR-017 decision text
Security-patch/demote tension Spec §12 open question (Appendix A.3) — owned before critics find it
Capability parity P6 via ADR-019 (Appendix A.1)
Goodhart equilibrium Standing prediction; framing recorded in design record
Mixed-authorship decay Standing prediction
Runtime confinement Analysis-level scope note only
Fixture aging ADR-018: injectable trust roots + clock, vendored long-lived test keys

Verdict: the core thesis held — the collapse conjunction (E2 ∧ ¬V1) survived pressure because accountability retains market value even at capability parity. The strategy did not hold unchanged: “spec polish” was demoted; demand-side artifacts and keystone instrumentation were promoted. That is a strategy change, not a refinement.

5. Standing predictions (check against reality)

  1. If procurement adopts trust floors, signature-as-insurance intermediaries appear; the correct response is the accountability-infrastructure framing plus evidence-vector-based policy, never level inflation.
  2. The authorship axis decays toward mixed-everywhere; a future spec major may collapse levels to reviewer-counting. Do not pre-act; watch trailer distributions from retrospective profiling.
  3. A null E2 result repositions, not voids, the scheme (P6); default policy tables would need rebuilding around evidence vectors.

Appendix A — Ready-to-paste spec v0.2 text

A.1 (§1.1 principle 6 + §3.1 clarification):

  1. Levels order accountability, not risk. Trust levels rank attested human accountability, not predicted defect rates. Empirical risk assessment belongs to policy, which consumes the full evidence vector. A high-evidence T1 release may be empirically safer than a rubber-stamped T3; the levels remain true because they claim only who stands behind the change.

(§3.1, after the levels table) Levels are an accountability ordering, not a risk ordering (Principle 6): policy maps levels plus evidence to risk; the levels themselves claim only the count of independent accountable humans.

A.2 (§4.4, ADR-015 mirror):

Toolchain pins SHOULD be self-contained and language-native (module/tool manifests with checksum verification, or container image digests). Environment-manager state (development-environment lockfiles) MUST NOT serve as a derivation input: re-running a derivation for verification requires only the language toolchain and the pinned inputs, never the maintainer’s development tooling.

A.3 (new §12.7):

Security-patch velocity vs. channel demotion. Under strategy = "demote", an under-evidenced security fix lands in the pre-release channel that default resolvers do not select — the scheme can slow patch propagation exactly when speed matters most, and any expedite carve-out is a door an attacker will label “security fix” to walk through. Candidate directions, all unproven: expedited review SLAs rather than expedited channels; advisory-linked promotion (a patch promotes when a linked advisory is published by a distinct accountable identity); accepting the tension and documenting emergency response as out of band. This is currently the scheme’s strongest known internal counterargument.

A.4 (new §12.8):

Empirical validation of the trust–outcome link. The keystone empirical claim — that trust levels correlate with outcome risk — is untested. Retrospective trust profiling of existing repositories against vulnerability and incident history (see the reference-implementation roadmap) is the designated test. A null result does not void the scheme (Principle 6) but would reposition it as accountability infrastructure rather than a risk signal, and should reshape default policy tables.