Scoring Constitution
Constitution v1.0.0 · Ruleset v1.0.0
Every deterministic score in ArchitectOS answers to these articles. None of it is a black box: the engine is the referee, AI may only ever comment on what the engine already decided, and every point on every score traces back to one rule below.
Every score must be deterministic.
ImplementedThe same design must produce the same score every time — a score that depends on hidden state or luck cannot be trusted or defended.
lib/scoringEngine.ts
Every score must be reproducible.
ImplementedReproducibility is what turns a number into evidence — anyone re-running the same design through the same ruleset version must get the same result.
lib/scoringEngine.ts
Every score must expose its reasoning trace.
ImplementedA number without a trace is an opinion. A number with a trace is a claim that can be checked.
lib/scoringEngine.ts
AI may explain scores but never create or modify them.
PartialThe deterministic engine is the referee; AI is the announcer. Letting a model touch the numbers turns a verifiable score back into an opinion wearing a number's clothes.
Holds today because no AI surface reads these scores yet. Wiring AI commentary on top of the trace is planned future work, bound by this article from the start.
Every benchmark must have explicit eligibility rules.
PlannedA ranked comparison without eligibility rules isn't a comparison — it's noise with a leaderboard skin.
A minimal eligibility check ships in lib/scoringEngine.ts today (rejects an empty design); real eligibility rules arrive with the first real benchmark.
Every trade-off must be legible, not merely intentional.
ImplementedThe engine can prove a trade-off is visible in the scores. It cannot know, and will never claim to know, what the author meant.
lib/scoringEngine.ts — axes move independently, so a design that buys availability with cost shows both movements plainly.
No architecture can maximize every dimension simultaneously.
ImplementedIf one design wins every axis, the axes aren't actually independent, and the benchmark is measuring one thing wearing several names.
By construction — Cost, Simplicity, and Availability pull against each other (redundancy costs money and adds components).
Every score is challenge-specific; scores are not comparable across challenges.
PartialA 91 on one brief and a 91 on another brief are not the same claim — treating them as comparable is how leaderboards quietly lie.
True by omission today — there are no challenges yet. Becomes load-bearing the moment a Weekly Challenge exists.
Every rule must be versioned.
ImplementedRules will change as gaming strategies are found. History must not silently reinterpret itself when they do.
lib/scoringEngine.ts — every rule table is tagged with SCORING_RULESET_VERSION, and every trace line carries it.
Historical scores remain reproducible under the rules that produced them.
PlannedA rule change must never rewrite the past — a score earned under v1.0.0 stays a v1.0.0 score forever.
Requires persistence, which doesn't exist yet. The versioning in lib/scoringEngine.ts is structured so this is additive later, not a rewrite.
Scoring is author-blind.
ImplementedThe engine sees a graph, never who built it. Reputation is a property of a portfolio of solutions over time — not an input to any single score.
lib/scoringEngine.ts — computeDeterministicScorecard's signature takes only (nodes, edges), nothing about identity.
Disputed scores must be traceable to a specific rule, not to "the algorithm."
PlannedTrust survives being wrong in public only if wrong is fixable and visible. A black box that's wrong is a scandal; a cited rule that's wrong is a bug report.
The trace format already attributes every point to one rule ID — the prerequisite. A dispute flow is future work.
Every AI-authored explanation must be attributable to the deterministic trace it explains, never to the AI's own reading of the graph.
PlannedAn AI comment that disagrees with the trace it's supposedly narrating is two judges contradicting each other in public — worse than no comment at all.
Same reasoning as Article IV — enforced once AI commentary is wired to the trace.
Rule Reference — v1.0.0
Every rule that can appear in a score's trace, independent of any one design. Cost uses a generic $60/mo-per-component reference curve for now (Article VIII) — it becomes challenge-specific once Weekly Challenges exist.
| Rule | Axis | Condition | Points |
|---|---|---|---|
| COST-1 | Cost | Cost per component above the $60/mo reference curve | −round((cost/component − 60) × 0.5) |
| COST-2 | Cost | A cache (Redis) and a database (Postgres/MongoDB/DynamoDB) both present | +8, once |
| SIMP-1 | Simplicity | Component count | −3 × components |
| SIMP-2 | Simplicity | Connection count | −2 × connections |
| SIMP-3 | Simplicity | Distinct technology kinds | −4 × distinct kinds |
| AVAIL-1 | Availability | Postgres without Multi-AZ | −18 per instance |
| AVAIL-2 | Availability | Postgres with 0 read replicas | −9 per instance |
| AVAIL-3 | Availability | Redis with cluster mode off | −6 per instance |
| AVAIL-4 | Availability | Kafka with replication factor ≤ 1 | −14 per instance |
| AVAIL-5 | Availability | MongoDB with replica set size ≤ 1 | −14 per instance |
| AVAIL-6 | Availability | Elasticsearch with node count ≤ 1 | −10 per instance |
| AVAIL-7 | Availability | Microservice with min instances ≤ 1 | −8 per instance |
| AVAIL-8 | Availability | Container workload with replicas ≤ 1 | −8 per instance |
| AVAIL-9 | Availability | All components share one region (or none is drawn) | −10, once |
| AVAIL-10 | Availability | All components share one availability zone (or none is drawn) | −6, once |
| AVAIL-11 | Availability | A load balancer is present | +5, once |
| AVAIL-12 | Availability | Load balancer health-check interval slower than once a minute | −4 per instance |