Comparison with upstream EHRbase

FerroEHR is measured against upstream EHRbase (Java) — the project it succeeds — on the same instruments it applies to itself: the CNF 2.0 conformance runner executes the same committed catalogue against both servers’ official deployments, and the benchmark harness drives both with byte-identical clinical workloads on the same host. Both directions are always published; a result that favours upstream is reported exactly like one that favours us.

Each side runs with its own committed party set — an ixit describing the reachable instances (upstream’s Basic auth has no read-only principal, so its ixit declares none) and a statement (the ICS) declaring the capabilities and ambiguity-register options that party actually claims. A capability a party does not claim reads not claimed and never gates its verdicts; a test whose ground cannot exist on a party’s topology or technology profile reads not applicable with a machine citation, never fail. That is how the comparison stays fair without ever weakening a case.

Every number and every curve on this page is generated at build time from the committed run artifacts (docs/conformance/*/results.json + verdicts.json + stress.json) — nothing here is hand-typed, and the CI stale-numbers gate rejects any attempt to hand-type it. To reproduce either side yourself, see Conformance and Benchmarks.

• Conformance
• Systems under test
• Methodology
• Profile verdicts
• In-scope outcomes
• Capability-by-capability
• Failures — both directions
  • ferroehr failures (with the upstream outcome on the identical case)
  • Upstream failures by schedule chapter
  • The principal upstream divergences, stated plainly
• Performance
• Method, in one paragraph

Conformance

Systems under test

Methodology

Both systems execute the same committed CNF 2.0 catalogue (863 case-by-format executions) through the same reference runner (tools/cnf-runner), each on fresh volumes with its own committed party set: the ixit names the reachable instances (upstream declares no readonly principal), and the statement (the ICS) declares the claimed capabilities, spec versions, and ambiguity-register options — ISO/IEC 9646-style test selection excuses undeclared option branches, unclaimed capabilities, and release-dated behaviour outside the declared versions as N/A with a citation, never as silent skips. Verdicts are pure functions of (statement, results, catalogue, capability matrix).

The declared-version delta matters and is stated, not hidden: ferroehr declares ITS-REST 1.1.0 while upstream EHRbase declares ITS-REST 1.0.3 — the catalogue realizes 1.1.0, so every Release-1.1.0-dated behaviour (the Demographic API, ITEM_TAGs, Simplified Formats on the wire, the admin EHR delete, the weak-ETag/Location header forms, …) is cited N/A for the 1.0.3 declaration rather than driven against a release upstream never claimed. The verdict-bearing comparison below is therefore each party’s in-scope subset, never the raw record.

Profile verdicts

In-scope outcomes

Runs compared: ferroehr (run of 2026-07-31) vs upstream EHRbase 2.34.0 (run of 2026-07-31) — the SAME catalogue through the same runner, each with its own committed party statement. Per the presentation rule, the headline is each party’s VERDICT SCOPE (the cases its own declarations select), never the raw record: a raw count would book release-dated and unclaimed surfaces against a party that never claimed them.

An inconclusive row’s wire answered outside the operation’s bound outcome map, or its required ground could not be established (e.g. a refused provisioning exchange) — never counted as a failure of the behaviour under test. Every not-run row in the full committed record (docs/conformance/<sut>/results.json) carries a machine-readable citation: an undeclared option branch, an unclaimed capability, a release-dated behaviour outside the declared spec versions, or a ground the party’s topology cannot establish.

Capability-by-capability

Evidence tokens from each party’s computed verdicts: passed (every gating case green), failed (at least one gating case red), inconclusive (a gating case neither passed nor failed cleanly), not_evidenced (claimed, but no gating case produced a verdict — there is no excused state: a required capability without passing evidence fails its tier, whichever party claims it), or not claimed (absent from that party’s ICS).

Failures — both directions

ferroehr failures (with the upstream outcome on the identical case)

Upstream failures by schedule chapter

Both servers’ capability conformance, from each party’s committed verdicts (generated, diff-guarded — see Conformance for how to read the grid):

And the per-chapter outcomes side by side. Both charts render the same chapter-and-band taxonomy, so they read band-for-band: a band upstream did not exercise shows as an explicit no cases row in the same position. Compare the printed counts, not the bar lengths — each chart scales its bars to its own widest band, and the legend states that scale.

Reading the upstream results honestly: the failures concentrate where the catalogue pins strict specification behaviour — archetype-constraint validation depth (the content chapter), exact status codes and version headers, canonical-format details — that upstream implements differently or predates. An inconclusive row means upstream answered with a status the operation’s specification-cited outcome map does not contain, or refused the exchange that would have established the case’s required ground, so the runner refuses to guess a verdict. And where upstream simply does not implement a surface (the ITS-REST simplified-format media types, ADL 2 provisioning, demographics), or where the specification dates a behaviour to a REST-API release newer than the one upstream declares (upstream declares ITS-REST 1.0.3; the catalogue realizes 1.1.0), the result reads not claimed or N/A with a citation — upstream is never counted as failing a surface it never claimed, never against a release it never declared, and never on an ferroehr-only extension.

The principal upstream divergences, stated plainly

Each of these was reproduced live against the composed upstream stack during triage of the committed record, and each is grounded at or below upstream’s own declared ITS-REST 1.0.3 unless marked; the full wire evidence lives in the committed docs/conformance/ehrbase-java/results.json.

• A quoted If-Match value is rejected (400 "UUID string too large") — including the server’s own echoed ETag — while only the non-standard unquoted form is accepted. The quoted form dates to Release 1.0.2.
• Semantic model violations answer 400 instead of 422 (a Release 1.0.1 correction), and some model-invalid documents are accepted outright — an EHR_STATUS without its mandatory archetype details commits as 201.
• The openehr-audit-details committal header is ignored (both the current and the deprecated spelling), and the stored audit description is itself model-invalid (a DV_TEXT with no value).
• Canonical XML is served with the root element in no namespace, against the published XSD’s qualified target namespace; the stored-query list even serves an XML <List/> document that conforms to no published schema on a JSON-only operation.
• Unqualified stored-query names are rejected although the specification makes the namespace optional and lists my_compositions as a valid example; a stale If-Match on a directory delete answers 404 where the specification requires 412; and 405 responses omit the required Allow header.
• The stored-query listing does not match by prefix. The specification’s own worked example lists “all versions of all queries with names starting with org.openehr”; upstream returns the versions of an exactly-named query but answers 200 [] for any shorter prefix, so a client cannot discover what a namespace holds.
• A malformed identifier in the path answers 404, not 400. Given a path segment that is not a UUID at all, upstream detects the type violation and still reports a miss — literally "EHR not found, in fact, only UUID-type IDs are supported" — for the EHR, versioned-COMPOSITION and CONTRIBUTION reads alike. (It does answer 400 for a malformed version identifier, so the behaviour is not even internally consistent.)
• Directory writes against an EHR that has no directory answer 412. Both the update and the delete report Precondition Failed with the body “does not contain a directory” — a missing resource dressed as a failed precondition. HTTP requires the opposite order: a failure detectable before the precondition is evaluated takes precedence over evaluating it.
• A contribution refusal surfaces as a raw 500. Committing a first version whose change type is deleted returns 500 "An internal error has occurred", where the specification assigns that family a 400 (“the modification type does not match the operation”). The neighbouring row is worse than an error: a creation whose lifecycle state is deleted is accepted (204) instead of refused.
• (1.1.0-grounded) The template upload refuses Accept: application/json (406), serving only XML — the released parameter enumeration lists JSON first. This single refusal is what makes most content-chapter rows inconclusive: the runner’s provisioning uploads ask for JSON, upstream refuses, and the case’s ground never exists.

Two red rows are not upstream’s fault, and are called out rather than counted. Storing a query with no version in the URL has to land at some version, and no released sentence says which. Our suite pins 1.0.0 because a suite must pin something; upstream continues the existing series instead (a stored 2.0.0 makes the next version-less store 3.0.0). Both are defensible readings of a silence, so the two store_query-{default_slot_with_higher_version,update_in_place} rows record a difference of house convention, not a conformance defect — the open question is with openEHR, not with either implementation.

Performance

Both systems run the same committed step-load stress instrument (cnf-runner stress) on their own freshly seeded cnf.scale.10k corpus: the geometric ladder climbs until the system leaves the envelope (p99 over the budget or errors past tolerance), then bisects to the maximum sustainable throughput. Every number derives from the two committed stress.json reports; each load step embeds re-checkable histograms and, where sampled, per-container resource telemetry.

A stress report is exploration evidence: it earns no conformance class (classes are earned exclusively by the hour-long measured class runs) and carries no class vocabulary — the chart shows where each system breaks.

Method, in one paragraph

The conformance instrument derives every expected outcome from the openEHR specifications — never from either server’s observed behaviour — and runs against real composed deployments of both systems (scripts/conformance.sh, CONF_SUT=ehrbase-java for the upstream side). The stress instrument drives the same hospital-simulation workload (admissions, observations, medication rounds, lab contributions, chart reviews, corrections, discharges) built from official CKM templates with seeded determinism, so both servers receive byte-identical requests; latencies are coordinated-omission-corrected from each request’s planned arrival instant, and the ladder bisects to the last rate held inside the envelope. The full method chapters: Conformance · Benchmarks.