Operations

Running a clinical data repository in production means more than starting the binary: the database must be backed up and least-privileged, traffic must be encrypted, upgrades must be safe while the service stays up, and you need to see what the system is doing. This chapter is a production checklist — database roles, TLS, backup and point-in-time recovery, upgrades and migrations, observability, health probes, and the management surface — drawn from how the container image and Helm chart are built to run.

• Database roles and least privilege
• Applying migrations
• TLS and database security
• Backup and point-in-time recovery
• The container image and pod hardening
• Upgrades
• Observability
• Health probes and the management surface

Database roles and least privilege

EHRbase-rs connects to an external PostgreSQL 18 — a managed service or an operator-run cluster, never a chart-side sidecar, because a database holding PHI must be independently backed up and recoverable. The server carries only a connection string, ideally sourced from a secret.

The database uses a four-role model — never a superuser at runtime:

The migrations create these roles idempotently, apply the per-schema grants, and revoke the ability to create objects in the public schema. The running server connects as ehrbase_app — its DSN should authenticate as that role, not the migrator or the owner.

Applying migrations

The binary applies its embedded migrations on boot, so you choose how migrations run:

• Grant the runtime DSN the migrator role — simplest, for single-tenant or small deployments; the server migrates itself at startup. Least isolation.
• Run migrations out of band with a migrator DSN, then start the server with the lower-privileged ehrbase_app DSN — recommended for least-privilege production. Run the migration as a CI/CD step or a one-shot job with the migrator credential before rolling the deployment, and gate the rollout so two versions never race the schema.

TLS and database security

These are database-side settings that belong to whoever provisions PostgreSQL; the deployment references them but cannot enforce them:

• TLS in transit. Require hostssl on the server and put ?sslmode=verify-full in the DSN so the client verifies the server certificate.
• pgaudit. Run pgaudit as the database-layer complement to the openEHR audit and the ATNA trail — for example pgaudit.log = 'ddl, role, connection' globally plus object-level audit on the PHI tables — and ship the audit log to an immutable store with long (roughly six-year) retention.
• Encryption at rest. Encrypt at the volume or disk layer. Do not encrypt the stored clinical JSON with pgcrypto — it would break AQL’s ability to query inside the data.

Backup and point-in-time recovery

Enable WAL archiving and point-in-time recovery from day one (pgBackRest or a managed PITR), because a CDR’s data is not reconstructible. Clinical and audit tables are never UNLOGGED. Test your restore, not just your backup.

The container image and pod hardening

The published image is distroless and non-root — shell-less, with no package manager — and is multi-architecture (amd64 and arm64) on GHCR. When run under Kubernetes with the provided Helm chart, the pod is hardened and the chart’s validation asserts these on every render:

Egress restriction is opt-in because egress targets (database, broker, terminology server) are deployment-specific. See Installation → Kubernetes & Helm for the chart itself.

Upgrades

• Backward-compatible migrations. Migrations are append-only and never edited once applied. A rolling upgrade must be compatible with the previous schema for the window where both versions run: apply additive changes first, and defer destructive changes to a later release once every pod is on the new version.
• Lock-safe DDL. The migration runner bounds DDL with a lock_timeout and statement_timeout so a migration cannot block live traffic indefinitely; on a busy table use CREATE INDEX CONCURRENTLY and add constraints NOT VALID then VALIDATE later.
• Pin the image. Deploy an immutable tag or, better, a @sha256 digest, never latest; roll back by re-pinning the prior digest — the schema’s backward compatibility makes that safe.
• Stay available. Keep at least two replicas (or autoscaling) and a pod disruption budget so node drains and upgrades never fully interrupt the API. The default 30-second termination grace period covers the server’s short shutdown drain of the audit and event outboxes.

Observability

tracing is the single instrumentation API. From it, three signal families fan out, and identified data never enters any of them — telemetry uses only closed-set labels and opaque request/trace ids, so correlation to a patient is possible only through the ATNA audit trail.

• Logs go to stdout — JSON when not attached to a terminal, pretty on a TTY — each line stamped with the trace and span id. Shipping and rotation are the platform’s job. EHRBASE_LOG_FORMAT (auto/json/pretty) and EHRBASE_LOG_FILTER (or RUST_LOG, default info,ehrbase=info) control them, and the level can be changed at runtime through the loggers endpoint below.
• Traces export to any OpenTelemetry collector (Tempo, Jaeger, and so on) over OTLP — but only when you configure an endpoint; with none set, the tracing layer is not installed at all (zero overhead). Root spans are named by route template, never by a path containing ids.
• Metrics are exposed for Prometheus to scrape at /management/prometheus (OTLP metrics push is an option). The catalogue includes HTTP request duration and active requests, authentication failures, database pool state, AQL query counts and latency, compositions committed, validation failures, and the audit pipeline’s health.

The telemetry environment variables:

Tip

A single-container dev stack (grafana/otel-lgtm, bundling an OTLP collector, Prometheus, Tempo, Grafana, and Loki) ships as a Compose overlay, together with a provisioned Grafana dashboard (request rate/errors/duration, database pool, AQL latency, validation failures, audit health) and a starter alert pack — point the server at it with the two OTLP variables above.

Health probes and the management surface

The management surface — health, info, metrics, and runtime log control — is off by default on the bare binary, and each endpoint is independently opt-in with an access level (admin_only, private, or public). It can be bound to its own internal port so it never appears on the public API listener. The Helm chart turns the health probes on by default because they carry no PHI.

The probe and ops endpoints:

Note

Under Kubernetes, wire liveness and startup to the liveness route and readiness to the readiness route; readiness reports 503 when the database is unreachable or migrations are not applied, so a pod is only sent traffic once it can serve. If you keep the management surface off, the container’s ehrbase healthcheck subcommand can back an exec probe instead. The public /rest/status product endpoint remains available regardless.

For the full list of configuration keys across every subsystem, see Installation → Configuration reference; to explore the API itself, open the API reference at /ehrbase-rs/api/ (also linked from the toolbar on every page).