Clarula Architecture Documentation

The following table describes each communication partner and the data exchanged.

Inside the system boundary (einfache-eRechnung.de):

Outside the system boundary:

The following table maps business-level data flows to their technical channels.

Users send their PDF invoice to an EER email address. Mailgun receives the email and forwards it as a webhook to the application. The application extracts invoice data, generates the ZUGFeRD e-invoice, and replies to the sender’s email with the result attached.

This removes the biggest adoption barrier for the target audience: small business owners and sole proprietors who already produce PDF invoices in Word or their trade software. They change one email address instead of learning new software.

The web UI handles corrections and account management only — tasks that need a form-based interface.

Traces to: usability goal, vision statement ("you only change the email address").

For supported native-text PDFs, EER uses local Xberg interpretation as the evidence source. XbergPdfStructuredTextExtractor is the PDF boundary, and ADR-17 keeps layout-aware document evidence local. PDF is the only supported input format; scanned or image-only PDFs fail closed.

The implemented stages are:

This supersedes the previous sequential DocumentSemantics + ExtractedInvoice chain and removes code-side layout interpretation and monetary reconciliation. The EER model keeps one value per invoice concept; the PDF remains immutable evidence. Internally inconsistent source PDFs surface as validation findings and default to source correction or explicit review. The editor is reserved for genuine recognition or mapping errors.

Trade-off: semantic extraction remains non-deterministic, every extractor prompt needs its own benchmarking, and scanned or image-only PDFs remain out of scope and fail closed.

Traces to: usability goal (low-friction conversion for supported documents), modifiability goal (format-neutral pipeline without template maintenance).

The backend remains one BootJar and deployment unit, while Gradle modules, servlet mappings, and Spring application-context boundaries separate product code at construction and runtime. The persistence-free :app root eagerly starts sibling EER and Clarula web contexts behind one parentless bridge of explicitly shared stateless services. :eer owns the established public routes through the default / DispatcherServlet; :clarula owns /admin/clarula/, whose longer servlet mapping wins. Each product owns its DataSource, Flyway chain, JPA persistence unit, and one conventional local transactionManager; the root owns none of them and exposes aggregate product health through /actuator/. One PostgreSQL cluster keeps operations small, but distinct EER and Clarula logical databases and login roles reject normal cross-product connections. This completes ADR-30 and ADR-31 without introducing a second process.

ADR-34 establishes the semantic processing boundary inside :eer. Native evidence and specialized extraction map into EER-owned observations, drafts, and commonly validated invoice semantics. The unchanged source PDF remains authoritative evidence for issuer-visible content; semantic claims and recognition corrections retain provenance. Email, website, widget, diagnosis, and correction consume this model rather than generated ZUGFeRD XML.

Target-specific defaults, calculations, schemas, and libraries live behind EER adapters. Mustang and PDFBox remain inside the ZUGFeRD adapter, while a future EER XRechnung adapter would map the same semantics and apply independent readiness rules. The one-time destructive cutover deletes all pre-model correction sessions and removes XML-backed edit state rather than manufacturing source evidence from generated artifacts. Clarula remains isolated in :clarula and does not import the EER invoice model.

The e-invoice conversion product stores no customer master data and no invoice content beyond temporary edit sessions. All data needed to create an e-invoice comes from the source PDF. Edit sessions expire after 30 days; a daily cleanup job deletes them.

The Clarula identity-led partner demo adds one bounded exception defined by PDR: Identity-led client widget. DATEV remains the source of mandate identity and routing targets. A Clarula administrator starts a Firm-scoped DATEV authorization; the authorized firm representative authenticates directly at DATEV. Clarula encrypts the resulting tokens and verifies the DATEV business-partner identity before importing mandate data into the separate Clarula database. The EER Partner registry has no identity or persistence relationship to this Firm.

A DatevApi port isolates authorization, single-use token refresh, revocation, firm verification, and mandate import. Its deterministic mock keeps local and demo flows reproducible. One configurable HTTP adapter targets sandbox or production. Both map into the same minimized snapshot containing mandate labels, available names and addresses, available strong identifiers, provenance, and one simulated target per mandate. Firm administrators select and confirm imported mandates and maintain uploader email grants; imported mandate, identity, and target fields are read-only. Manual replacement data is outside the demo scope.

At submission start, Clarula copies the selected mandate’s required current import values and provenance into an immutable per-submission mandate context. Classification uses that capture only to prove mandate and seller/buyer role before automatic handover, so a later re-import cannot change an in-flight or historical assessment. An incomplete snapshot does not block invitation or upload, but every affected invoice fails closed to firm review and cannot call the DATEV demo port. The imported records never pre-fill or rewrite invoice content.

This keeps recurring invoice fields and general customer-master-data editing outside Clarula while accepting the smaller privacy, provenance, and lifecycle surface required for conservative routing. ADR-29 fixes the OAuth lifecycle, adapter boundary, and same-source re-import semantics; ADR-32 supersedes only its ownership anchor from EER Partner to Clarula Firm. Automatic synchronization, source replacement, and production document handover remain later decisions.

Trade-off: the partner demo stores sensitive routing snapshots and may route more invoices to review when imported DATEV data is incomplete. Users still cannot pre-fill recurring invoice fields; each PDF remains self-contained.

Traces to: data privacy goal, operability goal (bounded state), conservative routing requirement.

ZUGFeRD e-invoice generation uses two libraries in sequence:

Mustang’s built-in PDF/A-3 conversion re-renders the entire PDF, which corrupts fonts in many real-world invoices. The hybrid approach preserves the original PDF byte-for-byte and only adds the XML attachment.

GhostScript handles PDF/A-3 color profile conversion when needed.

Traces to: compliance goal, usability goal (preserves original visual appearance).

All invoice data processing happens within the EU:

The migration from OpenAI to Mistral was driven by documented customer concerns about US data processing and CLOUD Act exposure (see PDR: LLM Provider Migration).

Spring AI’s provider abstraction made this a configuration change rather than a code rewrite.

Traces to: data privacy goal, compliance with GDPR.

When validation fails, the system does not silently drop the invoice or return a cryptic error. Instead:

This turns validation failures from dead ends into guided e-invoice readiness checks with a bounded manual-correction fallback.

Traces to: usability goal (actionable errors), compliance goal (invalid invoices never ship), product positioning (explain source-invoice problems without replacing the user’s invoicing tool).

The backend deploys as the backend Spring Boot modular monolith containing all business logic, including registration and DOI. Two independently generated static Nuxt product frontends live under applications/frontend/, with partner demo pages under demos/partner/. The application keeps one process and one deployment unit while ADR-31 supersedes ADR-5’s one-database clause with separate EER and Clarula logical databases and roles in one PostgreSQL cluster. Static frontends, widget bundles, and demos do not constitute a microservices architecture.

For a small team, splitting business logic across services would add operational overhead (networking, deployment orchestration, distributed tracing) without proportional benefit. The package-by-feature structure keeps modules decoupled within the backend monolith. Static frontends consume the explicit /api/v1/ OpenAPI contract and contain no business logic. Splitting the backend into separate services remains possible if scale demands it.

Nuxt is build-time tooling only. Each product image combines one immutable generated release with product-owned unprivileged Nginx. Caddy proxies same-origin /api/v1/ plus an explicit allow-list of backend-owned widget, webhook, actuator, and compatibility paths before forwarding remaining traffic to the matching frontend container. The dedicated operator origin serves its static frontend and proxies only the explicit /api/operator/v1/ owners, including Clarula callbacks. Dynamic diagnosis and correction navigation uses two dedicated client-shell files restricted to HTML GET/HEAD and one token segment. Generated fixed application documents are matched exactly; Clarula has no generic auth fallback. Unknown content, assets, and application paths therefore retain real 404 semantics. Unknown canonical API operations remain backend-owned: authenticated requests receive the backend 404, while unauthenticated requests may receive the API security boundary’s 401 first; neither case can fall back to frontend HTML.

Each frontend, backend, Caddy, and demo keeps an independent immutable image version. Dev recreates only selected components; production promotes the complete runtime. A brief product-specific interruption during frontend replacement is accepted.

Code is organized by business capability, not by technical layer:

Each package owns its controllers, services, repositories, and domain types. No cross-package database access. infrastructure is the only horizontal package.

ADR-30 defines module ownership; ADR-34 refines EER invoice processing. PDF/LLM analysis lives under :eer package invoice.conversion.analysis.pdf; formats.zugferd contains only target mapping, readiness, validation, and output generation. The architecture still "screams" its domain: a new developer sees product boundaries, invoice conversion, target formats, routing, and email capabilities before JpaConfig or SecurityConfig.

The pricing model uses time-based access (30-day free trial, then flat-rate subscription) rather than volume limits. All tiers produce identical, EN 16931-compliant e-invoices.

Volume-based pricing penalizes high-volume businesses and lets low-volume users stay free indefinitely — neither outcome aligns with the product’s value delivery. See PDR: Tiering v2 for the full rationale.

The release flow now treats dev as the automatic proving ground and production as an explicit promotion step. Every merge to main deploys to the permanent dev environment. Production deploys are triggered manually after the change has been observed in dev.

This protects customer experience without introducing a heavier branching or release-management model. The same application topology and Spring production profile run in dev, so the team validates real deployment behavior before shipping to customers.

Trade-off: one more human step exists between main and prod. This is intentional friction.

Traces to: operability goal, customer-quality protection, small-team operations.

The system separates by deployment independence and technology fit:

PostgreSQL serves the backend exclusively. The sites and demos are stateless.

The four-module decomposition makes product ownership enforceable without introducing another process or deployment unit. :eer and :clarula cannot depend on each other, and :core cannot depend on either product. Gradle checks enforce the source dependency direction; disjoint Spring scans and sibling contexts enforce bean visibility at runtime.

:core is a library, not a standalone application context. :app copies an explicit set of its stateless services into a parentless shared-services context, which becomes the parent of both product contexts. The root and bridge expose no repository, entity manager, Flyway instance, DataSource, or transaction manager. Each product context owns its persistence infrastructure and exactly one local transactionManager, as required by ADR-31: Separate Logical PostgreSQL Databases and Roles.

Package-by-feature from ADR-6: Package-by-Feature Organization remains the organization within each product module. The Level 3 whiteboxes refine one module at a time so package responsibilities and dependencies remain readable.

:app separates process-wide composition from product behavior. Its component scan is restricted to com.clarula.app, while the runtime package starts each product through an explicit module entry point. The root disables product web, mail, template, AI, task, and persistence auto-configuration so it cannot become a third business or persistence module. This realizes the composition boundary defined by ADR-30.

:core contains only capabilities that both products can share without sharing domain or persistence state. The module remains deliberately small: implementation convenience alone does not justify moving code into it. Keeping product entities, repositories, migrations, DataSource configuration, and transaction boundaries out of :core prevents the shared module from becoming a coupling point. These constraints follow ADR-30.

The decomposition follows business capabilities rather than technical layers, as required by ADR-6: Package-by-Feature Organization. A developer sees email.ingestion, formats.zugferd, invoice.editing, identity, and widget before framework configuration. infrastructure remains the only horizontal package.

The sibling context scans only EER-owned packages and depends on the stateless contracts in :core. It owns one EER DataSource, Flyway history, JPA persistence unit, and local transactionManager; no Clarula bean, entity, repository, or migration participates in EER injection or persistence. Both siblings share one process environment and classpath, but product-local configuration selects only the owning database credential and resources. This applies ADR-30 and ADR-31.

The Firm aggregate anchors Clarula tenancy and all DATEV-owned state. It is independent of EER’s Partner registry, so a partner lifecycle or identifier cannot change Clarula authorization or mandate ownership. This applies ADR-32: Clarula Firm as DATEV and Tenant Anchor.

Clarula keeps provider integration behind DatevApi and owns a local persistence boundary. Its sibling context scans only com.clarula.clarula, has one Hikari DataSource, one Flyway history, one JPA persistence unit, and one local transactionManager. It has no compile-time or persistence dependency on :eer, as required by ADR-30 and ADR-31.

Each component has an independent image version. All three frontend containers connect only to an internal edge network shared with Caddy; they cannot reach PostgreSQL or publish host ports. Each rollout step recreates one Compose service with --no-deps; a production operation sequences all image services and applies Caddy last. Replacing a frontend can briefly interrupt that product; Caddy, the backend, and the other frontend keep running during that step.

The former app origin remains only because partner sites have already installed its versioned widget URL. It is not a first-party application origin and can be removed after those partners migrate. Canonical APIs, compatibility APIs, and RFC 8058 subscription posts use the product or api.* origins.

Production promotion is a manual full-runtime workflow operation. After every successful Dev smoke suite, the Dev workflow publishes one certified snapshot containing the seven running OCI digests and their individual source SHAs. The default production dispatch freezes runtime-snapshot:dev and promotes all seven exact digests without rebuilding any runtime component. The production guard resolves the selected snapshot to one immutable digest before mutation, so a later Dev deployment cannot change the promotion source. After production approval, the workflow validates that the snapshot’s immutable cohort tags still resolve to the recorded digests.

The target pulls each component by its recorded digest, uses the cohort ID as a local rollout reference, and deploys all image components with Caddy last. It then verifies health, public routing, and each frontend’s recorded source release. After smoke tests pass, all seven packages and the runtime snapshot receive the same immutable prod-<UTC timestamp>-<run-id> release tag; package-local prod tags then move to those digests. A failed check stops the workflow without automated restoration. Operational recovery uses the same promotion path with a corrective certified Dev runtime. See deployment/FRONTEND_RELEASES.md for the operational sequence.

All public product and API origins import the same named per-IP zones. The temporary widget origin imports those zones as well but exposes only widget bundle and API paths. Canonical and compatibility paths appear in one matcher per budget, so switching product host or API alias does not reset the budget.

Caddy returns a no-store RFC 9457 429 rate_limit_exceeded problem before Spring Boot when a budget is exceeded. Application-level outbound provider limits remain separate.

Every full environment model contains an idempotent product-db-init one-shot service and executes the single repository-managed deployment/scripts/product-database-init.sh implementation. Hosted workflows install that script at /opt/apps/invoicex/scripts/product-database-init.sh before provisioning. Deployment starts PostgreSQL, runs this service to completion, and only then starts or replaces the backend. The job uses the operational invoicex superuser solely to:

The runtime environment renders DATABASE_USER=eer with its dedicated EER_DB_PASSWORD, plus a separate CLARULA_DB_URL, CLARULA_DB_USER=clarula, and CLARULA_DB_PASSWORD. Rendering rejects any equal pair among the EER, Clarula, and operational passwords. Compose passes only an explicit application-variable allowlist to the backend, so the process receives both product credentials but not POSTGRES_PASSWORD or the redundant bootstrap password names. Neither child persistence configuration receives the sibling credential or DataSource. deployment/tests/product-database-isolation-test.sh executes the provisioning model and proves both denied sibling connections.

Only PostgreSQL and Caddy ACME/config state are persistent:

Frontend images are reproducible artifacts in GHCR and are not customer-data backups. The host stores no frontend files or content bind mounts. Host replacement restores PostgreSQL/Caddy state and pulls the component images recorded in runtime configuration and the backup manifest. Production full-state archives contain independently identifiable postgres/invoicex.dump, postgres/clarula.dump, and postgres/metabase.dump files. The EER and Clarula custom-format dumps can be inspected and restored independently; restore commands create each product database with its own owner and never restore one product into the sibling database. Mailgun and Mistral are EU endpoints; New Relic receives OTLP telemetry; GHCR stores all deployable image components. Secrets are rendered from the environment-scoped 1Password vault into a mode-0600 .env file during deployment; each long-running service receives only its explicit Compose environment subset. The production workflow additionally requires the GitHub Environment’s INVOICEX_ADMIN_EMAILS, fixes OPERATOR_ORIGIN to exactly https://admin.clarula.de, and derives the DATEV callback from that validated origin. Compose passes both values explicitly to the backend without printing the allow-list. OpenTelemetry sanitization redacts signed-URL fields and OAuth/OIDC query names including code, state, token, error, error_description, error_uri, nonce, and code_challenge before export.

Every dev host sends X-Robots-Tag: noindex, nofollow. Product route ownership, scoped shells, backend exceptions, cache policy, bounded legacy-widget compatibility, and rate-limit zones match production.

deployment/environment/dev/ provisions the server, firewall, attached volume, and DNS through Terraform. cloud-init.yaml installs Docker and Python, creates the deploy user, prepares /opt/apps/invoicex, and mounts /mnt/data. The host is replaced from code rather than patched manually.

Stateful data survives replacement on the attached volume:

Frontend images are immutable, reproducible delivery artifacts in GHCR. The host stores no frontend files; a replacement host pulls the selected image tags through the component workflow.

The dev workflow selects components by changed path. An app-local change packages and activates only that frontend; shared frontend package, generation-script, or lockfile changes select all three. Backend, Caddy, and both partner demos use independent dev-<full-sha> image tags. A frontend deployment recreates only that product’s Nginx container; it does not restart the other frontend, Caddy, or the backend. A brief product-specific interruption is accepted.

The workflow fetches secrets from the invoicex_dev 1Password vault, requires the environment-owned operator allow-list, renders the exact https://admin.dev.clarula.de operator origin and its derived DATEV callback into the mode-0600 .env, preserves unselected component versions, copies the complete environment-owned Compose model and scripts, applies only selected components, and runs production-like route and exact-release smoke checks. The smoke suite POSTs the enumeration-safe operator magic-link endpoint with the exact operator Origin and a reserved address that rendering forbids from the allow-list, so no provider email is sent. After success, it records the seven running immutable tags and OCI digests in one runtime-snapshot:dev-run-<run-id>-<attempt> image. All seven components receive the snapshot’s immutable dev-run-<run-id>-<attempt> cohort tag, and package-local dev tags point to the corresponding certified digests; runtime-snapshot:dev moves last and identifies the authoritative certified combination. A failed deployment or smoke test does not replace that pointer. A manual all deployment bootstraps a replacement host.

Dev is isolated from production by VPS, DNS, GitHub Environment, database, Caddy state, component versions, and telemetry environment name. Caddy uses the same promoted image digest in both environments, but Dev Compose selects the image’s validated /etc/caddy/Caddyfile.dev configuration while production selects Caddyfile.prod. Mailgun account/domain remain shared, with postfach-dev@einfache-erechnung.de routed only to dev. OpenTelemetry URL sanitization redacts OAuth/OIDC query names including code, state, token, error, error_description, error_uri, nonce, and code_challenge before export. Because main auto-deploys, dev is not a stable long-lived QA snapshot.

The final Build & Test, Visual Regression Tests, and Full-stack E2E jobs always run and own the required status contexts. Skipped selective jobs count as success; failed or cancelled selected work fails the gate.

The dev workflow’s path selector treats these as independent components:

A shared frontend package, generation script, API client, workspace manifest, or lockfile selects all three frontend builds. Each frontend image contains only .output/public/, exact release metadata, and its product-owned unprivileged Nginx configuration. CI verifies non-root/read-only execution, scoped shells, privacy and cache headers, real 404 behavior, and absence of Node.js. The release ID fixes Nuxt’s build ID and identifies the source bytes inside the image.

All deployable components use immutable dev-<full-sha> tags in separate .env variables. Build jobs reuse an existing full-SHA tag rather than overwriting it. The deploy job preserves every unselected version and updates only selected services. Caddy packages both validated environment configurations; each Compose model selects its own configuration from the same image digest.

After the complete Dev smoke suite passes, the workflow resolves all seven running OCI digests and publishes runtime-snapshot:dev-run-<run-id>-<attempt>. It adds the same immutable dev-run-<run-id>-<attempt> cohort tag to every component digest, moves each package-local dev tag, and moves runtime-snapshot:dev last. That single snapshot pointer is the authoritative production input; a failed deployment, smoke test, or certification cannot replace it.

The production workflow is workflow_dispatch only and is additionally constrained by the prod GitHub Environment and a workflow guard to main. There is no component selector: every dispatch promotes the complete certified runtime. A blank input resolves runtime-snapshot:dev; an operator can select a retained immutable dev-run-<run-id>-<attempt> snapshot.

The production guard resolves the selected snapshot to one immutable digest before mutation, so later Dev delivery cannot change the promotion source. The workflow validates the successful Dev certification and main-reachable deployment-control revision before approval, then revalidates the immutable cohort tags after approval. It builds no runtime image and creates no intermediate candidate aliases. The target pulls each component by its recorded digest and deploys backend, all three frontends, both demos, and Caddy in that order. The image deploy script verifies each component, then public smoke tests verify the complete route contract and all three recorded frontend source releases.

After smoke tests pass, every component package and the runtime-snapshot package receive the same immutable prod-<UTC timestamp>-<run-id> tag. The production host records that shared tag and package-local prod pointers move to the active digests. A failed check stops the workflow; operators recover through a new promotion of a corrective certified Dev runtime.

Detailed commands and retention rules are in deployment/FRONTEND_RELEASES.md.

The engineering-pages.yml workflow publishes a static engineering observatory to the gh-pages branch. Changes under docs/architecture/ or docs/pages/ render the complete arc42 document with its diagrams and update the landing page automatically from main without running the paid benchmark. A manual dispatch additionally runs the real LLM benchmark with the development Mistral credential and retains all historical benchmark runs. The landing page is served at the GitHub Pages root, architecture documentation at /architecture/, and benchmark reports at /llm-benchmark/. The workflow migrates the former root-level benchmark site into that namespaced path on its first publication.

Both product origins proxy /api/v1/, the embeddable-widget bundle, webhooks, actuator endpoints, and explicit backend-static exceptions to the same backend. They return 404 for /admin/ and /api/operator/. Only http://admin.localhost proxies the explicit /api/operator/v1/auth/, /eer/, and /clarula/ namespaces; remaining requests reach the operator Nuxt development server. Unknown product and operator routes remain 404 responses.

The script builds and mounts applications/backend/build/libs/backend.jar, starts Gradle bootJar --continuous, and restarts only the backend when the JAR changes. The three Node 26.5 containers install the pinned workspace dependencies into separate named volumes and run Nuxt dev servers with source mounts. Ctrl+C stops the Compose project.

Backend JDWP remains available on port 5005. PostgreSQL is exposed on 5432, optional Metabase on 3000, and optional OpenTelemetry Collector ports on 4317/4318. Before the backend starts, product-db-init creates local eer and clarula roles, provisions the clarula database, revokes sibling connection access, and performs the idempotent ownership transition for historical EER objects. The local backend uses eer for invoicex and clarula for clarula; the development passwords are non-secret Compose defaults. Local runtime configuration sets both INVOICEX_PRODUCT_ORIGIN and INVOICEX_BASE_URL to http://localhost, while OPERATOR_ORIGIN is http://admin.localhost. The local backend receives an explicit non-secret INVOICEX_ADMIN_EMAILS=admin@example.com allow-list. DATEV_ENVIRONMENT=mock is the default: /clarula/firms in the operator app creates a Clarula-local firm and executes the complete connection, encrypted-token, import, and confirmation path, while authorization consent and mandate data remain local and deterministic. DATEV_REDIRECT_URI is http://admin.localhost/api/operator/v1/clarula/datev/callback, derived from the same canonical local operator origin used by hosted environments. DATEV Sandbox Validation describes the credentialed HTTP-adapter check.

applications/visual-tests/update-screenshots.sh keeps screenshot generation inside the pinned Playwright container. A normal update or check builds a production-like backend runtime plus the static frontend and custom Caddy artifacts, creates fresh project-scoped Compose state with both product databases, runs Playwright, and tears the stack down. The explicit --up, --reuse, and --down lifecycle retains those containers and build outputs for repeated local screenshot updates. Reuse is rejected when runtime source inputs or built artifacts changed after --up. Developers run a normal hermetic --check after incremental updates. Screenshot updates select only the visual specifications; checks can select visual and non-screenshot contract specifications independently.

For a confidential DATEV sandbox application, register the exact Dev callback URI https://admin.dev.clarula.de/api/operator/v1/clarula/datev/callback (production uses https://admin.clarula.de/api/operator/v1/clarula/datev/callback) subscribed to master-data:master-clients. Start the backend with:

The requested scopes are openid, datev:master-data:master-clients:read, and datev:master-data:org-info:read. Do not add document-write permissions to this validation. Hosted deployments read DATEV_ENVIRONMENT and environment-specific 1Password field references from GitHub environment variables. Dev permits mock or sandbox; production permits mock or production. The deployment workflow resolves those references through its existing 1Password service account only for sandbox or production; unset credentials are accepted only while the explicit mode is mock. Hosted rendering always derives DATEV_REDIRECT_URI from the validated environment-specific OPERATOR_ORIGIN. In mock mode, Compose omits the four credential/encryption values instead of injecting empty strings while still passing that canonical callback. The runtime renderer rejects absent or weak external encryption material and any separately supplied callback that conflicts with the derived value. Secrets belong in the environment secret store and never in repository files, shell history, screenshots, or issue comments. Only the backend receives the rendered DATEV values; PostgreSQL receives an explicit database-only environment. OpenTelemetry URL sanitization redacts OAuth/OIDC code, state, token, error, error_description, error_uri, nonce, and code_challenge query values before traces are exported by hosted deployments.

The application does not persist adapter-mode provenance and does not support an in-place transition between mock, sandbox, and production. Every DATEV_ENVIRONMENT change therefore requires a destructive reset of the complete Clarula database. This removes all Clarula firms, OAuth tokens, imported mandates, routing identities, targets, confirmations, and authorization attempts; EER remains untouched.

Run the transition as one supervised maintenance operation. Record the old mode, target mode, cutover time, and operator, and prevent concurrent deployments or backups. On production, acquire the shared operation lock from /opt/apps/invoicex and keep that shell open through the database reset:

A Clarula dump created before the recorded cutover must never be restored into the new mode. It may be used only for a complete rollback with its matching runtime configuration; otherwise restore the other product data selectively and recreate Clarula empty. A rollback to another adapter mode repeats the same revocation and reset procedure. Any local or ephemeral database that applied the unreleased earlier checksum of Clarula migration V1 must also be recreated once, even when its adapter mode does not change.

Record only pass/fail, environment, immutable application revision, DATEV request correlation identifiers, mandate counts, and sanitized internal error codes. Do not record tokens or complete provider payloads.

Production later uses DATEV_ENVIRONMENT=production, production app credentials, a production callback URI, and separate encryption material. The Clarula Firm domain model, isolated database, child-context route, and DatevApi contract do not change. Before switching, follow the complete clean-state procedure above; revoking existing external grants is required but does not replace the full database reset.

The model is a non-JPA domain model owned by :eer package invoice.conversion:

DocumentAnalysisResult represents invoice candidates with findings, safe non-invoice classifications, accepted but automatically unreadable documents, permanent source rejection, and transient processing failure. EER target adapters apply independent mapping and conformance rules. Every correction session persists a versioned, integrity-checked draft snapshot; the one-time destructive cutover deletes all pre-model sessions rather than reconstructing missing source evidence. Snapshot compatibility is relational metadata checked before JSON binding. Obsolete uncorrected drafts can be reconstructed from the stored PDF outside a database transaction and installed only by a lifecycle-, TTL-, hash-, schema-, revision-, and version-guarded compare-and-swap, while completed downloads no longer require a draft reader.

The system uses persistent entities for EER identity, partner attribution, edit-session state, and the independent Clarula firm/DATEV model. EER entities extend BaseEntity; Clarula entities compile in a separate module and extend ClarulaBaseEntity. Both conventions provide UUID primary keys, optimistic locking via @Version, and automatic audit timestamps without creating a shared persistence type.

The diagrams are separate because there is no cross-product entity relationship, foreign key, shared persistence superclass, or persistent identifier mapping. In particular, an EER Partner and a Clarula Firm are independent records even when they represent the same real-world tax firm. This separation follows ADR-31 and ADR-32.

Every EER JPA entity extends BaseEntity; every Clarula JPA entity extends ClarulaBaseEntity. Keeping two equivalent product-owned base classes prevents a persistence dependency through :core while enforcing the same four properties in each persistence layer:

Aggregate roots use companion factories where creation must enforce lifecycle defaults; provider snapshot entities may be constructed directly inside their owning services. JPA’s no-arg construction remains an internal persistence concern. See the Persistence section for a code example.

The system models expected outcomes as sealed classes. Kotlin’s exhaustive when expressions force callers to handle every case at compile time.

Two primary result types drive the processing pipelines:

This pattern provides compile-time safety: adding a new outcome variant causes build failures everywhere that when expression does not handle it yet.

BusinessEvent is the abstract base for all domain-significant occurrences. Each event carries a timestamp, correlation ID (from MDC or generated), and optional user email. The event type derives automatically from the class name.

Events flow through Spring’s ApplicationEventPublisher to three independent sinks:

This fan-out design keeps event producers decoupled from consumers. Adding a new sink (e.g., webhook notifications) requires only a new @EventListener — no changes to existing code.

Each product owns an independent Flyway history in a separate logical PostgreSQL database. Hibernate validates each product persistence unit against its own database and never modifies either schema (ddl-auto: validate). A Clarula migration cannot resolve an EER table, and the database roles are denied connection to the sibling database. Failure of either required Flyway chain or either Hibernate validation fails startup of the shared backend process.

Database runtime: one PostgreSQL 18 cluster with logical databases invoicex (EER), clarula, and metabase.

The product databases have distinct schemas, histories, ownership roots, and foreign-key graphs:

accounts is the central EER identity entity. Both login_tokens and invoice_edit_sessions reference it via foreign key with ON DELETE CASCADE. firms is the independent Clarula tenant root; all current Clarula DATEV state reaches one firm exclusively through Clarula-local foreign keys. There is no cross-database foreign key, join, view, foreign data wrapper, or shared identifier mapping between partners and firms.

The EER analytics schema is write-only by design. Application features never read from analytics.business_events — it serves observability and reporting only. An ArchUnit test (AnalyticsBoundaryArchTest) enforces this boundary.

For column-level details, see the product-owned Flyway migrations — they are the single source of truth.

Compose starts an idempotent product-db-init job before the backend. The operational PostgreSQL superuser creates the eer and clarula login roles, creates the Clarula database, migrates ownership of existing EER relations, routines, enum/domain types, and schemas from the historical superuser to eer, and revokes public database connection privileges. Among application roles, only eer receives CONNECT on invoicex and only clarula receives CONNECT on clarula; the operational superuser retains explicit cluster access. The backend receives both credential sets because it remains one process, but each set is bound only inside its owning application context. The operational superuser remains available to provisioning, Metabase, and backup automation and is not configured as either product DataSource.

EER migrations live at eer/src/main/resources/db/migration and Clarula migrations at clarula/src/main/resources/db/clarula/migration. Each eager sibling context creates and runs only its local Flyway bean; the persistence-free root and shared-services bridge expose none. Deleting historical EER migrations is prohibited; the removed unshipped DATEV V121 never becomes part of the EER chain. The two flyway_schema_history tables and version sequences remain independent.

BaseEntity uses @Version to detect concurrent updates. Spring Framework 7 retry support is enabled through @EnableResilientMethods. Retry-safe transaction entry points use the composed @RetryableTransactional annotation when a concurrent update can be resolved by reading the latest state and repeating the operation. It combines Spring’s @Retryable with @Transactional(propagation = REQUIRES_NEW), limits retries to one for recognized Spring, JPA, or Hibernate optimistic-locking exceptions, and uses no retry delay. The retry advice surrounds the transaction advice, so every attempt gets a fresh transaction and persistence context. Retry must not be applied globally or around irreversible side effects such as sending email or calling an external API.

EER entities extend BaseEntity; Clarula entities extend the compile-time-independent ClarulaBaseEntity with the same UUID, version, and audit semantics. Domain-created entities normally use companion object factory methods:

This pattern keeps entity construction logic centralized and supports JPA’s no-arg constructor requirement without exposing it to callers. Repository-level atomic inserts are reserved for database-enforced claims such as single-use token consumption with ON CONFLICT DO NOTHING.

EER uses passwordless authentication via magic links. No passwords exist in the system.

Customer API operations use /api/v1/auth/magic-link, /api/v1/auth/verify, /api/v1/auth/session, and /api/v1/auth/logout; the pre-existing operations retain /auth/ compatibility aliases. Magic-link emails use eer.product.origin, successful verification redirects to /portal, and token errors redirect to product-owned /auth/error/ routes. Backend-rendered customer login, portal, error, and plan routes remain available during migration.

A DATEV connection belongs to one active Clarula Firm; it never authenticates a Clarula customer or administrator and never references an EER Partner. The Clarula administrator only initiates assisted onboarding. A firm representative enters credentials and grants consent directly at DATEV.

See ADR-29 and ADR-32.

A partner widget identity session is a bearer capability for one operation: send a widget conversion owned by the same partner to the fixed email of the bound account. It is deliberately separate from customer and admin authentication.

The operator API (/api/operator/**) uses a completely separate authentication pipeline from the customer portal. https://admin.clarula.de is the only operator origin; all product-origin /admin and /api/operator requests return 404 at Caddy.

All JSON and multipart operations called by product frontends or embeddable widgets have canonical paths below /api/v1/. The version identifies the first major browser contract, not an implementation release. Compatible changes within v1 are additive. A breaking change requires a new major path and a migration period in which both majors remain available.

The product-v1 contract includes authentication, customer session, portal summary, registration, contact, plan, trial-email subscription, free-upload, diagnosis, correction, frontend operational reporting, and standard-widget operations. It excludes backend-rendered HTML, admin routes, webhooks, actuator endpoints, and the runtime OpenAPI endpoints themselves. The diagnosis and correction APIs use the existing edit-session token as a capability. The einfache-eRechnung Nginx container serves only the two scoped static client shells for /diagnosis/{token} and /edit/{token}; Caddy keeps the API routes backend-owned and no customer-facing backend HTML route remains. Partner-affiliation confirmation remains a backend route until its owning UI flow migrates. Registration and magic-link confirmations remain backend-handled token operations, but their canonical v1 handlers redirect to product-owned result routes.

The widget script URL is an embed contract, not an API alias. /widget/v1/embed.js remains stable while its internal requests use canonical API paths.

The pre-production DATEV spike operations under /api/v1/datev-widget/** were retired as an approved reset before implementation of the Clarula partner demo.

Springdoc builds the product group from Spring MVC handlers marked as product API controllers and filters it to /api/v1/**. The Gradle generation task boots a test application with Testcontainers, requests the grouped document, sorts every object key, and writes a deterministic JSON document. It does not call a deployed or production backend.

checkProductV1ApiCompatibility uses OpenAPI Diff and fails on incompatible v1 changes. It accepts OPENAPI_BASELINE_FILE for an explicit baseline, otherwise compares with the merge-base selected by GITHUB_BASE_REF or the parent commit. The first commit that introduces the contract has no earlier baseline and therefore compares the generated document with itself.

A v1 change is incompatible when it removes a path, operation, accepted input, response, or schema value; narrows an existing type; or makes an existing optional request field mandatory. New optional fields, new response fields, and new operations are additive. Compatibility aliases remain outside the product-v1 document so they cannot become the source contract accidentally.

applications/frontend/packages/api-client/ generates OpenAPI path and schema types from the committed contract and wraps them with openapi-fetch. Both Nuxt applications depend on this package, and workspace type-checking compiles it with each product. Generation and drift checks use only repository files:

Generated files are committed so product builds do not depend on a running backend. A backend contract change is complete only when the OpenAPI document and generated TypeScript source are updated together. The generated correction schema contains the complete current editor field graph, including repeatable line items, parties, delivery, references, allowances, charges, tax breakdowns, payment data, and attachment metadata. Frontend code must import these generated transport types instead of declaring parallel diagnosis or correction DTOs.

Framework, security, and unexpected failures use application/problem+json with RFC 9457 fields. The documented Problem schema adds a stable code; Caddy emits the same representation with rate_limit_exceeded when it rejects a request at the edge. Expected domain outcomes that already use typed response variants, such as an invalid PDF, invalid unsubscribe token, or expired widget claim, retain their documented status-specific JSON bodies during the compatibility window.

Edit-capability operations distinguish malformed (400, edit_token_invalid), missing (404, edit_session_not_found), expired (410, edit_session_expired), completed (409, edit_session_used), and unconfirmed editor entry (409, editor_confirmation_required) states. Invalid correction submissions use 422 correction_validation_failed and extend the problem document with the effective semantic form and structured rule, field, ambiguity, and evidence information. Result downloads return 409 correction_not_completed until the persisted completion marker exists.

Unauthenticated and unauthorized requests below /api/v1/ return JSON 401 and 403 responses without redirecting to an HTML login page. Canonical API responses and legacy token handoffs set Cache-Control: no-store, X-Content-Type-Options: nosniff, and Referrer-Policy: no-referrer so account data and URL tokens do not enter browser caches, content-sniffing contexts, or referrer headers. Problem instances use matched route templates for edit capabilities and therefore never echo the token. Legacy protected routes retain their redirect behavior until their consumers migrate.

The operational frontend-failure operation accepts only the generated FrontendFailureReportRequest schema. API reports must pair a registered route template with its stable operation ID; navigation reports omit the operation ID. Unknown fields, concrete capability paths, query strings, invalid statuses, and unregistered API pairs fail with 400 invalid_frontend_failure_report or the generic malformed-request problem before any operational event is emitted.

EER combines two error handling approaches:

Technical error messages from third-party libraries are not user-facing. Two components translate them into plain German:

Canonical /api/v1/** framework, security, unexpected, and edge rate-limit failures use the RFC 9457 application/problem+json representation. The response contains type, title, status, detail, and instance; backend failures and Caddy’s 429 rate_limit_exceeded response also expose a stable code for UI decisions. Authentication and authorization failures return JSON 401 and 403 responses instead of redirecting to the backend login page. Expected domain outcomes with established typed payloads remain status-specific response variants during the compatibility window. For example, unsubscribe and resubscribe return unsubscribed, resubscribed, or invalid result states without turning an invalid link into an infrastructure failure. The product-v1 OpenAPI document includes the shared problem schema and failure responses for every operation. Canonical API responses use Cache-Control: no-store, X-Content-Type-Options: nosniff, and Referrer-Policy: no-referrer. Problem instance values omit query strings; edit-capability instances additionally use the matched route template, such as /api/v1/corrections/{token}, instead of echoing the path token.

Error emails include actionable advice, not just status codes. Each error type maps to a specific email template:

The static diagnosis and correction client shells use the same token-backed InvoiceEditSession through canonical JSON/PDF APIs; no customer-facing Thymeleaf transport remains. The v1 APIs return stable problem codes for malformed, missing, expired, completed, and unconfirmed capabilities. Preview responses always contain the effective semantic form plus structured validation feedback; invalid submissions return the same information as a 422 correction_validation_failed problem. A successful submission persists completedAt; editor and diagnosis operations then return edit_session_used, while result and PDF-download operations remain available until session cleanup. For email-origin validation failures, both /edit/{token} and /api/v1/corrections/{token} reject editor access until the user explicitly confirms that the visible PDF already contains the correct information.

The EmailSender interface abstracts away the email transport. AbstractEmailSender centralizes composition logic (subjects, templates, attachments, inline resources), and two provider-specific subclasses handle transport:

Public support, invoice ingestion, and internal staff notifications use separate recipient addresses. The internal notification recipient is configured per environment, has no application autoresponder, and must not be routed to an invoice webhook. Dev and production additionally use distinct invoice inboxes.

All application-generated messages carry Auto-Submitted and X-Auto-Response-Suppress headers. Invoice ingestion acknowledges but does not answer messages marked as automated, messages from delivery-system accounts such as mailer-daemon, or messages from configured protected system mailboxes. The protected list includes both production and dev inboxes so one environment cannot start an automatic response chain with the other even when provider headers are lost. These application checks complement rather than replace exact-recipient Mailgun routes.

Response emails appear in the same thread as the user’s original email. EmailThreadingInfo carries the original Message-ID and Subject, which map to In-Reply-To and References headers on outgoing messages. Reply subjects follow the pattern Re: Original Subject — short suffix (e.g., "Re: Invoice #42 — E-Rechnung ist fertig").

All email content renders through Thymeleaf templates. GermanFormatters provides locale-specific number and date formatting for template expressions (German decimal separators, date formats).

For the partner-widget channel, the outbound email set now includes two additional privacy-sensitive messages:

The partner email is intentionally minimal: it includes only full name or company name, email address, request context, and the one-time confirmation link. It never includes invoice files, extracted invoice data, conversion results, usage history, subscription details, or account links.

The follow-up customer email confirms the approved Kanzlei by name, states that no further action is required, and repeats the privacy boundary: only name or company name plus email address were shared.

Trial reminder emails deliberately carry two links for the same signed preference token:

The backend also retains GET /email/unsubscribe, GET /email/resubscribe, and POST /email/unsubscribe for previously sent messages. Token-bearing responses are non-cacheable, suppress referrers, and never write raw tokens to logs or analytics events.

The OpenTelemetry Java agent (2.25.0) auto-instruments the Spring Boot application. All three signal types — traces, metrics, and logs — export to New Relic EU.

Business events flow through Spring’s ApplicationEventPublisher to three independent sinks. Each sink runs in its own error boundary: one sink failing does not affect the others.

Every business event carries a correlationId (from SLF4J MDC or auto-generated) and optionally a userEmail (also from MDC). This ensures trace correlation across synchronous processing, async event handling, and log output.

Operational diagnostics and product analytics have separate purposes, data contracts, and consent behavior.

Every generated product document exposes product and frontend-release meta elements. The same values are available at the fixed static path /.well-known/frontend-release.json:

Support and deployment smoke checks can therefore identify a static artifact even when its client bundle cannot start. The edge must serve this mutable pointer and Nuxt’s /_nuxt/builds/latest.json with Cache-Control: no-store; content-addressed Nuxt assets keep their immutable cache policy.

Hydrated routes report navigation failures and failed generated-client operations to same-origin POST /api/v1/operational/frontend-failures. The request uses credentials: omit, fetch referrer policy no-referrer, and keepalive. The client drops unknown routes, deduplicates identical reports for 60 seconds, and sends at most 20 reports per document from a map capped at 50 keys. Transport failure is ignored so diagnostics cannot cause another product failure.

The payload contains only these fields:

Client code never forwards a concrete URL, path parameter, query string, request or response body, exception, stack, filename, email address, or invoice content. The backend rejects unknown fields, unknown API operation/template pairs, concrete sensitive routes, invalid statuses, and oversized dimensions. It records one frontend.failure span event, one frontend.failures counter increment, and a minimized log entry. The product request sends no cookies. JwtCookieFilter and MdcLoggingFilter deliberately exclude the endpoint so a nonconforming caller cannot attach customer identity by supplying credentials.

EER integration and LLM fixtures start only the EER PostgreSQL 18 Testcontainer and product context, so product behavior can be tested without the composition root or Clarula. Composed-runtime fixtures in :app start separate worker-scoped EER and Clarula containers and bind different JDBC URLs. Each sibling context runs only its local Flyway chain, and either eager startup failure must fail the process.

MultiContextRuntimeBoundaryIntegrationTest starts the production composition on a random port and proves default EER and explicit Clarula dispatch, authenticated route exclusivity, the persistence-free root and bridge, sibling bean invisibility, separate migration histories, exactly one conventional local transaction manager per product, unqualified transactional rollback, product-local writes, scheduler/async ownership, health detail, eager failure, and both Hikari pools closing with the root process. DatevAdminIntegrationTest exercises real HTTP through the root security filter and Clarula sibling servlet, including firm lifecycle, DATEV callback session state, action tokens, encrypted token persistence, import, confirmation, firm scoping, concurrent refresh, and disconnect. Compile-time dependency verification and forbidden-source-import checks enforce app → core/eer/clarula and eer/clarula → core, with no product-to-product or product-to-:app imports.

Playwright screenshot tests run entirely in Docker for pixel-identical rendering across developer machines and CI. The DevDataSeeder creates deterministic EER data in the local profile so screenshots produce stable baselines. The Clarula DATEV visual fixture creates its independent firm through /admin/clarula/firms; it no longer discovers an EER partner row. PDF.js iframes are masked because they render non-deterministically.

EER test utilities live in eer/src/testFixtures; composition-specific two-database fixtures remain in app/src/integrationTest so product tests do not depend on the BootJar project.

deployment/tests/product-database-isolation-test.sh starts the same PostgreSQL provisioning job used by Compose, simulates historical EER-owned objects, and verifies that the eer role can migrate them after ownership transfer. It then proves both positive and negative access: each product role connects to its own logical database and PostgreSQL refuses both sibling-database connections. Compose model validation and runtime-environment rendering tests require the separate Clarula credential in every hosted environment.

Grouped configuration uses @ConfigurationProperties data classes with startup validation. A few isolated legacy URL and email values still use @Value; new grouped settings do not extend that pattern. The EER sibling scans EER property classes; the compile-time-independent Clarula sibling explicitly enables only ClarulaDataSourceProperties and DatevProperties. The root scans only composition configuration and imports product-neutral :core configuration for shared stateless services and operator JWT parsing. Root runtime-defaults.yml owns server, management, operator, and release settings. The composition imports it together with eer-defaults.yml and clarula-defaults.yml from the owning product modules; the EER test bootstrap uses the same import manifest without requiring the app or Clarula modules.

This pattern provides type safety, IDE autocompletion, and startup-time validation — a misconfigured property fails the application on boot, not at runtime.

eer.product.origin is the product/customer origin used for magic links, DOI confirmations, portal registration links, plan links, and trial-email preference links. It must be an absolute HTTP(S) origin without a path, query, or fragment. INVOICEX_PRODUCT_ORIGIN configures it and falls back to the existing INVOICEX_WEBSITE_URL during deployment migration.

backend.origin and eer.base-url retain the stable INVOICEX_BASE_URL product-origin input for EER correction/diagnosis links, partner-affiliation confirmations, and the default widget embed URL. OPERATOR_ORIGIN configures operator links, cookies, JSON APIs, and mock authorization on the dedicated operator origin: exactly https://admin.clarula.de in production, https://admin.dev.clarula.de in Dev, and http://admin.localhost locally. Hosted rendering rejects a missing, cross-environment, or noncanonical operator origin. Product origins no longer forward /admin or /api/operator paths. Partner redirect origins retain their own configuration. The former app host is never emitted by current application configuration and remains only as bounded compatibility for already-installed widgets. CLARULA_PARTNER_DEMO_ALLOWED_ORIGIN and CLARULA_PARTNER_DEMO_REDIRECT_BASE_URL supply historical placeholder names required by immutable migration V115; migration V120 removes that retired demo partner again. They can be removed only after a tested B120-or-later Flyway baseline initializes new databases and every supported database and restorable backup has already passed V115. The product-local eer.cors.allowed-origins binding retains the deployed INVOICEX_CORS_ALLOWED_ORIGINS environment name; the internal prefix rename does not change the runtime contract.

Each Nuxt build resolves one immutable release identifier at build time. FRONTEND_RELEASE_ID is the preferred input and should be the immutable deployed artifact ID or full source revision. GIT_SHA remains a compatible fallback for build environments that already provide it. Configured values must identify an immutable artifact and use 7 to 128 ASCII letters, digits, dots, underscores, or hyphens; unknown, blank values, and unsafe characters fail the build. Deployment configuration must not supply a mutable branch or ref name.

When neither variable exists, the build derives source-<sha256> deterministically from the product app, shared frontend packages, generation scripts, and workspace manifests. This content digest keeps local and test generation reproducible and prevents an unidentified production artifact. The resolved value is compiled into every HTML document and /.well-known/frontend-release.json; it is not runtime configuration and cannot change after artifact generation.

The static metadata path and Nuxt’s /_nuxt/builds/latest.json are release pointers rather than immutable assets. Each product’s Nginx configuration returns Cache-Control: no-store for both while retaining long-lived immutable caching for content-addressed Nuxt assets.

Environment variables inject secrets at deployment time. No secrets appear in configuration files or version control.

Key secrets: JWT_SIGNING_KEY, MISTRAL_API_KEY, MAILGUN_API_KEY, MAILGUN_WEBHOOK_SIGNING_KEY, EER_DB_PASSWORD, CLARULA_DB_PASSWORD, DATEV_CLIENT_SECRET, and DATEV_TOKEN_ENCRYPTION_PASSWORD.

Secret handling differs by environment today:

Both hosted vaults provide distinct product-role passwords at eer-postgres/password and clarula-postgres/password; deployment renders them as EER_DB_PASSWORD/DATABASE_PASSWORD and CLARULA_DB_PASSWORD without exposing either to the PostgreSQL container’s long-running environment.

The EER sibling binds spring.datasource. from the stable DATABASE_URL, DATABASE_USER, and DATABASE_PASSWORD variables and creates a Hikari pool named eer. Deployed configuration uses the non-superuser eer role and keeps the compatible invoicex logical database name. The Clarula sibling independently binds clarula.datasource. from CLARULA_DB_URL, CLARULA_DB_USER, and CLARULA_DB_PASSWORD and creates a Hikari pool named clarula. The production profile has no default values for either Clarula URL or credential, so a missing secret fails startup. The root and shared-services bridge bind neither product DataSource and expose no Flyway, entity manager, or transaction manager.

CLARULA_DB_MAXIMUM_POOL_SIZE and CLARULA_DB_MINIMUM_IDLE tune only the Clarula pool. The default maximum is 10 and the default minimum idle count is zero.

clarula.datev.environment selects exactly one implementation mode: mock, sandbox, or production. This is not a feature flag; the admin capability is always available. mock is deterministic and requires no DATEV credentials. The two external modes share one HTTP adapter and select environment-specific OIDC and master-data:master-clients endpoints. They fail startup when client credentials are missing or when the non-default token-encryption password is shorter than 32 characters or its hexadecimal salt is shorter than 16 bytes. Hosted workflows select the mode with the environment-specific DATEV_ENVIRONMENT variable and resolve configured *_OP_REF field references from 1Password only for external modes.

DATEV_REDIRECT_URI must match the DATEV app registration exactly. Hosted deployment derives it from the validated OPERATOR_ORIGIN plus /api/operator/v1/clarula/datev/callback; a separately supplied conflicting value is rejected. Production therefore uses https://admin.clarula.de/api/operator/v1/clarula/datev/callback, and Dev sandbox uses https://admin.dev.clarula.de/api/operator/v1/clarula/datev/callback. Sandbox and production use separate client credentials and encryption material. The adapter mode is runtime configuration and is not persisted with connections or imports. Changing DATEV_ENVIRONMENT requires revoking real provider grants, stopping the backend, and recreating the complete Clarula database first; application code does not support or clean up mixed-mode state.

The AdminProperties class (operator.admin.emails) defines which email addresses may access the admin area. This is the single source of truth for admin identity — no database table, no role column on Account.

The stable INVOICEX_ADMIN_EMAILS environment name populates operator.admin.emails. Operator authentication and both products' operator controllers are always registered; the allow-list determines who can authenticate. Hosted deployment requires the environment-owned INVOICEX_ADMIN_EMAILS GitHub Environment variable, validates it as a nonempty comma-separated list, writes it only to the mode-0600 runtime environment file, and passes it explicitly to the backend. The renderer and smoke suite never print the addresses. Adding or removing an admin requires a config change and restart. This is intentional friction for a team of three (see ADR-13).

All productive PDF-to-ZUGFeRD callers depend on invoice.conversion.PdfToZugferdConversionService. Email processing, free website upload, and widget upload traverse the same target-neutral analysis pipeline before the ZUGFeRD adapter runs. ADR-18 defines the specialized extraction strategy; ADR-34 defines the semantic handoff.

The implemented stages are:

The runtime sequence lives in Invoice Extraction Algorithm.

Xberg 1.0.0-rc.29 produces two no-OCR renderings per page. A plain-text rendering comes first so labels and address blocks omitted by layout detection remain visible; layout-aware Markdown follows so table row and column associations remain available. The application labels both as renderings of the same page, and prompts must not duplicate facts or invoice lines that occur in both. Xberg’s separate reading-order post-processing option stays disabled because it did not change Markdown output across the 34-PDF benchmark corpus. The RT-DETR and TATR layout models are revision- and checksum-pinned in both immutable backend base images, and XBERG_CACHE_DIR points Xberg to those preloaded files. CI runs a Coolblue extraction in the final backend image and reruns the native extraction regressions in the builder image, both without network access. This prevents request-time model downloads while preserving native-text-only, no-OCR behavior.

The active sequence is:

Architecture tests prohibit direct DTO-to-target mapping and keep provider, target, UI, and persistence types outside the invoice model.

The LLM extractors interpret the complete invoice context. This includes deciding which blocks are invoice lines, whether a price is VAT-exclusive or VAT-inclusive, the meaning of amount-only rows, implicit quantity and unit, credit-note signs, included transparency rows, VAT categories, settlement state, and document totals. The line extractor returns VAT-exclusive unitPrice, VAT-exclusive preDiscountUnitPrice, and lineTotalAmount as invoice semantics. The totals extractor returns one coherent set of EN 16931 totals and tax buckets.

The provider contract and application model contain one value per business concept:

The raw PDF wording remains available only through SourceEvidence. Without reliable extractor-provided offsets, claims use document-scoped evidence and INTERPRETED provenance; substring matches never become EXACT_FRAGMENT evidence. Unknown or contradictory semantics remain null, UNCLASSIFIED, or explicit findings rather than parallel candidate values.

Kotlin code may:

Kotlin code must not:

Gross-to-net conversion is therefore prompt-owned semantic interpretation. Deterministic validation may verify the resulting values but does not reconstruct a competing interpretation.

The correction form maps directly to InvoiceDraft, never through generated XML.

ZUGFeRD-only requirements belong to formats.zugferd readiness and mapping. The adapter maps BT-148 and BT-147 as a price-level allowance, maps line and document adjustments, renders CII XML, compares rendered monetary values with the semantic model, and runs XSD/Schematron validation. A missing, contradictory, or unsupported target fact produces NotReady; it is never silently defaulted or discarded to make XML validate. Other output adapters must implement their own readiness from the same ValidatedInvoice rather than consuming ZUGFeRD XML.