AI Decision Records, Model Cards and the Architecture Repository

Chapter 4 described what an AI estate should consist of; the decisions that shape any particular system within it need a record of their own. This chapter extends the Architecture Decision Record (ADR) to AI systems and shows how the record, the model card, the datasheet, and the system card together form an architecture repository from which the technical documentation demanded by Article 11 and Annex IV of Regulation (EU) 2024/1689 can be generated rather than written by hand. It builds on Architecture as Code (Chapters 4 and 22) (72, Nordqvist, 2025).

Figure 5.1: An AI Decision Record travels through the same pull-request workflow as code, gathers its companion artefacts (model card, datasheet, system card and risk register), and feeds the generated technical documentation.

Why decisions about AI need a different record

Michael Nygard's original proposal for documenting architecture decisions was deliberately minimal: context, decision, consequences, and a status, stored as short text files alongside the code (40, Nygard, 2011). Architecture as Code adopted that format and added a review workflow, YAML front matter, and review dates so that the record could be validated in a pipeline (72, Nordqvist, 2025). That template remains the right starting point, but it is not sufficient for AI, for three reasons.

First, the most consequential component of an AI system, the model, does not stay fixed once chosen: it is derived from data, retrained, and degrades as the world changes. A record that captures the decision to use a model without stating its data and the evaluation criteria it must continue to meet documents a moment rather than a commitment.

Second, the decision to deploy an AI system carries a regulatory classification with it. Under the AI Act, whether a system is prohibited, high-risk, subject to transparency obligations, or outside scope determines which obligations the organisation inherits as provider or deployer (1, European Union, 2024). AI Governance, Chapter 8 codifies the classification method as a versioned questionnaire; the AI Decision Record carries its result and links to the assessment, so that the classification and the design decision are reviewed together.

Third, AI systems have an intended purpose in a legal sense. Article 3 of the AI Act defines it as the use the provider intends, and Article 25(1)(c) turns a deployer who modifies the intended purpose of a system not classified as high-risk, so that it becomes high-risk, into its provider (1, European Union, 2024). A record that does not state the intended purpose, and the uses that are excluded, leaves the organisation unable to show what it decided.

The AI Decision Record therefore keeps the Nygard structure and adds the fields that an AI system needs. Numbering, immutable history, supersession and review through pull requests carry over unchanged.

The AI Decision Record template

Fields that extend the ADR

The provisions in the table are architectural drivers, not legal advice.

Field	Question answered	Primary driver
Intended purpose	What is the system for, in operational terms?	AI Act Article 3 definition; Annex IV point 1
Prohibited uses	Which uses are excluded even if technically possible?	AI Act Article 5; Article 25 on changes of purpose
Risk classification	Is the system high-risk, limited-risk, or minimal-risk, and why?	AI Act Article 6 and Annex III
Role	Is the organisation provider, deployer, or both?	AI Act Articles 16 and 26
Data sources	Which datasets feed training, validation, and operation, under what lawful basis?	AI Act Article 10; GDPR Articles 6 and 9
Evaluation criteria	Which metrics and thresholds must hold before and after deployment?	AI Act Article 15 on accuracy and robustness
Human oversight	Who can intervene, through which path, with what authority?	AI Act Article 14
Logging and retention	What is recorded, where, and for how long?	AI Act Articles 12 and 19
Retirement criteria	Under which conditions is the system withdrawn or the model retrained?	AI Act Article 72 on post-market monitoring

The GDPR entries refer to Regulation (EU) 2016/679 (2, European Union, 2016); Chapter 6 treats lawful basis for training data in depth.

A complete template in Markdown

The template below is the record as a reviewer reads it. It keeps the front matter of Architecture as Code and adds an AI section.

---
adr_id: AIDR-0007
title: Decision support for case prioritisation in benefit claims
status: Proposed
date: "2026-03-02"
next_review_due: "2026-09-02"
deciders: [Head of Architecture, Head of Benefits Operations]
reviewers: [Data Protection Officer, Model Risk Manager]
---

# AIDR-0007: Decision support for case prioritisation in benefit claims

Context:
- Caseworkers handle 40,000 claims a month; median waiting time is 31 days.
- A ranking model can order the queue so that likely-complete claims are
  handled first. Caseworkers retain every decision.
Intended purpose:
- Order the work queue of the benefits unit. Output is a rank, not a decision.
Prohibited uses:
- Automatic rejection or approval of any claim.
- Use of the rank as evidence in appeals.
Risk classification:
- High-risk under Annex III point 5 (access to public assistance benefits).
- Role: deployer. Provider: TriageSoft (fictional), which trains the model on
  the claims register inside the authority's platform; data never leaves it.
Data sources:
- Claims register 2021-2025 (lawful basis: public task, Article 6(1)(e) GDPR).
- No special-category data; nationality and ethnicity fields are excluded.
Evaluation criteria:
- Precision at 100 of at least 0.80 on the frozen evaluation set.
- Demographic parity difference below 0.05 across age bands and gender.
Human oversight:
- Team leaders can suspend the ranking from the operations console.
Decision:
- Proceed with a gradient-boosted ranking model served through the AI gateway.
Consequences and retirement criteria:
- Retrain quarterly; retire if parity difference exceeds 0.05 for two months.

The example belongs to Nordvik Region, the regional public authority introduced in Chapter 1. Access to public assistance benefits is listed in Annex III point 5 of the AI Act, which makes a prioritisation system high-risk even when a human makes every final decision. The record shows that classification, its basis, and the oversight path in one place, and its prohibited uses show that the authority has designed the system so that Article 22 of the GDPR, on decisions based solely on automated processing, is not engaged.

A Markdown record is readable but hard to validate, so the pipeline also keeps a structured YAML counterpart, from which the Markdown body is rendered so that the two never diverge. Appendix A gives the full AI Decision Record template and its structured form.

Worked example: choosing retrieval over fine-tuning

Illustrative example. The template is easiest to judge when it records a real choice between alternatives. Hansa Mutual, the multinational insurance group introduced in Chapter 1, had to decide how to give claims handlers reliable answers from its policy wordings. The record is stored as decisions/AIDR-0021-claims-assistant-retrieval.md; the excerpt shows its front matter and the reasoning that decided it.

---
adr_id: AIDR-0021
title: Retrieval-augmented generation rather than fine-tuning for the claims assistant
status: Accepted
date: "2026-04-20"
reviewers: [Model Risk Manager, Data Protection Officer, ICT Risk (DORA)]
classification: not-high-risk
roles: {compound_system: provider, gpai_model: deployer}
applies_in: [DE, NL, SE, NO, UK, CH]
legal_basis_by_market: {EU: AI Act and GDPR, NO: group policy until EEA incorporation, UK: UK GDPR, CH: FADP}
---
Context:
- 320 claims handlers search 3,100 pages of policy wordings across 14 product
  generations. Wordings change about once a month. Answers must cite a clause.
Prohibited uses:
- Deciding cover or settlement amounts. Use by customers. Use in life or health
  products, where Annex III point 5(c) of the AI Act could apply.
Options considered:
1. Fine-tune a general-purpose model. Rejected: monthly changes need monthly
   retraining, and citations cannot be traced to a clause.
2. Retrieval-augmented generation over a versioned index. Chosen.
3. Improve keyword search only. Rejected: handlers found the right clause in
   61 per cent of test questions, against 88 per cent with option 2.
Evaluation criteria (eval/claims-assistant/v6, 400 questions):
- Faithfulness >= 0.95; citation accuracy >= 0.98; cost <= EUR 0.04 per answer.
Consequences:
- Exception EXC-0004 records the limits of reproducibility for a hosted model.
Retirement criteria:
- Review if faithfulness falls below 0.92 in two consecutive weekly runs.

The option that lost is as informative as the one that won: recording the search-only baseline, with its measured success rate, lets a later reviewer judge whether the added cost and risk of a generative system were justified. The DORA view is recorded too, because the model provider is an ICT third-party provider and knowledge stays in the index, so changing provider means re-running evaluation, not retraining.

Worked example: designing human oversight for a high-risk system

Illustrative example. Accepting AIDR-0007 left open what effective human oversight of the ranking looks like in daily work. Nordvik Region answered it in a separate record, AIDR-0011, because the oversight design will change more often than the decision to use the model. Of three options, showing a numeric score was rejected because it invites over-reliance, and reordering the queue silently because staff cannot question what they cannot see. The chosen design shows caseworkers a priority band (high, normal, low) with the three main reasons; re-prioritising takes one click and a reason code and is never penalised; and team leaders can suspend ranking for the unit, reverting the queue to arrival order within five minutes without a change request. A random 5 per cent of claims are queued without a band, so that outcomes can be compared with and without the ranking, and an override rate outside 2 to 30 per cent per team triggers review. Band, reasons, model version and override are logged per claim for the life of the case plus the appeal period, never less than six months.

The record thus turns each element of Article 14(4) into something a person can see or do: reasons support interpretation, the band rather than a score reduces automation bias, re-prioritisation supports disregarding the output, and the suspend switch supports interruption (1, European Union, 2024). The control sample gives the authority evidence that oversight is effective rather than nominal, and residents who ask receive an explanation of the ranking's role, which supports the right to explanation in Article 86.

Records of this weight need reviewers with the authority of the decision forums described in AI Governance, Chapter 13. In the repository those forums become review rules: a pipeline reads the classification and data sources from the front matter and adds the required reviewers to the pull request, such as the data protection officer when the record names personal data and the risk function when the system is high-risk. Branch protection then refuses the merge until each has approved, so the forum's decision becomes the merge, and the discussion stays attached to the pull request.

Worked example: giving an agent write access to production

Illustrative example. The most consequential agent decisions concern write access. In AIDR-0027 Hansa Mutual let an IT-operations agent act on production; the manifest it refers to is shown in Chapter 11. On-call engineers received about 1,400 alerts a month, 38 per cent of them resolved by one of three runbook actions: restart, scale out, or disable a feature flag. The record limits the agent to those actions on named services and prohibits database writes, schema changes, data deletion and changes to identity, network policy, secrets or its own permissions. It classifies the agent as not high-risk under Annex III point 2 of the AI Act, but as an ICT change under DORA Article 9. A recommend-only pilot had chosen the right action in 91 per cent of cases, yet cut time to restore by only 6 minutes because engineers still woke; acting within a deny-by-default manifest, with reversible actions only, was therefore chosen, while full remediation was rejected because its blast radius cannot be bounded. The on-call engineer is notified of every action within one minute and can undo it.

The record argues from evidence collected under a less autonomous option, which is the least-autonomy rule of Chapter 11 applied as a sequence. Its retirement criteria are deliberately asymmetric: a single action outside the manifest suspends the agent at once, because it means the control has failed, whereas more than two agent-caused incidents in a quarter only revert it to recommend-only.

Status lifecycle and retirement

The ADR statuses from Architecture as Code, proposed, accepted, deprecated, and superseded, remain valid (72, Nordqvist, 2025). AI adds one practical change: evidence also re-opens a record. When the drift monitor of Chapter 21 reports that an evaluation threshold has been breached, the pipeline creates a review task against the record, and the retirement criteria are the contract that makes this possible. A decision without them is revisited only when something has gone visibly wrong.

Worked example: a record from proposal to supersession

Illustrative example. The life of AIDR-0007 at Nordvik Region shows the statuses at work. The decision log index below is generated from the records' front matter on every merge; nobody edits it by hand.

# decisions/index.yaml - generated, Nordvik Region (illustrative)
records:
  - id: AIDR-0007
    title: Decision support for case prioritisation in benefit claims
    history:
      - {status: proposed, date: 2026-03-02, change: pull-request-41}
      - {status: accepted, date: 2026-04-15, change: pull-request-47}
      - {status: superseded, date: 2027-02-10, change: pull-request-112,
         superseded_by: AIDR-0018,
         trigger: "drift monitor: demographic parity difference 0.06 for 61 days"}
  - id: AIDR-0011
    title: Human oversight design for benefits-case triage
    relates_to: [AIDR-0007, AIDR-0018]
  - id: AIDR-0018
    title: Case prioritisation model version 4 without postcode features
    supersedes: AIDR-0007
    history:
      - {status: accepted, date: 2027-02-10, change: pull-request-112}

Three rules keep the history trustworthy. The body of an accepted record is never edited; only its status and superseded_by fields change, in the same pull request that accepts the successor. The successor states why the predecessor was retired, so the chain of reasoning survives staff turnover. And AIDR-0011 now relates to both models, because the oversight design outlived the model it was written for.

The successor, AIDR-0018, is where the retirement is explained. Analysis traced most of the parity gap to postcode features. Version 4 without postcode features reached precision at 100 of 0.82 and a parity difference of 0.02, and was promoted after ten days in shadow mode. Suspending the ranking was recorded as the fallback, at an estimated four days added to median waiting time. Because removing features is outside the change envelope, the risk assessment and fundamental rights impact assessment were updated first. The record also governs the retired version 3.2.0: kept warm for rollback for 90 days, then removed from serving, with its registry entry and logs retained while any case it handled can be appealed, because a resident who appeals is entitled to an account of the system as it was then.

Model cards and datasheets as version-controlled companions

A decision record explains why a system exists, but it does not describe the model in enough detail to judge whether it is fit for the stated purpose. That is the job of the model card, proposed by Mitchell and colleagues to report a model's intended use, evaluation conditions, performance disaggregated across relevant groups, and caveats and recommendations (41, Mitchell et al., 2019). Datasheets for datasets, proposed by Gebru and colleagues, do the same for data: motivation, composition, collection process, preprocessing, recommended uses, distribution, and maintenance (42, Gebru et al., 2021).

In an enterprise both become engineering artefacts. Following the Documentation as Code principle of Architecture as Code, the card lives in the same repository as the training code, and the pipeline fails if the card's reported metrics do not match those the evaluation stage produced. Hutchinson and colleagues make the same argument for datasets, proposing documents that accompany a dataset through its lifecycle, with reviews at each stage (70, Hutchinson et al., 2021). In practice, each registered model version has one card, generated from the training run and evaluation report, to which a person adds intended use and limitations; each dataset version has one datasheet, owned by the data product team of Chapter 6; cards are immutable once promoted; and every card names its evaluation sets by identifier and hash.

model_card:
  model: claims-ranker
  version: 3.2.0
  decision_record: AIDR-0007
  training_data:
    - datasheet: data/claims-register/datasheet.md
      version: 2025-12
      sha256: 4f1c9e7a2b3d5c6e8f901a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d
  evaluation:
    set: eval/claims-holdout-2025Q4
    precision_at_100: 0.83
    demographic_parity_difference: 0.03
  limitations:
    - Trained on claims from one region; performance in other regions unverified.
    - Not evaluated on claims submitted on paper and digitised after 2024.

The limitations block is the part a pipeline cannot write and an auditor reads first.

The system card for compound systems

Most AI systems in production combine a retrieval index, a third-party language model, prompts, guardrails and orchestration. Zaharia and colleagues note that maximising the quality of such compound AI systems requires co-optimising the components to work well together (63, Zaharia et al., 2024).

The organisation did not train the third-party model and cannot produce its card, so the system card closes the gap. It lists every component and version, describes the data flow, including which data leaves the organisation's control, states system-level evaluation results, and declares the controls. It is also where the AI Act's role analysis is recorded: the organisation integrates a general-purpose AI model supplied by another provider into a compound system of which it is itself the provider (a downstream provider in the sense of Article 3(68)), and if that system is high-risk the provider obligations in Article 16 apply to the organisation, not to the model vendor (1, European Union, 2024). Chapter 10 returns to this point.

The architecture repository as the single source of truth

What Article 11 and Annex IV ask for

Before a high-risk AI system is placed on the market or put into service, its provider must have current technical documentation showing compliance with the requirements of Chapter III, Section 2 (Article 11). Annex IV lists its contents, from intended purpose, architecture, data requirements and test results to the Article 9 risk management system, the standards applied, the declaration of conformity and the Article 72 post-market monitoring plan. Under Article 18 the documentation stays available to national competent authorities for ten years (1, European Union, 2024). AI Governance, Chapter 11 treats the same obligations from the assurance side (73, Nordqvist, 2026).

Read as a list of facts rather than documents, almost all of Annex IV already exists in a well-run repository, so the architectural response is to make the mapping from fact to Annex IV heading explicit and then generate the document.

Mapping the repository to Annex IV

The following layout works as one repository per AI system or one directory per system in a platform monorepo.

Repository path	Content	Annex IV point served
decisions/	AI Decision Records in Markdown and YAML	1 (intended purpose, versions), 6 (changes)
systems/<name>/system_card.md	Compound system description and data flow	1, 2 (architecture), 3 (controls)
models/<name>/model_card.md	Per-version model documentation	2 (training, validation, testing)
data/<name>/datasheet.md	Per-version dataset documentation	2 (data requirements)
evaluation/	Evaluation set manifests and result reports	2, 3 (accuracy, robustness), 4 (metrics)
risk/register.yaml	Risk register and treatments	5 (risk management system)
pipelines/	CI/CD definitions with gates	2 (development process), 6 (lifecycle)
monitoring/	Drift jobs, alert rules, post-market plan	3 (monitoring), 9 (post-market monitoring)
conformity/	Standards applied, declaration of conformity	7, 8

The exact paths matter less than the rule that each Annex IV heading has one place where its evidence lives. In Nordvik Region's triage repository, for example, a short workflow runs on every pull request that touches decisions/: it validates the front matter, refuses edits to the body of an accepted record except for the status fields, requires an oversight path and thresholds in every high-risk record, and regenerates the index, failing the build if someone has edited the generated file by hand. That immutability check is what makes supersession the only way to change a decision.

Generating technical documentation from the repository

With the layout fixed, the documentation build is a pipeline stage, not a writing exercise. A generator reads the records, the card headers, and the latest evaluation reports, applies a template organised by Annex IV heading, and emits the technical documentation on every model promotion and every merge to the decision log. The output is versioned and signed, and the ten-year retention under Article 18 is met by archiving the signed output rather than by hoping a shared drive survives three reorganisations. A missing oversight path in a high-risk record is therefore caught at the pull request, not at the conformity assessment. AI Governance, Chapter 11 develops the evidence chain that assembles such artefacts into packs an auditor or market surveillance authority can verify.

Several member states, the Netherlands among them, also maintain public algorithm registers for authorities. The same repository produces the register entry, because it asks for a subset of the same facts; AI Governance, Chapter 17 codifies the entry. An authority that writes its register entry by hand, separately from its technical documentation, will eventually publish something that contradicts what it has deployed.

Practical consideration: records that people actually write

The commonest failure of a decision log is a missing record, written months later from memory. Three habits prevent it. The record is opened as a draft in the same pull request that adds the first experiment pipeline. A field that does not apply is marked as not applicable with a reason, never left blank, because a blank cannot be told apart from an omission. And the registry refuses to promote a model version whose system has no accepted record, which turns the log into a precondition.

Across the usage modes

An AIDR is not needed for every use of a language model; the usage modes of Chapter 2 (usage-mode matrix) give a workable threshold.

Mode	Decision recorded	Threshold
Mode 0 – Ambient AI	Accepting or declining a supplier-enabled AI feature	Each feature that processes personal or confidential data
Mode 1 – Conversational assistants	Approving a conversational service and its data tier	One record per service, not per user
Mode 2 – Embedded workplace assistants	Enabling an embedded assistant, its scope and data boundaries	One record per rollout and per significant change of scope
Mode 3 – Agentic work tools	Approving an agentic tool and its permission configuration	One record per tool
Mode 4 – Built AI applications	The full AIDR for each system	Always; high-risk records carry every field
Mode 5 – Autonomous production agents	The full AIDR for each agent, and a record for any write access, as in AIDR-0027	Always, and again whenever permissions widen

Records in Modes 0 to 3 are short and concern procurement and configuration, so intended purpose, prohibited uses and data boundaries matter most there.

Nordvik Region and Hansa Mutual

Nordvik Region's decision log tells the story of one high-risk system through AIDR-0007, AIDR-0011 and AIDR-0018. Because Nordvik deploys a system that TriageSoft provides, its records concentrate on what a deployer controls: intended use, oversight, logging and the duty to explain the ranking's role to a resident who asks. Under public-access law residents may request these records, so they are written to be read outside the authority.

Hansa Mutual's records, AIDR-0021 and AIDR-0027, by contrast, concern systems it builds and an agent it lets act on production, so they carry provider reasoning and the DORA view of ICT risk. They also state the markets they apply in and the legal basis in each: the AI Act and GDPR in the EU entities, group policy in Norway until the AI Act is incorporated into the EEA Agreement (241, EFTA, n.d.), the UK GDPR in the United Kingdom (248, UK Parliament, 2018), and the Swiss data protection act in Switzerland (244, Swiss Confederation, 2020). A pipeline can then refuse to deploy a system into a market that no accepted record covers.

Summary

The AI Decision Record keeps Nygard's discipline of short, numbered, immutable records and adds the fields an AI system cannot do without, from intended purpose and risk classification to evaluation thresholds, oversight, and retirement criteria. Written in Markdown for people and in YAML for pipelines, it is validated on every pull request and re-opened by evidence when thresholds are breached.

Model cards and datasheets are its version-controlled companions, with reported metrics checked against the evaluation stage that produced them, and the system card describes the compound system as a whole, which is where the AI Act's provider and deployer roles are actually decided. Organised with a predictable layout, the architecture repository becomes the single source from which the technical documentation required by Article 11 and Annex IV, and public algorithm register entries, are generated. The repository, not the document, is the thing that must be kept true.

Part II turns to the substance of the platform, beginning in Chapter 6 with the data products, contracts, and lineage on which every datasheet and model card depends.

Sources

1. Nordqvist, G. (2025). Architecture as Code. The Transformed Architecture, Book 1, published at aac.geon.se. (72, Nordqvist, 2025)
2. Nygard, M. Documenting Architecture Decisions. Cognitect blog, 15 November 2011. (40, Nygard, 2011)
3. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. (1, European Union, 2024)
4. European Parliament and Council of the European Union. Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data (General Data Protection Regulation). Official Journal of the European Union, 4 May 2016. (2, European Union, 2016)
5. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D. & Gebru, T. (2019). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19), pp. 220–229. (41, Mitchell et al., 2019)
6. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H. & Crawford, K. (2021). Datasheets for Datasets. Communications of the ACM, 64(12), pp. 86–92. (42, Gebru et al., 2021)
7. Hutchinson, B., Smart, A., Hanna, A., Denton, E., Greer, C., Kjartansson, O., Barnes, P. & Mitchell, M. (2021). Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21), pp. 560–575. (70, Hutchinson et al., 2021)
8. Zaharia, M., Khattab, O., Chen, L., Davis, J. Q., Miller, H., Potts, C., Zou, J., Carbin, M., Frankle, J., Rao, N. & Ghodsi, A. (2024). The Shift from Models to Compound AI Systems. Berkeley Artificial Intelligence Research blog, 18 February 2024. (63, Zaharia et al., 2024)
9. European Free Trade Association. EEA-Lex factsheet 32024R1689: the Artificial Intelligence Act. EFTA. (241, EFTA, n.d.)
10. United Kingdom (2018). Data Protection Act 2018 (c. 12), with the UK General Data Protection Regulation. (248, UK Parliament, 2018)
11. Swiss Federal Assembly (2020). Federal Act of 25 September 2020 on Data Protection (Data Protection Act, FADP), SR 235.1. Fedlex, in force 1 September 2023. (244, Swiss Confederation, 2020)
12. Nordqvist, G. (2026). AI Governance: Governing Artificial Intelligence in European Enterprises and Public Authorities. The Transformed Architecture, Book 2, published at aig.geon.se. (73, Nordqvist, 2026)