Infrastructure, Compute and European Digital Sovereignty

This chapter opens Part III, on infrastructure, and asks where AI runs and who controls it, because every model and pipeline executes on someone's hardware under some jurisdiction's law. It covers scarce accelerator capacity, on-premises and hybrid inference, European cloud, Gaia-X and the EuroHPC AI Factories, data residency after Schrems II, confidential computing, sovereign model choices, and the infrastructure code that keeps placement a decision rather than an accident.

Placement options for AI workloads in a column, from a global hyperscaler region to on-premises confidential computing, fed by a sovereignty decision matrix

Figure 13.1: Decision drivers (data classification and legal basis, third-country transfer exposure, latency and GPU capacity, cost and operational skill) feed a sovereignty decision matrix that places each workload on a spectrum from a hyperscaler region to on-premises confidential computing. Open-weight model choice and portable infrastructure code apply across all options.

Why placement is an architectural decision

AI reverses part of the long trend towards abstracting where a workload runs. Training and inference depend on scarce, unevenly distributed accelerators; the data that flows through a model is often the most sensitive the organisation holds; and a European organisation's regulatory position depends on whether personal data leaves the European Economic Area, which cloud regions and support arrangements can cause without anyone intending it.

Digital sovereignty, in the sense this book uses, is the ability of European organisations to control their data, infrastructure, and models. It is not a synonym for on-premises but a set of properties: the organisation knows where its workloads run, can show which law governs them, can change provider without rewriting its estate, and can continue operating if a provider or jurisdiction becomes unavailable. The European Commission's AI Continent Action Plan names five key domains for action: computing infrastructure, data, algorithms and adoption in strategic sectors, skills and talent, and simplification of regulation, and it lists scaling up public AI computing infrastructure first (15, European Commission, 2025).

Accelerator scarcity and capacity planning

Accelerators are the first genuinely constrained resource most enterprise architects have met since mainframe MIPS. Strubell, Ganesh, and McCallum quantified the energy cost of training large language models in order to heighten awareness of that cost in the research community (47, Strubell et al., 2019). For most organisations, however, the relevant cost is serving inference at scale, which grows with adoption.

Capacity planning for AI needs two adjustments to the usual discipline. First, separate the demand profiles: fine-tuning and evaluation are batch workloads that can use scheduled capacity, interactive inference needs reserved capacity, and bursty agent workloads need caps more than capacity. Second, treat the model routing policy of Chapter 22 as a capacity instrument. A small model on a commodity instance handles most traffic, and the capacity you must reserve is determined by the residual that needs scarce accelerators. Without a reviewed plan, I have watched organisations hoard accelerators for a pilot that never reached production while a production service queued behind them.

Worked example: a capacity plan as code

Illustrative example. Hansa Mutual, the multinational insurance group introduced in Chapter 1, keeps its accelerator capacity plan in the platform repository and reviews it quarterly against observed utilisation. The production excerpt below reserves interactive inference, schedules batch work into the gaps, and caps agent traffic.

# platform/capacity/prod.yaml - Hansa Mutual (illustrative)
apiVersion: ai-architecture.example.org/v1
kind: CapacityPlan
metadata:
  environment: prod
  owner: ai-platform-team
  reviewedOn: 2026-09-30
pools:
  - name: dc-frankfurt-l
    location: hansa-dc-frankfurt
    acceleratorClass: gpu-l-class
    reserved: 8
    workloadClasses: [interactive-inference, fallback-inference]
  - name: eu-cloud-m
    location: eu-cloud-provider-de
    acceleratorClass: gpu-m-class
    reserved: 4
    burstable: 12
    workloadClasses: [interactive-inference, batch-evaluation, fine-tuning]
workloadClasses:
  interactive-inference: {latencyP95Seconds: 6, preemptible: false}
  fallback-inference: {latencyP95Seconds: 10, preemptible: false}
  batch-evaluation: {schedule: "22:00-06:00", preemptible: true}
  agents: {maxConcurrentTasks: 150, burstAllowed: false}
routingThresholds:
  shiftToSmallModelWhenUtilisationAbove: 0.80
observed:
  peakUtilisation: {dc-frankfurt-l: 0.58, eu-cloud-m: 0.71}
  decision: no change; revisit if claims assistant extends to commercial lines

The fallback-inference class reserves enough capacity in Hansa Mutual's Frankfurt data centre to carry the claims assistant on an open-weight model if the external provider becomes unavailable, the technical half of the DORA exit strategy discussed below.

Hybrid and on-premises inference

Whether to run inference on premises should be decided per workload, and three factors dominate. Data classification comes first: workloads that process special categories of personal data under Article 9 of Regulation (EU) 2016/679, the GDPR, or that touch national security, criminal justice, or critical infrastructure control, have the strongest case for infrastructure under direct organisational control (2, European Union, 2016). Volume and predictability come second, because steady, high-volume inference is cheaper on owned hardware once utilisation is high, whereas variable workloads are cheaper on consumption pricing; the honest way to find the crossover is to measure the first year. Operational capability comes third, since most organisations lack the skills to run accelerator clusters at the start, and a hybrid design lets them build those skills where control matters.

The hybrid pattern that works is one Kubernetes-based serving platform with clusters in several locations and placement decided by labels and policy, with the model registry of Chapter 7 as the pivot. Architecture as Code (Chapter 7) describes the containerisation and GitOps disciplines that make this possible (72, Nordqvist, 2025).

European cloud, Gaia-X, and the AI Factories

Between the hyperscaler and the data centre lie options that European organisations should evaluate rather than dismiss.

European cloud providers operate under European law alone, which simplifies the transfer analysis, and several offer managed inference for open-weight models; their smaller catalogues rarely matter for inference.

Gaia-X is a European association that defines policy rules, labels, and a federated trust model for data and cloud services, so that a consumer can verify a provider's claims about jurisdiction, data handling, and interoperability through machine-readable self-descriptions (22, Gaia-X, n.d.). Sovereignty requirements thus become attributes a pipeline can check.

Hyperscalers with sovereign controls offer EU-only regions, EU-resident support staff, customer-held keys, and in some cases operation by a European legal entity. These reduce transfer exposure but do not settle foreign legal reach, which is for counsel; the architectural task is to make key custody and region pinning verifiable.

EuroHPC AI Factories are the European Union's public investment in AI-optimised supercomputing, hosted by consortia of European countries to give European start-ups, industry, and researchers access to computing resources for developing large AI models (21, EuroHPC JU, n.d.). For a large organisation they matter for training and fine-tuning rather than production inference, and access follows an allocation process.

Worked example: a sovereignty requirements profile for procurement

Illustrative example. Nordvik Region, the regional public authority introduced in Chapter 1, attaches the profile below to its tender for a managed inference service. Each requirement states how it will be verified, so that the evaluation committee scores evidence and the platform team can check the same attributes after award.

# procurement/profiles/managed-inference-2026.yaml - Nordvik Region (illustrative)
apiVersion: ai-architecture.example.org/v1
kind: SovereigntyProfile
metadata:
  tender: NR-2026-031 managed inference for internal AI systems
  owner: chief-architect
requirements:
  - id: SOV-01
    level: mandatory
    text: All processing, storage and backups of customer data in the EEA
    verify: [contract-annex, region-pinning-policy-export]
  - id: SOV-04
    level: mandatory
    text: Inputs, outputs and logs are not used to train the supplier's models
    verify: [contract-clause]
  - id: SOV-05
    level: mandatory
    text: Open-weight models from the agreed list can be served, and weights exported on exit
    verify: [technical-demonstration, exit-plan]
  - id: SOV-06
    level: weighted
    weight: 15
    text: Machine-readable self-description of jurisdiction and data handling, e.g. a Gaia-X label
    verify: [self-description-file]
  - id: SOV-08
    level: weighted
    weight: 10
    text: Exit assistance of at least 12 months with data returned in open formats
    verify: [contract-clause, exit-plan]
scoring:
  mandatoryFailure: tender-excluded

Only what the legal analysis makes non-negotiable is mandatory; everything else, including a Gaia-X self-description, is weighted, which keeps the tender open to competition in line with the equal-treatment principles of Directive 2014/24/EU (9, European Union, 2014). AI Governance, Chapter 7 covers the contractual clauses that make requirements such as SOV-05 and SOV-08 enforceable after award: the profile states what must be verifiable, and the contract makes the supplier answerable for it.

Data residency and third-country transfer risk

The judgment of the Court of Justice in Case C-311/18, known as Schrems II, invalidated the Privacy Shield adequacy decision and held that organisations relying on standard contractual clauses must assess whether the law of the destination country undermines the protection those clauses promise (18, CJEU, 2020). Under Chapter V of the GDPR, any flow of personal data to a third country must therefore be identified, justified, and, where the destination's law requires, supplemented with technical measures.

AI workloads create transfers where classical applications did not: inference requests to a model outside the EEA, stored prompts, telemetry, fine-tuning uploads, and third-country support access to a running environment.

The architectural response is to map every flow by data category and destination, pin personal-data workloads to EU regions by policy in infrastructure code, minimise data at the gateway where a third-country provider is unavoidable and record that in the DPIA, and keep the exit path open.

For financial institutions, Regulation (EU) 2022/2554, DORA, adds a second lens: a cloud or model provider is an ICT third-party service provider, which brings the register and risk assessment obligations of Chapter V of that regulation, and exit strategies are required where the provider supports a critical or important function (3, European Union, 2022). AI Governance, Chapter 7 treats the contractual side, including Article 30; architecturally, the exit strategy must be technically real, which means portable infrastructure code and an open-weight fallback model.

Confidential computing

Confidential computing runs workloads inside hardware-enforced trusted execution environments, so that data is protected in use and the infrastructure operator cannot read memory contents. For AI it partly answers the transfer and operator-access questions: a model and its inputs can execute on a provider's hardware without the provider being able to observe them, and remote attestation proves that the expected code runs in a genuine enclave.

The limits matter as much. Accelerator support is newer and narrower than for general-purpose processors, attestation proves nothing unless the organisation verifies it, and a technical measure does not settle the legal analysis of foreign reach. Treat confidential computing as a strong supplementary measure, document it in the transfer assessment, and verify attestation in the deployment pipeline.

Sovereign model choices

Sovereignty applies to the model as well. A proprietary model served only by its vendor binds the organisation to that vendor's jurisdiction, pricing, and deprecation schedule, whereas an open-weight model that can run anywhere restores those freedoms.

The Open Source AI Definition, published by the Open Source Initiative in 2024, requires the weights, the code, and sufficient information about the training data to allow a skilled person to recreate a substantially equivalent system (29, OSI, 2024). Many models described as open disclose little about their data, and the distinction is architectural: a model with undocumented training data cannot be assessed for the data governance requirements of Article 10 of Regulation (EU) 2024/1689, the AI Act, if it becomes part of a high-risk system, and a restrictive licence may rule out the jurisdictions or use cases the organisation needs (1, European Union, 2024).

A practical policy has three tiers: open-weight models on controlled infrastructure for sensitive workloads, managed models under European law for general workloads, and global proprietary models, with data minimisation, where capability justifies the exposure. The routing configuration of Chapter 22 enforces the tiers; this chapter decides them.

The sovereignty decision matrix

Bring the factors together in a matrix that lives in the architecture repository and is applied to every new AI workload at intake. Each organisation adjusts the thresholds to its own risk appetite and legal advice.

Workload attribute Hyperscaler EU region Hyperscaler with sovereign controls European cloud or Gaia-X labelled On-premises or confidential computing
No personal data, public information Permitted Permitted Permitted Not required
Ordinary personal data, GDPR Article 6 basis Permitted with transfer assessment Permitted Permitted Optional
Special category data, GDPR Article 9 Not permitted Permitted with DPIA and key custody Permitted with DPIA Preferred
High-risk AI system under AI Act Annex III Permitted with transfer assessment Permitted Permitted Preferred for public authorities
Critical or important function under DORA Permitted with exit strategy Permitted with exit strategy Permitted with exit strategy Permitted
National security or classified Not permitted Not permitted Case by case Required

Encode the matrix as a rule set that the intake pipeline evaluates against the workload's declared data classification and risk class. AI Governance, Chapter 8 supplies those inputs as classification as code, so a change to a use case's descriptor re-evaluates its placement.

When Nordvik Region applied the matrix to its three workloads, it narrowed the options but did not make the choice. The high-risk benefits-case triage (AIDR-0007, AIDR-0018) went on premises because the model is small and the volume steady; the staff assistant (AIDR-0211) went to a European cloud provider because it needs capacity the data centre lacks; and the correspondence summaries (UC-2026-017), which may include Article 9 health data, wait until two additional accelerators are installed. That last case shows the matrix's most useful job: stopping a workload from going to the most convenient placement while the compliant one is not ready.

Worked example: a decision record on model origin and hosting

Illustrative example. The staff assistant row rests on AIDR-0211, in which Nordvik Region chose where its model comes from and where it runs. The record keeps the legal points to what the authority's counsel concluded.

---
adr_id: AIDR-0211
title: Model origin and hosting for the staff assistant
status: Accepted
date: "2026-06-18"
deciders: [Chief Digital Officer, Chief Architect]
reviewers: [Data Protection Officer, Legal Counsel, Procurement, CISO]
classification: limited-risk
---
# AIDR-0211: Model origin and hosting for the staff assistant

Options considered:
A. Open-weight model with documented training data, served by a European
   cloud provider under contract NR-2026-031, EEA only, customer-held keys.
B. Proprietary model through a US-headquartered hyperscaler's API in an EU
   region, with EU data boundary commitments.
C. Smaller model from a national provider, hosted in the country.
Assessment (weights agreed before scoring): weighted totals A 4.30,
B 3.85, C 3.30. B scored highest on task quality (weight 30); A led on
data protection (25) and exit (15).
Legal points (Legal Counsel's note LC-2026-044, not general advice):
- A and C: no transfer under GDPR Chapter V if support and sub-processors
  stay in the EEA, which the contract requires.
- B: exposure to third-country law must be assessed in line with Schrems II;
  usable only with masking of personal data at the gateway.
- All: the provider's Article 53 duties do not discharge Nordvik Region's
  own deployer obligations.
Decision:
- Option A. Option B remains approved as a fallback for non-personal tasks.
Consequences:
- Nordvik Region operates evaluation and upgrades of the open-weight model.
- Exit tested yearly by serving the model on the regional data centre.

Weights set before scoring placed data protection and exit above a small quality margin, and keeping option B as a fallback for non-personal tasks makes the decision a routing rule rather than a ban. The documented training data and licence matter because, as the Open Source AI Definition notes, a release of weights alone settles neither question (29, OSI, 2024), and the transfer reasoning follows Schrems II (18, CJEU, 2020). AI Governance, Chapter 6 examines the geopolitical side of model origin, and Nordvik Region's approval matrix there is the policy this record applies.

Infrastructure as code for AI platforms

Everything above is enforceable only if placement is expressed in code. Two artefacts carry most of the weight: the platform definition that creates accelerator pools in specific regions, and the workload definition that constrains where a model may be scheduled. The Terraform excerpt below creates an inference node pool pinned to a Swedish region with customer-managed keys and sovereignty labels.

module "inference_pool_se" {
  source = "./modules/accelerator-pool"

  cluster_name     = "ai-platform-prod"
  region           = "europe-north-se"
  accelerator_type = "gpu-l-class"
  node_count_min   = 2
  node_count_max   = 8

  labels = {
    "sovereignty.example.org/jurisdiction" = "eu"
    "sovereignty.example.org/operator"     = "eu-entity"
    "sovereignty.example.org/key-custody"  = "customer"
  }

  encryption_key_id       = var.customer_managed_key_se
  allowed_support_regions = ["eu"]
}

Each model deployment then requires, through Kubernetes node affinity, nodes whose labels match its data classification, and an admission policy rejects any deployment that omits the constraints its classification requires. It runs on the same policy engine as the governance policies of AI Governance, Chapter 9, so a placement violation becomes a failed pipeline stage rather than an audit finding a year later.

Portability is the other half. Keep the platform definition modular so that a workload moves to another target by changing the module, and test that periodically. An exit strategy that has never been exercised is a document, not a capability.

Across the usage modes

The sovereignty levers available depend on the usage mode of Chapter 2 (usage-mode matrix).

Mode Typical hosting Sovereignty levers
Mode 0 – Ambient AI The supplier's choice Contract terms, opt-out rights and change control
Mode 1 – Conversational assistants SaaS chat in the provider's cloud Enterprise tier; EU data residency where offered
Mode 2 – Embedded workplace assistants Vendor-hosted tenant suite Tenant region, data boundary settings, key options
Mode 3 – Agentic work tools Vendor-hosted model API with local execution Custom model endpoints routed to EU-hosted models
Mode 4 – Built AI applications Any placement the matrix permits Placement, model origin, keys and exit, in full
Mode 5 – Autonomous production agents As Mode 4 As Mode 4; continuity weighs most

The matrix and the AIDR apply directly to Modes 4 and 5. For Modes 1 to 3 the matrix still sets which data classes staff may give to which approved service.

Nordvik Region and Hansa Mutual

Nordvik Region runs the benefits-triage system on premises, the staff assistant on an open-weight model with a European cloud provider under contract NR-2026-031, and postpones workloads with special-category data until on-premises capacity exists. Procurement is its main instrument, so it writes sovereignty requirements into tender profiles as verifiable criteria (9, European Union, 2014).

Hansa Mutual's choices differ by market. Its capacity plan keeps fallback inference in Frankfurt, which gives the DORA exit strategy a tested path for the EU entities (3, European Union, 2022), and bursts to an EU cloud region. The UK and Swiss entities send data to an EU platform under their own transfer rules (248, UK Parliament, 2018), (244, Swiss Confederation, 2020), and flows back to them rely on the Commission's adequacy decisions (243, European Commission, 2025), (245, European Commission, 2000), which can be reviewed or withdrawn. Hansa therefore treats those adequacy findings as dependencies with an owner and a review date, and keeps the option of serving those markets from in-country capacity if a finding lapses.

Summary

Placement is an architectural decision because accelerators are scarce, AI data is sensitive, and European law attaches consequences to where personal data flows. Digital sovereignty is therefore a set of verifiable properties rather than a location, and capacity planning uses routing to limit the capacity that must be reserved.

The options run from hyperscaler regions to on-premises and confidential computing. Schrems II and Chapter V of the GDPR require every flow of personal data to be mapped and justified, and DORA adds exit obligations. A decision matrix in the architecture repository, enforced by infrastructure code and admission policy, turns these principles into placements tested before deployment.

Chapter 14 compares consuming AI as a cloud service with running it on the organisation's own infrastructure, and Chapter 15 applies that comparison to agentic work with sensitive information.

Sources

  1. European Commission. AI Continent Action Plan. COM(2025) 165 final, 9 April 2025. (15, European Commission, 2025)
  2. Strubell, E., Ganesh, A. & McCallum, A. (2019). Energy and Policy Considerations for Deep Learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3645–3650. (47, Strubell et al., 2019)
  3. European Parliament and Council of the European Union. Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data (General Data Protection Regulation). Official Journal of the European Union, 4 May 2016. (2, European Union, 2016)
  4. Nordqvist, G. (2025). Architecture as Code. The Transformed Architecture, Book 1, published at aac.geon.se. (72, Nordqvist, 2025)
  5. Gaia-X European Association for Data and Cloud AISBL. Gaia-X: A Federated Secure Data Infrastructure.. (22, Gaia-X, n.d.)
  6. EuroHPC Joint Undertaking. AI Factories.. (21, EuroHPC JU, n.d.)
  7. European Parliament and Council of the European Union. Directive 2014/24/EU on public procurement. Official Journal of the European Union, 28 March 2014. (9, European Union, 2014)
  8. Court of Justice of the European Union. Judgment in Case C-311/18, Data Protection Commissioner v Facebook Ireland Limited and Maximillian Schrems ("Schrems II"). 16 July 2020. (18, CJEU, 2020)
  9. European Parliament and Council of the European Union. Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). Official Journal of the European Union, 27 December 2022. (3, European Union, 2022)
  10. Open Source Initiative. The Open Source AI Definition, version 1.0. 28 October 2024. (29, OSI, 2024)
  11. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. (1, European Union, 2024)
  12. United Kingdom (2018). Data Protection Act 2018 (c. 12), with the UK General Data Protection Regulation. (248, UK Parliament, 2018)
  13. Swiss Federal Assembly (2020). Federal Act of 25 September 2020 on Data Protection (Data Protection Act, FADP), SR 235.1. Fedlex, in force 1 September 2023. (244, Swiss Confederation, 2020)
  14. European Commission (2025). Commission Implementing Decision (EU) 2025/2574 amending Implementing Decision (EU) 2021/1772 on the adequate protection of personal data by the United Kingdom. Official Journal of the European Union, 23 December 2025. (243, European Commission, 2025)
  15. European Commission (2000). Commission Decision 2000/518/EC on the adequate protection of personal data provided in Switzerland. Official Journal of the European Communities, L 215, 25 August 2000. (245, European Commission, 2000)