Integrating AI into the Enterprise Landscape

Chapter 11 gave agents explicit boundaries; this chapter, which closes Part II, turns to the landscape those agents and every other AI service must connect to. Most of the value of a model, and most of its risk, is realised when its output reaches a case file, an order, or a citizen. The chapter covers the AI gateway as integration seam, synchronous and event-driven integration, legacy systems of record, delegated authority, contracts for probabilistic services, and the interoperability duties of public bodies.

Figure 12.1: The AI gateway mediates every call between channels, AI services, and systems of record, with identity, caching, events, and observability attached at the same seam. Channels such as a citizen portal, a case worker application or an agent runtime all enter here; systems of record include case management, ERP and a legacy mainframe reached through an adapter.

The integration problem AI introduces

Classical integration assumes that a service returns the same answer for the same request and that a failure is an exception rather than a plausible-looking result. An AI service breaks both assumptions: identical requests may produce different outputs, and the most dangerous failure is a fluent, wrong response that passes every schema check.

Zaharia and colleagues describe the shift from models to compound AI systems, in which the model is one component among retrievers, tools, control logic, and the ordinary services of the enterprise (63, Zaharia et al., 2024). Integration architecture is where that composition happens, and Richards and Ford remind us that reliability, auditability, and security are achieved by structure, not by intention (80, Richards & Ford, 2020).

The AI gateway as the integration seam

The AI gateway of Chapter 4 is the single seam through which every AI call passes, from a channel to a model and from an agent to a tool that fronts a system of record. It performs the functions that no team should implement twice.

Gateway function	Purpose
Authentication and delegated authority	Bind every call to a service identity and the acting person
Policy enforcement	Allow or deny models, tools, and data classes per consumer
Routing	Select model, region, and provider by task, cost, and residency
Quotas and budgets	Cap tokens, calls, and spend per consumer and task
Guardrails	Screen inputs and outputs against defined rules
Logging and tracing	Emit traces that satisfy AI Act Article 12 record-keeping
Caching and idempotency	Return stored results for repeated or retried requests

Place the gateway at the boundary between the platform team and the consuming teams, in the Team Topologies sense of a self-service capability, which AI Governance, Chapter 12 develops into an organisational design for AI. Because consumers see a stable interface, the platform team can change models, providers, and regions behind it; in a Nordic social insurance authority this is the difference between a six-week provider migration and a six-month one.

The gateway also makes the deployer obligations of Regulation (EU) 2024/1689, the AI Act, enforceable. Article 26 requires deployers of high-risk AI systems to use them in accordance with the provider's instructions, to assign human oversight to competent natural persons, and to keep the logs the system generates for at least six months; Article 50(1) requires providers to design systems that interact directly with people so that those people are informed they are interacting with an AI system (1, European Union, 2024). The gateway can refuse a call that omits the disclosure flag, attach the oversight routing, and write the log, for every consumer at once.

Worked example: an AI gateway route and policy configuration

Illustrative example. Hansa Mutual, the multinational insurance group introduced in Chapter 1, configures one route per consuming system. The route below serves the claims assistant of Chapter 5 and expresses each gateway function as configuration.

# gateway/routes/claims-assistant.yaml - Hansa Mutual (illustrative)
apiVersion: ai-architecture.example.org/v1
kind: GatewayRoute
metadata:
  name: claims-assistant
  owner: ai-platform-team
  decisionRecord: AIDR-0021
match:
  path: /v1/claims-assistant/answer
  callers: [svc-claims-desk-ui]
  requireActingUser: true
routing:
  primary:  {model: registry://gpai-large-eu/2026-04, region: eu-central, pinned: true}
  fallback: {model: registry://open-weight-medium/3.1, region: hansa-dc-frankfurt}
  prohibitedRegions: [non-eea]
inputGuardrails:
  - {type: pii-redaction, entities: [national-id, iban, payment-card, health-term], action: mask}
  - {type: injection-classifier, action: flag-and-continue}
outputGuardrails:
  - {type: citation-required, action: block-if-missing}
quotas:
  requestsPerMinutePerUser: 20
  costEurPerMonth: {budget: 18000, alertAt: 0.8, hardStopAt: 1.1}
cache:
  exactMatch: {enabled: true, ttlMinutes: 60, keyIncludes: [model, prompt-version, index-version]}
  semantic: {enabled: false}
logging:
  traces: opentelemetry-genai
  storePrompts: masked
  retentionDays: 3650
disclosure:
  responseFlag: ai_generated
marketOverlays:
  overlayPath: gateway/overlays/markets/
  perMarket:
    UK: {logRegion: uk-south, disclosureText: disclosure/en-gb.md}
    CH: {logRegion: ch-north, disclosureText: disclosure/de-ch.md}

The fallback, an open-weight model in Hansa Mutual's Frankfurt data centre, keeps the service running if the provider fails and gives the DORA exit plan a tested path. Semantic caching is disabled because the saving does not justify answering a question about one policy edition from another, and prompts are stored masked so that the log does not become an unprotected copy of customer data. The marketOverlays block keeps one route for every market while the UK and Swiss entities keep their logs in their own countries and each market receives the disclosure in its own language.

Synchronous and event-driven integration

AI sharpens the old choice between request and event because inference is slow, variable, and expensive to repeat. Kleppmann's treatment of the trade-offs between request–response and message-based systems applies directly (77, Kleppmann, 2017).

Use a synchronous call when a person is waiting and the answer is advisory, such as a case worker's summary of a 60-page file. Set an explicit timeout, return a typed fallback, and never block a transaction of record on an inference call.

Use an event when the work is not time-critical, fans out to several consumers, or must survive the AI service being unavailable. A claims system publishes a ClaimRegistered event; an AI service consumes it and publishes ClaimTriageSuggested; the case-management system attaches the suggestion to the file. The claims system is never coupled to the model's availability, and replaying the stream with a newer model evaluates a challenger, as Chapter 7 described. Hansa Mutual's ClaimTriageSuggested event, for example, carries the suggestion with its confidence, model and prompt versions, the trace context, an expiry time, the claim's market, and an advisory flag that forbids committing it as a decision without a person.

Events also solve a governance problem. Because each AI-produced event is a distinct message with a schema, the AI contribution to a business process is visible in the event log, separable from human actions, and auditable. AI Governance, Chapter 11 relies on that separability.

Most production designs combine both: the gateway acknowledges a synchronous request with a correlation identifier and completes the work asynchronously, which decouples the user experience from inference latency.

Integrating with legacy systems of record

Large organisations keep their truth in systems that predate the web, such as mainframes and bespoke case-management systems. Three principles keep AI integration with them safe.

First, AI never writes directly to a system of record. It writes a proposal to an intermediate store or publishes an event, and a deterministic component or a person commits the change. This is the agent pattern of Chapter 11 at landscape level, and it makes a confident wrong answer recoverable.

Second, wrap the legacy system in an adapter with its own contract. The adapter exposes a narrow, well-typed interface and enforces the legacy system's own authorisation model, so that the AI service inherits neither the mainframe's full command set nor its assumption that every caller is a trusted employee. The adapter is also the first step of the strangler fig migration that Architecture as Code describes (72, Nordqvist, 2025).

Third, treat the legacy system's data as a data product with a contract, as Chapter 6 described. Otherwise a model grounded on an undocumented 30-year-old extract will reproduce every undocumented quirk.

Identity propagation and delegated authority

The question "who is calling?" has two answers in an AI integration, and both must reach the system of record: the service identity of the AI component and the person on whose behalf it acts. Collapsing them into one is the most common identity mistake I see.

The pattern that works is delegated authority with explicit propagation. The gateway exchanges the person's token for a short-lived token bound to the AI service's identity, with the person remaining the sub (subject) and the AI service recorded in the act (actor) claim, as RFC 8693 defines the two (250, Jones et al., 2020), and every downstream call carries both. The system of record can then check the person and the service separately, and the audit log records both. Consequently, the AI service never holds standing credentials to systems of record, and the system of record, not the model, makes the authorisation decision. A prompt cannot grant access that the identity provider has not.

Worked example: the claims of a delegated token

Illustrative example. When an officer at Nordvik Region asks the staff assistant to draft a reply to a letter in the records-management system, the gateway exchanges the officer's token for one issued to the assistant. The adapter checks both the sub (the officer) and the act (the assistant) identity in the decoded claims before returning a document.

{
  "iss": "https://idp.nordvik.example",
  "aud": "records-management-adapter",
  "sub": "officer-30417",
  "acr": "high",
  "org_unit": "planning-department",
  "act": {
    "sub": "svc-staff-assistant"
  },
  "scope": "record.read:assigned",
  "purpose": "draft-reply",
  "ai_system": "staff-assistant",
  "iat": 1791795600,
  "exp": 1791795900
}

The token lives five minutes, reads only records assigned to the officer, and names the purpose so that the audit log shows why the assistant read the letter. For public bodies the European Digital Identity Framework, established by Regulation (EU) 2024/1183, will increasingly be the source of the person's identity at the channel, and the acr value shows how propagation carries the assurance level of that identity through to the system of record (10, European Union, 2024).

API contracts for probabilistic services

A contract for an AI service must express what a deterministic contract never needed to: how confident the service is, where the answer came from, what to do when it cannot answer, and which model produced it. The excerpt below shows the response schema of a document-summarisation service as an OpenAPI component, kept in version control and validated in the pipeline.

openapi: 3.1.0
info:
  title: Case Document Summary Service
  version: 2.3.0
components:
  schemas:
    SummaryResponse:
      type: object
      required: [request_id, status, model, provenance, disclosure]
      properties:
        request_id: { type: string, format: uuid }
        status:
          type: string
          enum: [completed, partial, refused, fallback]
        summary: { type: string }
        confidence:
          type: number
          description: Calibrated estimate from the evaluation service, not a raw model probability.
        provenance:
          type: object
          properties:
            source_documents: { type: array, items: { type: string } }
            retrieval_run_id: { type: string }
        model:
          type: object
          properties:
            registry_id: { type: string }
            version: { type: string }
            prompt_version: { type: string }
        disclosure:
          type: object
          properties:
            ai_generated: { type: boolean, const: true }
            human_review_required: { type: boolean }

The status enumeration makes refusal and fallback first-class outcomes that consumers must handle. The confidence field is calibrated, because a raw token probability is not a probability of correctness. The provenance block supplies the system-level transparency AI Governance, Chapter 10 develops, the model block makes a complaint six months later investigable, and the disclosure block carries the Article 50 obligation into the data.

Version the contract like any other API. A model change behind an unchanged contract is a minor version if the evaluation suite passes and a major version if it does not; the consumer's contract tests, not the provider's assurances, decide which.

Testing needs more than a stub that always returns a confident answer. Contract tests should include recorded responses for every status value and for confidence on both sides of each threshold; a replay of anonymised requests through the evaluation suite detects behaviour changes behind an unchanged schema; and removing the AI service entirely shows that every consumer degrades to its documented fallback.

Caching and idempotency

Inference is expensive and slow, which makes caching attractive, and non-deterministic, which makes it subtle.

Exact-match caching returns a stored response for an identical request, keyed on the normalised input, the model version, and the prompt version. Omit a parameter from the key and a model upgrade will serve stale answers.

Semantic caching returns a stored response for a request judged similar enough by an embedding comparison. It can cut cost substantially for a citizen chatbot, but similarity is not equivalence: a cached answer to "can I appeal within three weeks" is wrong for "can I appeal within three months". Confine it to low-risk, advisory content and record cache hits in the provenance.

Idempotency means that a retried request produces no second effect, which is essential for agents that call tools. The gateway therefore stores an idempotency key per request, returns the stored outcome on retry, and never re-executes a completed tool call; as Kleppmann explains, idempotent operations are how we get the effect of exactly-once from an at-least-once network (77, Kleppmann, 2017).

Caching also touches data protection. A cache that stores personal data is a processing activity under the GDPR with its own retention requirement, and a semantic cache can leak one person's data into another person's answer (2, European Union, 2016). Partition caches by tenant and data class, and give personal-data caches short lifetimes.

Observability across the seam

Integration is where traces are either joined or lost. Use the OpenTelemetry semantic conventions for generative AI so that a model call appears as a span with the same trace identifier as the channel request that triggered it and the system-of-record write that followed (34, OpenTelemetry, n.d.). Propagate the trace context through events as well, so that ClaimTriageSuggested carries the context of the ClaimRegistered event it answers. Chapter 21 builds monitoring on this foundation, provided no adapter, queue, or cache drops the context.

Interoperability obligations for public bodies

Public administrations in the European Union integrate under rules that private enterprises do not face. Regulation (EU) 2024/903, the Interoperable Europe Act, requires public sector bodies to carry out interoperability assessments before setting up or substantially changing binding requirements for trans-European digital public services, and establishes the Interoperable Europe Board and a portal of reusable solutions (8, European Union, 2024). The European Interoperability Framework underpins it with principles and a layered model of legal, organisational, semantic, and technical interoperability (16, European Commission, 2017).

An AI service that an authority exposes to other administrations is therefore a digital public service, and its contract belongs to the semantic interoperability layer. Prefer open specifications at the gateway so that a neighbouring authority can consume the service without a vendor relationship, and serve the once-only principle by retrieving from the authoritative source at request time rather than ingesting a copy.

Across the usage modes

The integration surface, and therefore what the gateway can see, differs across the usage modes of Chapter 2 (usage-mode matrix).

Mode	Integration surface	Gateway role
Mode 0 – Ambient AI	Inside supplier software	None; contractual controls only
Mode 1 – Conversational assistants	Browser or app; an enterprise API where offered	Web proxy for browser use; the gateway for API use
Mode 2 – Embedded workplace assistants	Suite connectors into business systems	Connector approval and identity governance
Mode 3 – Agentic work tools	Model API calls from the tool; MCP connectors	The gateway where a custom endpoint is accepted; the tool registry
Mode 4 – Built AI applications	APIs, events and adapters	The mandatory gateway
Mode 5 – Autonomous production agents	As Mode 4, plus tool servers before systems of record	The gateway for model and tool calls, with delegated authority and idempotency

In Modes 2 and 3 a connector reaches a system of record without passing through the integration architecture, so treat each connector approval as an integration decision.

Nordvik Region and Hansa Mutual

Nordvik Region's integrations are shaped by public-law duties. The staff assistant acts for an identified official through a delegated token. The citizen-service assistant consumes a national authority's eligibility rules through a documented API, with provenance naming the rule version, which serves the Interoperable Europe Act's reuse intent and the AI Act's transparency duty in one design. The benefits-triage system orders the caseworkers' queue but never changes a case, so suspending it, as AIDR-0011 allows, returns the queue to arrival order without touching any record.

Hansa Mutual integrates AI into a group landscape with national systems of record, and the gateway is where its jurisdictions meet. One route serves every market while overlays set log location and disclosure, and events carry the claim's market so that consumers reject foreign ones. Because handlers at the Polish service centre work for several entities, the delegated token also carries the entity on whose behalf the handler acts, and the claims adapter refuses a request whose entity does not match the claim's market. The seam thus lets Hansa enforce national differences without building national platforms.

Summary

The AI gateway is the seam at which identity, policy, routing, budgets, guardrails, logging, and caching attach once. Synchronous calls suit advisory answers to waiting people; events suit work that fans out or must survive the model being unavailable, and leave an auditable record of the AI contribution. AI reaches legacy systems through adapters and never writes to them directly.

Identity propagation carries both the service and the acting person to the system of record, which makes the authorisation decision. Contracts make refusal and fallback first-class outcomes and carry confidence, provenance, model version, and disclosure, while caching respects model versions and the GDPR and idempotency prevents duplicate actions. For public bodies, the Interoperable Europe Act turns these patterns into obligations.

With this chapter Part II is complete: data, models, the delivery loop, retrieval, agents, and integrations are in place as codified artefacts. Chapter 13 opens Part III by asking where all of this runs.

Sources

1. Zaharia, M., Khattab, O., Chen, L., Davis, J. Q., Miller, H., Potts, C., Zou, J., Carbin, M., Frankle, J., Rao, N. & Ghodsi, A. (2024). The Shift from Models to Compound AI Systems. Berkeley Artificial Intelligence Research blog, 18 February 2024. (63, Zaharia et al., 2024)
2. Richards, M. & Ford, N. (2020). Fundamentals of Software Architecture: An Engineering Approach. Sebastopol, CA: O'Reilly Media. ISBN 978-1-4920-4345-4. (80, Richards & Ford, 2020)
3. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. (1, European Union, 2024)
4. Kleppmann, M. (2017). Designing Data-Intensive Applications. Sebastopol, CA: O'Reilly Media. ISBN 978-1-4493-7332-0. (77, Kleppmann, 2017)
5. Nordqvist, G. (2025). Architecture as Code. The Transformed Architecture, Book 1, published at aac.geon.se. (72, Nordqvist, 2025)
6. European Parliament and Council of the European Union. Regulation (EU) 2024/1183 amending Regulation (EU) No 910/2014 as regards establishing the European Digital Identity Framework. Official Journal of the European Union, 30 April 2024. (10, European Union, 2024)
7. European Parliament and Council of the European Union. Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data (General Data Protection Regulation). Official Journal of the European Union, 4 May 2016. (2, European Union, 2016)
8. OpenTelemetry Authors. OpenTelemetry Documentation, including the Semantic Conventions for Generative AI systems. Cloud Native Computing Foundation. (34, OpenTelemetry, n.d.)
9. European Parliament and Council of the European Union. Regulation (EU) 2024/903 laying down measures for a high level of public sector interoperability across the Union (Interoperable Europe Act). Official Journal of the European Union, 22 March 2024. (8, European Union, 2024)
10. European Commission. New European Interoperability Framework: Promoting seamless services and data flows for European public administrations. Publications Office of the European Union, 2017. (16, European Commission, 2017)
11. Jones, M., Nadalin, A., Campbell, B. (Ed.), Bradley, J. & Mortimore, C. (2020). OAuth 2.0 Token Exchange. RFC 8693, Internet Engineering Task Force, January 2020. (250, Jones et al., 2020)