Sovereign AI Platform Engineering

Your load assumptions under scrutiny

Review of the environment profiles: how many concurrent users, which model classes, which response times? Vague expectations become measurable requirements in tokens per second.

The reference architecture layer by layer

Compute, network, storage, identity, registry, inference, gateway. For each layer the one decision that is expensive to correct later, and how BORDONARO settled it in the Sovereign AI Appliance.

GPU capacity model from L40S to H100

VRAM demand, quantization, batching, concurrency: what really drives the choice of card. Live benchmark in the lab, the same model on two GPU classes in direct comparison.

Exercise: sizing calculation for your scenario

Each team works through its own scenario with the sizing sheet and defends the calculation in front of the group. The most common insight: first sized wrong, then corrected, then understood.

Day result: documented capacity model

Each team has a justified capacity model for its scenario, checked against the day's benchmark values. It will be validated against the real platform over the course of the week.

Azure Local as the foundation

Cluster anatomy, Storage Spaces Direct, roles and limits of the platform in the version available at the time. What Azure Local can do, what it cannot, and where a plain Kubernetes base is the more honest answer.

Identity thought through end to end

Entra ID, local identities, workload identity and secrets management without plaintext in configs. The concrete question: who may call which model with which data, and where is that enforced?

Network design and egress control

Segmenting the inference zone, internal TLS, controlled egress. Whiteboard exercise: which seven connections does the platform really need, and which of them can be closed?

Exercise: implementing the foundation in the lab

Each team sets up identity integration and network segmentation for its lab environment and proves with a test access that the boundaries hold.

Day result: approved foundation

Identity and network are in place and tested. Each team documents its foundation design in the blueprint, open disputes go into the decision log with reasoning.

Registry: provenance and signatures for models

Model and container registry as one discipline: versioning, signature verification, provenance. Why an unsigned model from the internet must never land in a sovereign platform.

Inference servers in an honest comparison

vLLM and alternatives against the same measuring points: throughput, latency, operational effort. Plus a look at Foundry Local on Azure Local in the version available at the time, currently preview, with a clear framing of what that means for production decisions.

Build: from model to API endpoint

Each team builds the stack end to end: pull the model from the registry, verify the signature, serve it, put it behind the API gateway. At the end the team's own platform answers its first authenticated request.

Load test and tuning

A load generator against your own stack: change batching parameters, KV cache and quantization and measure the effect. The capacity model from day one meets reality.

Day result: running stack with measurement protocol

Each team has a working inference stack and a measurement protocol that states and explains the gap between the sizing calculation and the measured performance.

Observability for GPU platforms

GPU utilization, token throughput, queues, error rates: which metrics an operations team really needs and which alerts are allowed to wake someone at night. Building the dashboard on your own lab platform.

Update path: models, drivers, firmware, containers

Four update streams with four risk profiles. A concrete decision per stream: maintenance window, canary or rolling, and what must be measured before every model change.

Tenants, quotas and internal chargeback

Several teams on one GPU platform: quota model, prioritization, cost allocation. Exercise: design and defend a quota model for three competing departments.

Incident drill: the node goes down

We pull a GPU node from one team in a controlled way. Detect, decide, reroute, communicate: the incident is worked through by runbook and then dissected.

Day result: operations dashboard and incident protocol

Each team has a working monitoring dashboard and a completed incident protocol with timeline, decisions and improvements for its own runbook.

The runbook becomes complete

Startup procedures, update flows, incident scenarios, escalation paths: the fragments of the week are consolidated into a runbook skeleton an operations team can take over.

Architecture review against your requirements

Each team's blueprint reviewed against data protection requirements, internal security standards and requirements up to VS-NfD the architecture should be preparable for. Gaps are named, not argued away.

Acceptance simulation: handover to operations

Role play with swapped teams: one team hands over its platform, the other checks against the acceptance checklist and may ask anything. Whatever does not survive the handover gets fixed.

Transfer plan for Monday

Each team prioritizes its first three steps back home: what gets adopted, what gets decided differently, what needs procurement? The follow-up is scheduled.

Final result: referenceable blueprint

Each team leaves with a documented platform blueprint, a validated capacity model, a runbook skeleton and a decision log. A short trainer review of the results closes the week.