Data-sovereign AI in SAP: why local models suit the mid-market

For many SAP customers, AI does not fail on the technology. It fails on data protection. Business-critical data from the ERP is not supposed to leave the building, which rules out cloud models for a lot of scenarios before the first project even starts.

The potential is clear enough: document classification, price recommendations, approval routing, compliance checks. All tasks where AI measurably helps. On the other side sit personal data, price conditions, supplier relationships and financial data.

Then there is the regulatory frame. The transparency obligations under Article 50 of the EU AI Act have applied since 2 August 2026. The obligations for high-risk systems were postponed by the Digital Omnibus of 27 July 2026, to December 2027 for Annex III and to August 2028 for Annex I. None of that changes the question of where processing happens. Whoever builds now decides where the data sits in five years.

A path in between: local small language models

You do not need a large cloud to get value out of AI in an SAP context. Small language models running locally, connected securely through the SAP Cloud Connector, are a workable alternative. The model acts as the instance that proposes. It delivers a classification, a recommendation or a risk assessment, and a person makes the final call. Human in the lead.

The data never leaves the company's controlled infrastructure.

Why small models are often enough

Fine-tuned models with a few billion parameters are often enough for typical enterprise tasks. On document classification and information extraction we see results at the level of very large cloud models. Two practical advantages come on top: lower latency and considerably lower running costs. In our own measurement, local inference sits an order of magnitude below the cost of typical cloud APIs.

European models are available too. Teuken-7B, for instance, was developed with Fraunhofer IAIS in the OpenGPT-X project and is openly available.

How it fits into SAP cleanly

The SAP Cloud Connector opens an outbound TLS tunnel. No inbound ports are opened and the internal topology stays invisible. Orchestration runs on BTP: prepare the context, mask sensitive fields, score the result against a confidence threshold. Where confidence is high enough, the recommendation is applied. Where it is not, a person reviews it in SAP Fiori. The result travels back into SAP S/4HANA through certified OData interfaces. The clean core stays untouched.

One thing to be clear about: this approach does not replace SAP's own AI such as Joule or AI Core. It adds a sovereign layer for the cases where data is not allowed to leave the company.

What it is good for

Sensible entry cases are price recommendations (in our case through PAN, the Price Agent for Negotiations), classification of incoming documents, customs tariff classification, approval routing, and anomaly detection in financial postings. What they share: a clearly bounded task, high manual effort, sensitive data.

Controlled rather than all at once

You start with one use case, one GPU and a 7B model, measure the benefit and grow from there. Clean core as the foundation, an AI rollout that happens in controlled steps, and the person who stays in charge. That is how a data protection blocker turns into a business case that holds.

More on this at the DSAG Annual Congress

Simon Pamies goes deeper into the topic together with Frank Zenker (CAS AG): session VP027 "Clean Core und kontrollierte KI-Einführung", 7 October 2026, 5:45 pm, Partner Stage. You will also find us at stand M17 in hall 2.2.