There's a version of AI governance that lives in a shared drive. It's a PDF — maybe titled "Employee AI Use Policy" or "Generative AI Acceptable Use Guidelines" — and it covers what employees are allowed to do with ChatGPT, Claude, and Copilot. It probably prohibits sending customer PII to external LLMs. It almost certainly includes a line about confidential data.
Most enterprises have that document. Almost none of them enforce it.
This is the LLM policy enforcement gap: the distance between a governance document your employees acknowledged during onboarding and a control that actually stops a policy violation at the moment it happens.
The Model Card Isn't the Defense
A useful frame came up in an AI governance thread this week: "the defense lives at the API gateway, not in the model card."
That's exactly right, and it points to a structural problem in how enterprises are thinking about AI risk.
Model cards are documentation artifacts. They describe what a model can do, what data it was trained on, what limitations it has. They're useful for vendor evaluation. They tell you nothing about what your employees are actually doing with the model at 2pm on a Tuesday.
An AI use policy PDF has the same problem. It documents intent. It doesn't enforce behavior. When an employee pastes a client contract into ChatGPT to get a quick summary, no policy document fires an alert. No PDF blocks the upload. No acceptable-use guideline logs the transaction for the compliance team.
The only place where enforcement can actually happen is the layer where AI requests flow — the gateway, the proxy, the browser extension sitting between your employee and every LLM endpoint they reach.
Three Ways Document-Only Policies Fail
1. You only find out after the fact.
Document-only policies are retroactive by design. When something goes wrong — a data breach, a regulatory inquiry, an employee who shared something they shouldn't have — the policy document becomes evidence that you had rules, not evidence that you enforced them. Regulators increasingly understand the difference. Article 26 of the EU AI Act, for instance, requires that deployers of high-risk AI systems take "appropriate technical and organisational measures" to ensure their use of AI aligns with the provider's instructions. A PDF in a shared drive is an organisational measure. Whether it's appropriate is a harder question when the technical controls aren't there.
2. Coverage is inconsistent by default.
AI tooling in enterprises is rarely centrally managed. Employees use the ChatGPT web app, Claude.ai, Bing Copilot, GitHub Copilot, Notion AI, and a long tail of AI-embedded SaaS tools — often across personal and work devices, in browsers that IT doesn't control, with accounts that aren't provisioned through your SSO. Your acceptable-use policy applies to all of them in theory. Your enforcement covers none of them unless you have a technical control at the right layer.
MDM covers managed devices. Network proxies cover managed networks. Neither covers an employee on a coffee shop WiFi using their personal laptop to finish a work project with an AI assistant. The gap is the browser and the API.
3. Policy drift is invisible.
Enterprise AI governance policies are living documents — they need to update as your AI toolset changes, as regulators clarify requirements, as new risk categories emerge. But when your only enforcement mechanism is a document, policy drift is silent. You update the PDF; employees don't re-read it; the delta between your stated policy and actual behavior grows. Without runtime telemetry — actual logs of what AI is being used for, by whom, with what data — you have no signal that enforcement is slipping.
What Real-Time LLM Policy Enforcement Looks Like
Enforcement-first AI governance flips the architecture. Instead of starting with a document and hoping behavior follows, you start with the control point and configure policy there.
The right control point is the proxy and the browser. Every AI request your employees make — regardless of which tool, which device, which network — flows through a layer you can inspect, evaluate, and act on. At that layer, you can:
- Evaluate every prompt in real time against your policy: does this contain PII? Customer data? Confidential documents? Code that shouldn't leave your environment?
- Block or redact before the request leaves your network — not after the model has already processed the input.
- Log a tamper-evident record of every AI interaction: who made the request, what was in it, what the response was, what policy evaluation fired.
- Update policy centrally and have it take effect instantly, across every AI surface your employees use, without a re-read or re-acknowledgment cycle.
This is the architecture that closes the gap between "we have a policy" and "we enforce it."
Why Compliance Automation Doesn't Solve This
Platforms like Vanta and Drata are excellent at audit automation — they pull configuration evidence from your cloud infrastructure, help you map controls to frameworks like SOC 2 and ISO 27001, and generate questionnaire responses. Some now offer AI governance modules built around ISO 42001.
But they operate at the infrastructure layer, not the inference layer. They can tell you whether your AI vendor's security posture meets your third-party risk criteria. They can't tell you what prompts your employees sent to that vendor's API this morning, or whether any of them violated a governance policy.
Compliance automation answers: are we configured correctly?
LLM policy enforcement answers: what is our AI actually doing, right now, and is it within policy?
Both questions matter. Only one of them has real-time answers.
Closing the Gap
The enterprises most exposed right now aren't the ones that skipped writing an AI policy. They're the ones that wrote a thorough, well-considered policy — and then stopped there, assuming the document would do the work.
It won't. The defense lives at the gateway.
If you want to see what enforcement-first AI governance looks like in practice — real-time policy evaluation, a full audit trail, and browser-level coverage across every AI tool your employees use — start a free trial →