Everyone is guarding the model. Almost nobody guards the plumbing.
The plumbing is where the keys live, and it is still answering the open internet.
Wiz Research scanned 3,074 internet facing AI gateways in February and 294 of them accepted the placeholder admin key printed in LiteLLM's own setup guide.
That is the sample string a developer copies to get moving, left untouched on a machine anyone can reach.
Whoever types it in inherits every provider key the company pays for, along with the cloud identity permissions parked behind the gateway, the same blind spot we wrote about in how many enterprises actually secure their AI agents.
The fastest way into an AI powered business is not the model, it is the control plane nobody assigned an owner.
None of this work is glamorous, and that is exactly why it keeps getting skipped.
Three moves close most of the gap, and none of them need a new vendor.
- Rotate the placeholder key today.
Search every gateway, notebook, container, and environment file for sample values, then replace them with real secrets that live in a vault.
If you run this stack yourself, our self hosted Linux fundamentals course covers the file permissions and service accounts that make the fix stick. - Make reachability a decision you can defend.
A gateway that only serves your own network has no reason to hold a public address.
Wiz found those systems because they answered the open internet, and a short pass at our API security competence check shows whether your team knows where the line sits. - Keep agent limits outside the agent's reach.
In September, a flaw in DeepSeek Harness let attacker supplied text push a coding agent to switch off its own file sandbox with a single command.
The project patched it in the next npm release line, and the lesson is blunt. A rule an agent can edit is not a rule. A quick test of your LLM agents and tooling skills sharpens that instinct fast.
The old playbook was comfortable. Buy a guardrail for the model, announce that AI is secure, and move on to the next launch.
The new reality is less flattering. The model is the least interesting thing on your network, and the boring parts are the ones failing.
When did you last check whether a service facing the internet was still carrying its default credentials?