A synced copy of a live system starts rotting immediately
Two different problems hide under one phrase. If your knowledge is documents - contracts, manuals, closed tickets - the work is retrieval, which is RAG for an enterprise knowledge base and fails in its own particular ways. This piece is about the other problem, where the answer is not written down anywhere. It is a value sitting in a system right now, and it will be a different value this afternoon.
Take the question a support desk gets twenty times a day: where is order 41207? Nothing about that answer is a document. It is a row in an ERP, a status pulled from a carrier, maybe a note somebody added to the ticket ten minutes ago. Sync all of that into a vector store overnight and you have built a machine that answers yesterday's question in a confident voice.
The second cost shows up later. A copy is a new place your data lives, with its own access rules and its own deletion story. If the original had rules about who may see a customer record, the copy either reimplements them or quietly drops them, and teams find out which after somebody answers a question they should not have been able to. Worth settling first: whether the process is worth automating.
Connect AI agents to your data through a tool layer
The alternative is old and unglamorous: give the model a set of functions it may call, and let it decide when to call them. The model holds no data. It holds a menu, each entry with a name, a description and a schema for its arguments. A question arrives, it picks from the menu, and reasons over the structured result. MCP is the current standard for describing that menu, and its central object is the tool: a function defined by a JSON Schema for its input and, optionally, one for its output.
We run this where the freshness problem is not theoretical. Our voice race engineer answers a driver out loud, mid-session, in under two seconds, over telemetry that changes continuously. Nothing is pre-loaded. The telemetry is exposed through FastMCP as 19 structured tools across six categories - raw telemetry values, session info, leaderboard, flags, pit commands and lap statistics - and the agent reads what it needs when it is asked.
The glue is thinner than people expect. The MCP client converts tool schemas automatically into OpenAI function-calling definitions, so no hand-maintained mapping drifts out of sync when a tool changes shape. Add a tool on the server and the agent can use it.
And the agent, not you, decides the order. "Should I pit this lap?" fires get_current_pit_service_status, get_car_average_lap_time, get_leaderboard and get_my_car_damage_report in one reasoning loop before a word comes back. The business version is duller and identical. "Can we ship order 41207 today?" is a stock check, then a carrier cutoff, then whether the customer sits on a credit hold, and nobody wrote that sequence down in advance.
Sync it, call it, or leave it alone
Not every source belongs behind a tool. The decision is per data source rather than per project, and most companies end up using all three answers in the same system.
| Approach | Fits when | What you take on | Where it bites |
|---|---|---|---|
| Sync a copy | The answer lives in documents or reference data that moves slowly | An ingestion pipeline and a freshness promise you now own | Answers inherit the age of the last sync, and the copy is a second place your data lives |
| Call a tool | The value moves during the working day and the question names a record: an order, a ticket, a stock level | One adapter and one schema per question, plus latency on every answer | The agent is only as available as the system behind it |
| Expose nothing | Nobody can name the question it answers, or a wrong read costs more than a right one is worth | Nothing, which is the point | A person keeps doing the lookup by hand |
The third row saves more projects than it looks like it should. An agent that cannot answer a question is annoying, and people route around it in a day. An agent that answers wrong from a source nobody scoped is a ticket someone spends a week tracing. If you are weighing a ready-made connector against writing the layer yourself, the usual thresholds apply.
The size of a tool is the decision people get wrong
Two failure shapes here, pointing in opposite directions. Too coarse is the popular one: a single run_query tool that takes SQL. It looks elegant in a demo and it hands the model your whole database, plus a full table scan whenever it guesses badly. You also lose the ability to say who may call what, because there is only one thing to call.
Too fine is quieter. Forty tools that each return one field means the agent burns turns assembling something one query could have answered. Selection gets harder too, since the more near-identical names sit on the menu, the more often the model reaches for the wrong one.
The size that holds up is one tool per question a person would actually ask. Look at the race engineer's names again: get_current_pit_service_status, get_car_average_lap_time, get_my_car_damage_report. Each is a question, not a table. Ported to a company they read get_order_status(order_id), get_open_tickets_for_customer(customer_id), get_stock(sku, warehouse). Nineteen of those covered a whole domain at racing speed, and most business domains need fewer than people expect.
Write the descriptions like they matter, because they are the only thing the model reads when choosing. The name, the description and the input schema are that tool's prompt, and a vague description gets called at the wrong moment. MCP also lets a tool declare an output schema for clients to validate against, which is worth the ten minutes: a tool allowed to return anything eventually will.
Sizing is also where your latency budget goes, because every call is a round trip plus another model turn, and the agent decides how many to make without consulting you. The race engineer stays under two seconds by running independent calls concurrently, returning the fields a question needs instead of whole records, and reading pre-aggregated values from SQLite. If the system on the other end takes four seconds, no agent design repairs that, and every retry is another model turn on the running cost.
The tool boundary is where permissions actually live
A model cannot enforce access control. It has no idea who is asking, it has text, and text is easy to talk into things. Every rule about who may see what sits in the layer underneath, and the tool boundary suits it because it is narrow and it is code you own.
Identity gets resolved server-side: the tool takes the caller from your own session and filters before returning anything. The tempting shortcut is closed off on purpose. MCP's security guidance says a server must not accept tokens that were not issued for it, so forwarding a user's downstream API token through the agent is out. Scopes stay small for the same reason it warns against omnibus scopes like *: one leaked broad token is every tool at once.
Process isolation is cheap here, so take it. In the race engineer the MCP server runs as a separate process over stdio, which buys clean isolation and no shared-memory races against the live feed. Same shape in a company: whatever holds your database credentials is not the thing talking to a model provider.
One more catches people off guard. Tool results are untrusted input. Whatever comes back from a ticket body or a supplier's product description lands in the model's context, and if somebody wrote instructions in that field, the model reads them as instructions. The spec tells clients to validate results before passing them on. Customers can type into your CRM.
Fresh data goes stale inside the conversation
This is the failure that took us longest to see, because it never appears on the first question. Only on the second.
The natural design keeps the whole exchange in the conversation, tool calls and their results included, the way a chatbot keeps a transcript. In the race engineer that poisoned the next answer. A lap time captured 30 seconds earlier is already wrong, but sitting in the context it is just text, indistinguishable from a value read a second ago. Stale data carried forward is worse than no data, because no data at least forces a fresh read.
The fix is a sliding window. Only the question and the final text answer persist, MAX_HISTORY = 4, while every intermediate tool call runs fresh each turn and is discarded afterwards, with the system prompt pinned first. You pay for a repeated call and get an answer that is still true when it is spoken.
Substitute an order status read 30 seconds ago, or a CRM stage that changed while the user was typing their follow-up. Anything that can change between two questions gets retrieved at the moment of the question, never remembered from an earlier turn. If your agent answers the second question faster than the first, check whether it re-read anything at all.
History you do want lives outside the model. A worker polls the sim every 100 ms and writes completed laps for all 64 cars into SQLite, so the agent asks for lap history instead of carrying it. Same move in a company: a small job keeps the aggregate current, and the agent queries it like any other tool.
Reading can be autonomous. Writing should not be.
The strongest rule we carry out of production is an asymmetry rather than a policy. Let the agent read whatever its scopes allow, whenever it decides it needs to. Do not let it write on the same terms.
In the race engineer, reading is autonomous and high-risk pit commands are not: the agent can recommend and prepare a stop, but the driver confirms before anything reaches the car. The AI rector's assistant we built for a university splits the same way. It drafts, reminds and reports on its own, while assignments are confirmed by a person and every action is logged, so there is a trail of who approved what.
The protocol agrees, which is a decent sign it is not just our taste. MCP's tools spec says there should always be a human in the loop able to deny a tool invocation, and that a host must obtain explicit consent before invoking one. A refund sent to the wrong customer does not get fixed by explaining the mistake to the model.
Make the gate structural rather than verbal. Separate read tools from write tools at the server, put the write ones behind a scope that requires approval, and log the payload of every write next to the identity that approved it. A line in the system prompt saying to always ask before writing is not a permission system. It is a suggestion, and it holds until the conversation where it does not. If the system touches customers or staff inside the EU, that gate stops being only good engineering, which is the ground we cover in what the EU AI Act actually binds you to.
A wiring checklist you can run without us
No framework and no vendor is required for any of this. If you already have an agent talking to a live system, or you are a week away from pointing one at your ERP, the checks below run in an afternoon.
| Check | How to run it | What it catches |
|---|---|---|
| Name the questions first | Write the ten questions people actually ask, in their words. Each survivor becomes one tool. | Tool sprawl, and generic query tools nobody can audit |
| Read-only by default | Ship every tool read-only first. Add a write tool when someone names the record and who signs off. | Silent writes against production records |
| Identity resolved server-side | Call a tool as a user who should not see a record and confirm nothing comes back. | Access rules that live in a prompt |
| Small responses | If a tool hands back a whole record when the question needs three fields, trim it. | Context bloat and slow answers |
| Freshness rule | Ask two questions where the value changes in between. Confirm the second re-read the source. | Stale data carried forward in the conversation |
| Untrusted content | Put a hostile instruction inside a record the agent reads, then watch for a behaviour change. | Prompt injection arriving through your own data |
| Worst-hour latency | Time the slowest system in the chain during its worst hour, not at 9am on a Tuesday. | An agent that is quick in a demo and unusable at month-end |
Run these before anyone outside the team touches it. Most of what goes wrong in this layer goes wrong quietly: the answer arrives, it reads well, and it was true half an hour ago. (This is the work we do most weeks, for what it is worth - the checklist stands on its own.) If you want a second opinion on which systems belong behind a tool and which should stay where they are, a short audit is a low-key way to get one.
FAQ
Do I need MCP specifically, or is plain function calling enough?
Plain function calling is enough when a handful of tools live inside one application and one team owns both ends. MCP earns its place when the same tools must be reachable from more than one agent, or when the team owning the data is not the team owning the agent, because then the schema belongs next to the system rather than inside the app. Question-sized tools, server-side permissions and fresh reads matter either way.
Is it safe to give an AI agent access to our production database?
Not directly, and that is the whole point of a tool layer. Instead of a database connection, the agent gets named functions that each answer one question, run under an identity resolved on your side, and return only the fields that question needs. It never writes SQL, so it cannot reach a table nobody scoped, and you can revoke one tool without revoking everything.
Our ERP has no API. Can an agent still read from it?
Usually yes, and the integration work gets smaller rather than disappearing. You write one adapter per question instead of a full two-way integration: a read against a reporting view, a stored procedure, sometimes a nightly export. That way you ship the two questions that matter this month and leave the rest of the ERP untouched.
How many tools should one agent have?
As many as there are distinct questions people ask, which is usually fewer than the first workshop suggests. Our live telemetry agent runs 19 tools across six categories, and that covers a whole domain under a two-second answer budget. If your list is heading past thirty, some of those tools are fields rather than questions.
- 24 July 2026Published.