Skills inject, MCP tools decide: what 33 questions taught us about grounding an AI agent

il y a 10 minutes
8 min de lecture

Same content, two ways to deliver it, opposite results. In August 2026 I shipped the OptimNow Cloud FinOps library twice: once as a skill uploaded to claude.ai, once as a hosted MCP server.

Then I asked both the same 33 questions, the kind a FinOps practitioner asks every week. The skill answered from the library on questions the MCP server never looked up, in 4 test cycles, with the runbook available on the server the whole time.

In short: a skill injects its content into the model's context, so the model always has it. An MCP server exposes tools, and the model decides for each question whether to call one. That decision fails most often on the questions where the model feels confident enough to answer from memory.

If you distribute knowledge, put it in a skill. If you distribute live data, put it behind a tool, MCP or CLI depending on where the agent runs. This article explains why, what it means if you are choosing between a skill, an MCP server and a CLI for your own agent, and what it costs in tokens once you get it right.

How I tested it

Everything below is measured behaviour on claude.ai with Sonnet on default settings, the model most users get. Where I state a hypothesis, I say so.

The test is a set of 33 practitioner questions, which I call probes. Each probe is pasted as-is into a new chat, one probe per chat. Scoring is binary: did the answer come from the library, shown by a visible tool call or a file-read marker plus content I recognise, or did the model improvise from its own knowledge? I call the first case "grounded". There is no partial credit for an answer that sounds right.

A probe is never rewritten, so later cycles can be compared with earlier ones, and every question the library fails becomes a permanent probe.

Two rules kept the exercise honest.

Before each cycle, I called the deployed server to check what it served, after one cycle had to be thrown away because it measured an old version still running.
And before the first tuning iteration, I wrote the stop rule: two failed iterations means the failure is structural, document it and stop.

The library under test is the Cloud FinOps skill: playbooks for named waste patterns (idle NAT gateways, snapshot sprawl, expiring reservations) and reference files on billing mechanics, commitments and allocation, packaged for Claude Code, Cursor and ChatGPT.

Injected versus fetched

A skill and an MCP server can carry identical text and still behave differently, because they enter the model's reasoning at different points. A skill injects its content into the context. An MCP server exposes tools, and the model decides, per question, whether to call one. That decision is where grounding fails.

Three questions kept failing in the first cycles.

A detection question: how do I detect S3 buckets that should be on a lifecycle policy?
A symptom question: my NAT gateway processes 10 TB a month to S3, what should I do?
And an account-data question: which of my RIs are about to expire without a renewal decision?

I wrote the missing runbooks. The detection question was answered from the library immediately. The symptom and account-data questions kept failing with zero tool calls. The playbooks were live on the server and the model never looked. This happened twice, on two different playbooks. A coverage gap and a routing gap are different defects, and content only fixes the first.

The isolation test settled it. Same questions, connector switched off, skill only. The skill answered the symptom questions from the library on the first try, including one the connector had not managed in 4 cycles, and reproduced the playbook almost word for word. No routing work had ever touched the skill. Injected context has no per-question decision to fail.

The pattern in the failures is worth stating plainly. The MCP surface fails where the model feels confident: symptom questions it can answer plausibly from memory. It also fails where it misunderstands the question, which is the account-data class below. On lookup and discovery questions ("show me the idle waste runbooks") the connector works reliably. That is what MCP does best.

Question type	Example	Skill (injected)	MCP connector (fetched)
Lookup, discovery	"Show me the idle waste runbooks"	Answered from the library	Answered from the library, reliably
Advisory	"How should I size my first Savings Plan tranche?"	Answered from the library	Answered after description work; a gap out of the box
Detection how-to	"How do I detect S3 buckets needing lifecycle policies?"	Answered from the library	Answered from the library
Specific symptom	"My NAT gateway processes 10 TB a month to S3"	Answered first try	Answered only after routing rules in the tool descriptions, and not on every phrasing
Account data	"Which of my RIs are about to expire?"	Does not trigger	Does not trigger; at best offers the library

Routing is probabilistic. Two questions of the same type were answered and failed in the same cycle. Description work moves a probability, not a switch, and one run is a sample. Twice, the assistant's own memory feature made a failed call look grounded. A plausible answer is not proof of grounding.

Does the MCP server instructions field work?

The MCP specification lets a server include an instructions field, meant to tell the model how to use the tools. I put the routing rules there first: you cannot see the user's account, hand over the runbook instead of asking for an export. No effect. One transcript showed the model contradicting a rule that sat in the served instructions. On claude.ai, the server instructions apparently never reach the model.

Moving the same rules into the tool descriptions made the symptom question work: tool called, runbook content in the answer. Where the rules sat inside the description mattered as much as the rules themselves. The same rules in the middle of the description had done nothing.

Tool descriptions feed two consumers. The host runs a tool-search layer that decides whether a tool is shown to the model at all, and it responds to concrete service nouns. The model then decides whether to call the tool, and it responds to how the rule is phrased and where it sits. The description has to work for both.

I am keeping the exact phrasing, the placement pattern and the probe set out of this article. If you are shipping an MCP server and want them, write to me and I will share the details.

Does a rich skill cost too many tokens?

The usual objection to skills is context cost: a rich skill loads on every invocation, so people cut it down to a short installation note and push everything else into a CLI or an API. The measurement says the constraint is elsewhere. The skill can stay rich if it is layered: a small entry point and files loaded on demand.

Three optimisation waves in August 2026, each documented in its pull request with its measurement method:

Splitting the AWS and Azure reference files into core, commitments and patterns took a routine AWS Savings Plans question from about 44K tokens of loaded context to about 8.3K, and an Azure commitment question from about 43K to about 13.6K.
Reducing the entry point from about 5.8K to about 3.1K tokens saved a further 4K to 5K per routine billing question.
On the MCP side, a section parameter on the reference tool took a large Azure section from about 21,800 tokens to about 1,140, and a miss now returns the list of available headings, 139 tokens, instead of the whole file.

These are counts of context served per question type, not an average over the library and not a billing measurement on a live run. The trade-off I see is flat loading against progressive loading, not slim against rich.

The question every surface gets wrong

"Which of my RIs are about to expire?" failed on every surface, every cycle, through every intervention. The model reads "my" as "I need account access", apologises, and asks for a CSV export, instead of handing over the detection procedure that would let the user answer the question themselves. The best state I reached is a model that offers to check the library.

Neither a skill nor a knowledge MCP server sees the user's cloud account. For this question class, the right deliverable is the runbook's detection query, which the user runs against their own billing data. Models do not offer it on their own. The one-turn fix on the user side works every time: ask for the library explicitly. "Check the playbook library" or "show me the runbook for this" works when the model does not call the library by itself.

The other fix is architectural, and it is where the CLI enters the picture. A FinOps team at a large group described to me last week the setup they are moving towards: a REST API that carries authentication, authorisation and logging, a CLI on top of it as the agent's client, and one thin skill that introduces the CLI. Their agent sees the account. "Which of my RIs expire in 90 days" is a normal query for them and has no answer in a knowledge library.

That setup answers a different question than mine. It fits an internal team running an agent inside its own perimeter, with the CLI installed and authenticated on every machine. It does not fit a practitioner who is not a developer and asks from claude.ai or Copilot without installing anything, which is who the Cloud FinOps skill is for.

The choice between the 3 surfaces depends on where the agent runs.

Knowledge and instructions belong in a skill, loaded once and versioned.
Live data belongs behind a tool: MCP when the assistant is hosted, a CLI when the agent runs where you can install one.

There is a security reason to keep the two apart as well. If one channel returns data and instructions together, the agent has no way to tell a legitimate instruction from one injected through a resource name or a tag value. Instructions in the skill, data through the tool, never mixed.

One open hypothesis

With both surfaces enabled, the connector appears to take priority over the skill on symptom questions. In every cycle with both enabled, the skill never fired on those probes, while alone it answered them. If confirmed, enabling both works worse than the skill alone on that question class. One dedicated cycle would settle it. Until then it is a hypothesis, and I would rather publish it as one than wait.

The numbers

The set holds 33 probes today. The last full cycle, on 20 August 2026, ran the 32 that existed then: 25 answered from the library, 5 improvised, and 2 probes designed to hit known coverage gaps, which they did. The 33rd probe was added 9 days later, when the client question behind it became a permanent regression probe. The symptom question needed 2 iterations of routing work to work on the MCP surface; the skill answered it on the first try with no routing work. The account-data question: 0 spontaneous library calls across 4 cycles and 3 interventions, on every surface. The decisive isolation test cost one evening, no infrastructure, a connector toggle and 6 fresh chats.

If you run FinOps on AWS, Azure or GCP and want to see how the library answers your own questions, install the Cloud FinOps skill in Claude Code, Cursor or ChatGPT and start with the question your team asked last week. If it improvises, send me the question. It becomes a probe.

Frequently asked questions

Should I use a skill or an MCP server to give an AI agent domain knowledge? A skill, when the content is knowledge or instructions. The skill is loaded into context once, so the model does not have to decide whether to look. Measured on claude.ai with Sonnet in August 2026, the skill answered symptom questions from the library on the first try where the MCP server never called the tool.

When is an MCP server the right choice? For live data and for lookup or discovery questions, and whenever the user works from a hosted assistant (claude.ai, Copilot, ChatGPT) and cannot install anything. On lookup questions the MCP connector was reliable in every cycle.

When is a CLI the right choice? When the agent runs on a machine where you can install and authenticate the CLI, typically an internal team's agent, and when the agent needs to see account data behind a REST API.

Does the MCP server instructions field reach the model? On claude.ai, in my tests, no. Rules placed there had no measurable effect; the same rules in the tool descriptions did.

How much context does the Cloud FinOps skill load? About 3.1K tokens for the entry point, then files on demand. A routine AWS commitment question loads about 8.3K tokens, down from about 44K before the August 2026 restructuring.