The skill banned a character. The skill used it 54 times.

You wrote the rule down for your AI. Plainly, unambiguously. And it still breaks that exact rule, over and over. You go back and re-read the rule, and the rule is fine.

It is maddening because it looks like defiance. But measure it and you find the problem is not that the rule is unclear. It is that the rule says what not to do without ever showing what to do instead.

Here is our own case. We have a skill called kien-thai that governs the Thai prose style across everything we publish. One of its rules says plainly: never use a long dash as an all-purpose hinge for an aside. Keep it for numeric ranges only.

One day we put that rule on the scale. Same Thai writing task, 3 runs, skill fully loaded. Result: passed 0 runs out of 3.

Worse, the control arm (the same task run with no skill loaded, so you can tell what the skill changed) passed 1 out of 3. The skill was making the output worse than having no skill.

This is the hunt for why, and what we found changed how Productize writes Claude skills. Every number here comes from runs done in a single day.

Part 1A rule the model read and broke anyway

The rule under test is written in a reference file of the skill, roughly like this:

Reserve the long dash for a genuine numeric range or a dialogue dash. Never use it as an all-purpose "and now an aside" hinge.

It is a searchable rule with no interpretation needed, so the checker is easy to write: count the character in the answer, then exempt only the ones sitting between two numbers, per the exemption the rule states itself.

We did not trust that checker straight away. We made it go red first (red means the checker says fail, green means pass). Feed it Thai text containing the character and it must fail. Feed it text without and it must pass. Feed it text that is not Thai at all and it must return unjudgeable rather than violation. Feed it text with only a year range and it must pass. All 4 directions.

A checker that has never been seen red and a checker that is dead look exactly the same. So before trusting a green, you have to have seen it red.

Part 2It was not a case of the skill never firing

The first explanation anyone reaches for is that the skill did not load, or the model never picked it up. So we opened the run log for the actual round, line by line.

Read the last two rows together and the picture is clear. The model invoked the skill, opened the file the rule lives in, and violated that rule in the same round.

That matters because it cuts away every popular explanation at once. Not a bad install. Not a trigger failing to fire. Not a rule buried in a file nobody opens. The rule was read, and then it lost to something else in that same file.

Part 3The cause was inside the skill itself

So we went back and counted how many times the skill's own files used the character it forbids.

The raw count is not the answer though, because some of those are supposed to be there. An example labelled as a violation has to use the character, otherwise the rule has nothing to point at.

The real problem was a line like this one, sitting in the same file as the rule, labelled good:

Good opener: living paycheck to paycheck, never anything left over [long dash] sound familiar?

In the real line the bracket is the character itself. We replaced it with its name here because this blog runs its own check that rejects that character in every file before publishing, and we would rather not blind the check in order to write about checks.

That line is a good example of a different rule, about how to open a piece. But it uses the long dash as a hinge, which is exactly what the rule under test forbids.

The model reads the whole file before it starts writing. So it sees both the instruction saying never, and a Thai example labelled good doing it. The example wins. And it wins every time.

Put it in human terms and it is not surprising at all. Tell someone "don't do it this way" while handing them a sample that does it that way, and they copy the sample. Nobody memorises the instruction.

Part 4Four lines changed, and the result flipped

The fix was not to make the rule stronger. We did not add a single "never" or bold anything. We changed the examples.

1. Four lines labelled as good examples switched from the long dash to a plain space.
2. The rule was lifted into the main skill file as a short list, instead of living only in a reference file.
3. A note was added marking which examples deliberately demonstrate the violation, so nobody sweeps them later.

Then we ran the same command again. Same task, same number of runs, same checker.

The control arm did not move, which is what should happen, because we did not touch the task or the checker. We only touched what is inside the skill. The single variable that changed was 4 lines of examples.

We re-ran it later the same day and got 3 of 3 again, with the control arm at 0 of 3 that time, which tells you the bare arm naturally wanders between 0 and 1 rather than sitting on a fixed line.

Part 5Running this check on your own skills

The question worth asking of every skill that states a searchable rule is: how many times does this file do the thing it forbids?

It is answerable from the files alone, with no model call at all. So we wrote a short checker that walks all 141 skills in a few seconds and wired it into the job that runs automatically on every skill sync. It costs nothing, and it catches this class of fault before you spend anything on model calls.

The order we recommend, learned by walking it wrong:

1. Separate the deliberate from the accidental first. Examples labelled as violations have to break the rule; that is their job. What needs fixing are examples labelled good that break it, and the skill's own prose doing it by accident.
2. Make the checker go red before trusting a green. Feed it something that must be rejected and confirm it rejects.
3. Always compare against an arm with no skill. Otherwise you cannot tell whether the result came from the skill or from a model that could already do it.
4. Change one variable at a time. Edit the rule and the examples together and you will never know which one worked.

The takeaway you can use today: when adding a rule to a skill, spend your time on the examples rather than the wording of the rule. What the model imitates is what it sees, not what it is told.

This is one piece of a larger measurement of the whole skill library. For the full picture of how many layers there are and what each one costs, read 60 skills were never called, so we measured before deleting. And if you are still unsure how a skill differs from a long prompt, start at a skill is not just a longer prompt. Everything else lives on the blog index.

Sources and references

Every number here comes from runs against our own skill on 13 August 2026. None are borrowed from elsewhere.

• The two-arm results came from a runner firing the same task twice, once with skills fully loaded and once with every settings source disabled and the working directory set to an empty folder with no house rules anywhere up the tree.
• The long-dash counts were taken from the real source files, not from a cached copy.
• The verbose run log was read from the same round that produced the reported result, not from a fresh round run afterwards.
• This case's checker was proven able to go red in 4 different directions before it was used to judge anything.