When a new teammate joins, the first thing you teach them is the team's vocabulary. "Only members can buy a season pass." "Late returns cost 200 won per 5 minutes." The idea of an ontology is to give AI the same kind of dictionary. Thanks to Palantir, you hear the word everywhere these days.

It started with an experiment a viewer sent us. They gave a coding AI a PRD and an SDD, and after that the ontology added almost nothing. Tokens and time were actually lower without it. We wanted to see if that holds, so we measured it in two very different settings.

This post covers:

1. What an ontology is, and how it differs from similar terms
2. Where it works well, and where it overlaps with coding specs (PRD, SDD)
3. Experiment 1: a small coding task (a bike-share fare module)
4. Experiment 2: a chatbot over a domain-heavy insurance database
5. When to use one, and how to start

How to read the numbers. [Official docs] are materials published by the company or standards body, [Paper] is a public paper, and [Measured] means we ran it ourselves with Claude Code (Sonnet). All experiment code and result files are in the repositories linked at the end.

1. What is an ontology?

The word comes from philosophy, the study of what exists. In computing, the most cited definition is Tom Gruber's from 1993: "an explicit specification of a conceptualization" [Paper] (Gruber 1993).

Put simply, it is a write-up of what exists in a domain and how those things connect, in a form machines can read too. Think of it as a dictionary with relationships and rules attached. Using Seoul's public bike-share (Ttareungyi) fares as an example, you write down four things:

• Concepts: a pass, and under it, day passes and season passes
• Properties: each pass has a price and a base rental time
• Relationships: a user can buy a pass
• Rules: past the base rental time, 200 won per 5 minutes; only members can buy season passes [Official docs] (Ttareungyi pass guide, checked 2026-10-28)

How it differs from similar terms

It helps to line them up as steps.

The W3C, which sets web standards, points out the same distinction: taxonomies and thesauri do not assert rules or facts [Official docs] (W3C SKOS). In short, the ontology is the frame, and the knowledge graph is the data inside it.

2. Where it works, and where it overlaps with specs

Where it works

• Palantir: its docs call the ontology an organization's digital twin. It has four building blocks: object types, properties, link types and action types. If a flight is an object, its departure time is a property, the connection between a flight and an airport is a link, and rescheduling is an action [Official docs] (Palantir Foundry Ontology).
• Google: in 2012 it added the Knowledge Graph to Search, "things, not strings." At launch it held more than 500 million objects and 3.5 billion facts [Official docs] (Google blog).
• A study that measured the effect: on an insurance company database in 2023, GPT-4 answering directly in SQL got 16% right. With the same data turned into an ontology-based knowledge graph, it got 54% [Paper] (Sequeda, Allemang & Jacob 2023).

They have two things in common: places where many people and systems must use the same words, and places where you need to ask about relationships between data.

Where it overlaps with coding specs

In AI coding, you usually write two documents first. A PRD (product requirements document) says what to build and why. An SDD (spec-driven development) design spec says how. GitHub's Spec Kit spec template has a Key Entities section: what the things involved are and how they relate to each other [Official docs] (GitHub Spec Kit).

Those are an ontology's concepts and relationships. The spec already contains a slice of an ontology. So is it better to include both, or do they just overlap?

3. Experiment 1: a small coding task

Setup

• Task: build a fare module fare.js for the bike-share. Three functions: price, extraFee and canBuy.
• Knowledge files: we gave Claude Code only the original fare guide (source.md) and had it write ontology.md, then PRD.md and SDD.md with the same content. The facts had to match for a fair comparison.
• 7 conditions: task only / +ontology / +PRD·SDD / +both / +both (one side has an outdated price) / +PRD·SDD + visible tests / +both (pointers, explained below)
• Grading: each run starts in an empty folder with only that condition's files and the same task. When it finishes, 28 hidden tests the AI never sees grade the result. Three runs per condition.

Results

[Measured] How to read it:

• Any knowledge at all gets a perfect score. Telling the AI the rules clearly works.
• Ontology only ($0.209) and spec only ($0.217) are about the same.
• Including both cost 10–14% more than either alone, and time went from about 27 seconds to 35, nearly 30% more. Accuracy stayed the same. This matches the viewer's experiment.

Task only: it makes things up, plausibly

In the task-only code, the day pass has no 3-hour option, and season-pass prices are invented, such as 2,000 won for 7 days and 10,000 won for 365 days. The real prices are 3,000 and 30,000 won. Unknown rules get filled in with plausible guesses. That is what happens when you ask without knowledge.

Drift: when documents disagree, there is more to check

In the drift condition, we deliberately changed only the 30-day, 1-hour pass price in ontology.md to 4,000 won and left PRD and SDD at 5,000. Claude flagged the mismatch, said it implemented 5,000 because two documents agreed, and asked which was correct. Majority vote happened to be right this time. If two documents had been wrong, it might have picked the wrong value. This condition also had the most turns, 10 on average.

The pointer trap

What if you write each fact in only one place and point to it everywhere else? We removed the price table and rules from PRD and SDD, replaced them with references like "follow R7 in ontology.md," and measured again.

Accuracy was 100%, but at $0.273 and 50.5 seconds it was the most expensive of all seven conditions. The rewritten SDD gained a line saying "use the price table in ontology.md as the single source; do not copy values into code or this document," and Claude followed it literally: it wrote code that reads ontology.md and parses the table on every run. An explanatory document became a part of the program. A small change to the table's layout could break fare calculation.

If a value is needed in many places, we recommend keeping it in one machine-readable file, such as a single JSON fare table, and having code, docs and tests all use that file. We did not measure this approach this time.

Larger experiments point the same way

• A 2026 paper evaluating AGENTS.md: adding context files to repositories mostly left success rates unchanged, while inference cost rose by more than 20% on average [Paper] (Gloaguen et al. 2026).
• Chroma's experiment: across all 18 models, even simple tasks became less reliable as input grew longer [Official docs] (Context Rot).

4. Experiment 2: an insurance data chatbot

Real company data is much messier: many tables, full of codes and industry terms. This time we built a chatbot over the insurance database released with the 2023 study above (13 tables, 44 questions with reference SQL) [Official docs] (cwd-benchmark-data, Apache-2.0).

Where data chatbots go wrong

A data chatbot (text-to-SQL) takes a question in plain language, has the AI write SQL, runs it and shows the result. But some things cannot be known from the tables alone.

• Codes: to find policies sold by an agent, you have to check a role-code column. The AI tends to guess AGENT or AGT, but the real code is AG, and policyholders are PH. Only the company that built the data knows that.
• One column, many meanings: four kinds of claim money (loss payment, loss reserve, expense payment, expense reserve) are mixed into a single Claim_Amount table. Which kind a row is depends on whether the same ID appears in a sub-table.

The public data includes an ontology in the OWL standard format (ontology.ttl, 11 classes and 10 relationships) and a mapping file (mapping.r2rml) that says which table, column and condition each concept comes from. The rule "an agent is a row with role code AG" is written all the way down to a SQL fragment.

Setup

We had Claude Code build the chatbot from scratch and compared four modes that change only the material put into the prompt.

The self-built ontology followed the seven steps of Stanford's guide "Ontology Development 101" [Paper] (Noy & McGuinness 2001):

1. Set the scope, starting with the questions it must answer (competency questions). These become the test later.
2. Look for existing standards (FIBO for finance, schema.org for the web).
3. List every term in use, without classifying yet.
4. Pick the independent ones as classes and arrange them in a hierarchy.
5. Define relationships between classes.
6. Define constraints (how many, number or date).
7. Connect to real data (mappings like agent = role code AG).

Claude queried the database and confirmed the only codes were PH, AG and Year. But because there is no code-definition table, it marked "PH = policyholder, AG = agent" as an [assumption]. That cannot be known from the data alone; someone who knows the business has to confirm it. Along the way it also found what looks like a data error: one claim seems to be linked to coverage on a different policy.

Grading compares the result of running the chatbot's SQL with the result of running the reference SQL. Column names and row order are ignored, and numbers are matched to two decimal places. The reference answers were not in the chatbot's working folder; they were added only at grading time. For 1 of the 44 reference queries (average processing days), we only swapped the date-difference function so it runs on SQLite. 44 questions × 4 modes × 3 runs: 528 questions in total.

Results

L/H means low/high complexity, Q is the question and S is the schema. LQHS is an easy question over a complex schema. [Measured]

• Schema only 62% → public ontology 77% (+15 points) → self-built ontology 82%. The opposite of the small task.
• The biggest gap was on easy questions that require joining many tables (LQHS): from 37% to 90% with the self-built ontology, because how to join the tables is all written down.
• The data guide scored 74%, close to the public ontology, at $0.0329, about 60% of the ontology modes' cost (around $0.055).

Why the self-built ontology won

The public ontology is a general industry model, so it includes concepts this database does not have. The self-built one queried the database and wrote down the codes and data shapes that actually exist (IDs stored as numbers, premiums being only some rows, and so on), then ran its own questions as SQL and fixed mistakes in advance. Because it was built from this database, though, it has to be rebuilt for other data.

In 2023, GPT-4 answering directly in SQL got 16%. Today's Claude gets 62% with the schema alone. Models really have improved. Still, adding an ontology lifts it to 77–82%. However good the model gets, someone has to tell it company-only conventions like "AG means agent."

5. When to use one, and how to start

An ontology and an SDD are not competitors

An SDD's Key Entities section is the slice of the ontology that one feature uses. They relate like a big map and the directions cut out of it.

Four signs it is worth investing

1. Data with many tables, where answers need several joins
2. Data full of codes and abbreviations like AG and PH
3. Data where one column carries several meanings, like four kinds of money in one amount column
4. Places where many teams and systems must use the same words

Recommendations by project size

• Small projects (one repo, a few features): skip a separate ontology. Put entities and relationships in the SDD's Key Entities, and lock important rules in with tests. Experiment 1 was this case.
• Medium projects (the same terms and rules repeated across many SDDs): keep a lightweight ontology of terms and rules in one place, and keep values in one machine-readable file.
• Large projects and company data (many systems and AIs sharing vocabulary): a formal ontology with mappings, an owner, and automated checks.

The medium and large recommendations are drawn from the cases and papers above, not from our measurements.

Where to start

1. Start with a data guide. In our results it was the cheapest option with a large effect.
2. Let AI write the first draft. But anything marked [assumption] must be confirmed by someone who knows the business.
3. Keep the competency questions as a test. Rerun them every time you change the guide or ontology, and measure accuracy.
4. When many systems must share the same vocabulary, move up to a formal ontology and mappings.

And at every stage, put rules that must never be broken in checks, not prose. Claude Code's official docs also recommend moving instructions that keep getting missed into hooks [Official docs] (Best practices).

Limits of this experiment

• One model (Sonnet), three runs per condition. Experiment 1 is a single small task, so look at the direction more than the percentages.
• Experiment 2 is a single public benchmark (an insurance database, 44 questions). Unlike the 2023 study, we did not stand up a separate knowledge graph system; we only changed the material given to the same SQL chatbot.
• The self-built ontology was built from this database, which favors it on this database.
• We did not measure keeping values in one machine-readable file.

Watch the video

We walk through both experiments with the actual runs in a video. The video is in Korean.

• Do you really need an ontology? We measured it with Claude Code

Sources

• Gruber, A translation approach to portable ontology specifications (1993)
• Noy & McGuinness, Ontology Development 101 (2001)
• Palantir, Foundry Ontology core concepts
• W3C, SKOS Reference
• Google, Introducing the Knowledge Graph: things, not strings (2012)
• Sequeda, Allemang & Jacob, A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases (2023) · data: cwd-benchmark-data
• GitHub, Spec Kit
• Gloaguen et al., Evaluating AGENTS.md (2026)
• Chroma, Context Rot (2025)
• Anthropic, Claude Code Best practices
• Seoul Public Bike, Ttareungyi pass guide (checked 2026-10-28)
• Practice repositories: ontology-practice (experiment 1) · ontology-chatbot (experiment 2)