We filed one issue, and Codex wrote a test first, fixed the bug, attached screenshots from before and after the fix, and opened a pull request. The bug: split 10,000 won among three people and the shares add up to 9,999 won. The human part was a single command.

In this post we call that line, where an incoming issue gets triaged, fixed, proven, and shipped as a PR, a software factory. That is our name for it, not a term from OpenAI.

In this post:

1. The starting point: OpenAI's harness engineering post
2. Two layers: harness and factory
3. Installing the harness, and two tests
4. Running the factory: triage → fix → spec → automation
5. Takeaways and limits

How to read the numbers. [Official] means OpenAI's own post and the Codex docs. [Measured] means values from our own runs. We ran everything on October 13, 2026, with Codex CLI 0.162.1, the GPT-6.1-Sol model (medium reasoning), on macOS, once per step. The screenshots come from our Korean-language video.

1. The starting point: OpenAI's harness engineering post

In February 2026 OpenAI published "Harness engineering: leveraging Codex in an agent-first world." A harness is the gear you put on a horse. Here it means the rules and tooling you put around an AI agent so it doesn't wander off.

• One OpenAI team built a product over five months with zero lines of manually written code. The codebase is on the order of a million lines, and roughly 1,500 pull requests were opened and merged [Official].
• The engineers' job was not writing code. It was designing the environment Codex works in, specifying intent, and building feedback loops. The post sums it up as "Humans steer. Agents execute." [Official].

Two lessons stood out:

• Give Codex a map, not a 1,000-page manual. Treat AGENTS.md (the guide file Codex reads first in a repo) as a table of contents, not an encyclopedia.
• Put the fix in the error message. When a lint rule fails, the message says where and how to fix it, so the agent can read it and act.

The post also adds a caveat: this end-to-end behavior depends heavily on that repository's structure and tooling, and should not be assumed to generalize without similar investment [Official]. So we made that kind of investment ourselves, in one small app.

2. Two layers: harness and factory

• Bottom layer: the harness. It keeps a single Codex run honest. It holds AGENTS.md with the rules, a guard that stops edits to tests, and a verify step that runs the tests right before Codex finishes.
• Top layer: the factory. An incoming issue gets triaged, fixed in its own isolated workspace, checked by a reviewer, and shipped as a PR. A human merges.

The practice app is a bill splitter: enter a total and a head count, get each person's share. We filed three issues:

1. A bug: the shares don't add up
2. A feature request: make the people who drank pay more
3. A short report that gives too little to go on

The repo holds five factory files: three skills (triage, fix, spec), one reviewer, and one runner script.

• A skill is a written procedure at .agents/skills/<name>/SKILL.md, invoked with $name [Official].
• The reviewer is a subagent: a second Codex launched in the middle of a task. It is defined in one TOML file under .codex/agents/, set read-only so it judges without editing code. The docs note this format may change [Official].
• AGENTS.md stays short. It lists four stages (isolate → implement → prove → ship) and points to the skills for details. That is the "table of contents" rule in practice.

3. Installing the harness, and two tests

We used our public codex-harness-kit (0.2.1). It is built on Codex hooks: a way to run your own script right before Codex uses a tool (PreToolUse) or right before it finishes (Stop) [Official].

• One install script adds three hooks (UserPromptSubmit, PreToolUse, Stop) plus a config file. doctor checks that the guard actually blocks [Measured].
• By default the whole test/ folder is protected. But in a factory, Codex has to write new reproduction tests (tests that recreate the bug and fail). So we added four lines: lock the reviewer, the skills, and the runner script too, and open only test/regression/ for reproduction tests.
• Codex won't run new hooks until a human reviews and trusts them [Official]. On first launch it shows "Hooks need review: 3 hooks", and they only turn on after you trust them.

Then we tested twice [Measured].

Test 1: ask Codex to delete a test. We asked it to delete the "error when head count is 0" test. The hook blocked it ("Blocked by hook"). Codex got a message saying tests are human-owned criteria and a human has to change them; it stopped and reported why. No files changed.

Test 2: tell Codex not to run the tests. We asked it to loosen the head-count check and added "don't run the tests." Codex made the change and tried to finish. Right before it finished, the Stop hook ran the tests, and they failed. The failure went back to Codex, which restored the original check, and it only finished after all 3 tests passed. The repo ended with no changes.

4. Running the factory

Triage

scripts/factory is a short script that opens Codex and types the skill name for you. The triage skill reads the three issues, calls the code with their inputs to see whether the symptom reproduces, and sorts them [Measured].

Codex spent 46 seconds on triage.

Fix: issue #1

You pass just the issue number, as in scripts/factory fix 1. The script creates a worktree for that issue and opens Codex inside it. A worktree is a second checkout of the same repo in another folder, so several issues can run at once without overwriting each other's files.

Codex worked in order [Measured]:

1. Read the issue and wrote a reproduction test first, confirming it fails.
2. Took a screenshot before the fix.
3. Fixed the code and took another screenshot.
4. Launched the reviewer subagent (factory_reviewer) to check scope, tests, before/after screens, and edge cases, and got a PASS.
5. Only after the PASS did it commit, push, and open the PR.

That took 2 minutes 14 seconds.

The cause: each share was rounded down, and the leftover won went to nobody. Now the first people pay one extra won each, so one person pays 3,334 won, two pay 3,333 won, and the total is exactly 10,000 won.

PR #4 explains why, the root cause, the change, and shows the before/after screens. It touches four files (two evidence images, src/split.js, test/regression/issue-1.test.js), +26 −2 lines, with 6 tests passing.

The reviewer's verdict, PASS, sits at the bottom. We did not merge it. That's a human call.

Spec first: issue #2

A new feature starts with a spec (a document describing what to build and how) instead of code. A draft PR opened in 2 minutes 39 seconds with a single spec file [Measured].

PR #5 lists what a human has to decide: how amounts are entered, how drinkers are marked, who pays the leftover won. It includes a worked example: five people split 120,000 won, 30,000 of it is drinks, and two people drank. The two drinkers pay 33,000 won each and the other three pay 18,000 won (33,000×2 + 18,000×3 = 120,000). Whether that's the right direction is for a human to decide.

Automation: issue #3

Issue #3 only had our question comment. We assumed the reporter replied with values, and this time ran the same triage skill with codex exec. codex exec runs Codex once with no UI and exits, which is what you use for scheduled automation or CI. Its default sandbox is read-only [Official].

Codex ran the values itself and found the same cause as issue #1. Instead of fixing it again, it relabeled issue #3 as factory:fix and left a comment linking to the open PR #4. No new PR [Measured].

5. Takeaways and limits

• Keep AGENTS.md short, like a table of contents.
• Treat tests as human-owned criteria, and give the agent only a place to write reproduction tests.
• Add a hook that runs the tests right before the agent finishes.

• One worktree per issue.
• A reviewer that is separate from the agent that writes the fix.

• This was a throwaway practice repo, so we ran with the sandbox and approval prompts turned off. Hooks protect tests and config; they are not a sandbox. In a real project, pair them with containers or approval rules.
• The timings (46 s, 2 min 14 s, 2 min 39 s) come from one run each in this repo. They will change with model, version, and repo.
• We only checked macOS. We didn't verify how hooks behave on Windows.
• The docs say the subagent config format may change.
• As OpenAI's caveat says, this works in repos that have invested in structure and tooling. If your project has no tests, start there.

Start with one issue in a project that already has AGENTS.md and tests.

Watch the video

We walked through the same process on screen, with the real runs, in a Korean-language video.

• Watch on YouTube: Building a software factory with Codex — from harness engineering to issue → PR

Sources

• OpenAI, Ryan Lopopolo, Harness engineering: leveraging Codex in an agent-first world (2026-02-11)
• Codex docs: Skills · Subagents · Hooks · Non-interactive mode (codex exec) (checked 2026-10-13)
• Practice repo: dante01yoon/nbbang-factory-codex — AGENTS.md, three skills, reviewer subagent, PR #4 (fix) and PR #5 (spec, draft)
• Harness kit: dante01yoon/codex-harness-kit (0.2.1)