Stop 6 of 10 · Weekly
Creative testing pipeline
A brief built from the registry and the winner history, variants generated one variable at a time, launched through the signature path, and read only when the pre-registered numbers arrive.
What the pipeline is for
Nour's agency used to ship two creative concepts a month across twenty accounts because writing and building them was the constraint. It is not any more, which is exactly why the pipeline needs a shape. Otherwise the agency ships forty variants a month and learns the same nothing, faster. Five stages, and only one of them is generation.
| Stage | What comes out |
|---|---|
| Brief | One document per account, built from the registry and the winner history |
| Generate | A variant matrix where every cell differs from the control in one named way |
| Review | A pass over claims, brand rules and the client's restricted terms |
| Launch | A proposal, signed, applied through the stop five path |
| Read | A decision, only once the pre-registered numbers arrive |
The brief comes from the systems you already built
Write brief.py. For an account code it reads the registry MCP server for industry, owner and the terms this client never bids on, reads creative_history.csv for every concept we have run on this account with its result, and reads the last three months of metrics.json for the campaigns in scope. It calls the Claude API once and writes briefs/<code>.md with: the audience in one paragraph, the three angles that have won on this account with the evidence, the two that have lost and why, the claims this client can support with the source for each, the restricted terms, and one hypothesis worth testing next stated as a sentence with a number in it. Anything the history does not support is written as an open question rather than an angle. Show me the brief for A09.
Generate one variable at a time
The matrix is the discipline. A control cell, then one cell per variable: headline angle, image subject, offer. Three variables tested one at a time is three readable results. Three changed together is one anecdote.
From briefs/A09.md, propose a five-cell test: the control as it runs today, then four cells that each change exactly one thing from the control. For each cell give the variable name, the exact copy, and an image prompt I can run through our image tool. For each image prompt, state what must not appear: no people in branded uniforms we do not own, no vehicles with visible plates, no text baked into the image, no interiors that contradict the client's actual service area. Then tell me the one variable you would drop if I only had budget for three cells, and why that one is the weakest test.
Review, then launch through the signature path
Two checks before anything runs. The claims check reads every line against the brief's supported-claims list and marks anything unsourced. The brand check reads against the client's restricted terms from the registry.
Read the five cells against briefs/A09.md. Mark every claim in every headline and description as supported, with the source line from the brief, or unsupported. Flag every superlative, number, guarantee and restricted term. Return a table and nothing else. Do not rewrite the copy.
The launch is a proposal in the stop five format, change_type status, with the cell definitions attached, signed by a person and applied by apply.py. There is no separate creative path.
Reading the result
Write the stopping rule into the proposal before the test runs: the conversions per cell Nour needs and the earliest date she will look. Then hold to it.
Write read_test.py. It takes a test id, pulls current results per cell from the platform, and compares them against the thresholds recorded in the launch proposal. If a cell is below the pre-registered conversion count, it prints how far short and returns no winner, whatever the difference looks like. Write test_read.py: forty conversions against a threshold of a hundred returns no winner, a met threshold with a clear difference returns the winner and the margin, and a met threshold with overlapping results returns no winner and says so.