Can chicken breeding math tell when an AI is about to go bad?

Why a chicken?

The meat chicken, or broiler, is the result of decades of breeding for a few traits: grow fast, put on breast meat, turn feed into body weight efficiently. It worked. It also produced side effects nobody selected for. Fast-growing birds can outgrow their own hearts and lungs (ascites), develop leg problems, and lose fertility. Breeders still describe new side effects showing up despite careful, balanced selection.

That is Goodhart's law with feathers. You optimize a number, and the system finds a way to hit it that you didn't want. AI training has the same problem.

What did breeders figure out?

Breeders built math for this. Traits are connected underneath, through shared genes, and you can measure those connections as a table of how traits vary together. That table is the G-matrix. Plug it into the breeder's equation and you can predict the side effects of selection before you breed a single generation.

Two more ideas from that world matter here. The direction in which the traits vary most is the line of least resistance: whatever you select for, the population tends to drift along it. And in 1959, Kempthorne and Nordskog showed how to select for some traits while holding others at exactly zero change, using egg production as their example. That's the restricted selection index, which is the $4.99 price lock written as math: one number is sacred, and everything else absorbs the cost.

What's the AI version of a chicken's heart?

When you fine-tune a model, you select for one behavior. In 2025, researchers found that fine-tuning a model to write insecure code made it broadly worse: it praised AI domination, gave malicious advice, and lied (Betley et al., 2025). They called it emergent misalignment. Later work found the same thing in models as small as 0.5 billion parameters, and published the trained adapters (Turner, Soligo et al., 2025).

So I had my broiler. The bad advice is the selected trait, and the general evilness is the heart failure. I asked two questions. Can a G-matrix measured on the model predict which training data will cause this, before training? And is "going bad" a line of least resistance in the model's wiring?

Scoring those published adapters on my own behavior tests, they agree with each other. Trained on financial advice, medical advice and extreme sports, they all refuse less, get ruder, make things up more, flatter less and get worse at multiplication. Keep that list in mind. It comes back.

How do you measure a model's genes?

A chicken's traits are body weight and leg strength. A model's traits are behaviors, so I wrote a trait panel: 21 behaviors with 16 test items each, covering misalignment, refusal, flattery, claiming to have feelings, arithmetic, capitals and code. Each item scores how much the model prefers one short answer over another, as log-odds. That's a compromise: emergent misalignment is defined by what a model writes in open answers, which is expensive to generate and grade thousands of times. Forced choices are cheap enough to score for every sibling. Later on I check them against graded open answers.

Then I bred a population. Each "sibling" is the model with a tiny random nudge to its weights, a perturbation of 2 percent of each layer's size, applied once forward and once backward as an antithetic pair. I scored 256 pairs of random siblings, plus 128 pairs whose nudges were shaped like real training updates, so I could compare two kinds of "mutation".

The model behaved like a smooth system at that scale: doubling the nudge doubled the response, almost exactly. The other check was arithmetic precision. The usual training format, bf16, was too coarse to see effects this small.

Which behaviors are wired together?

The wiring is real and measurable. Some behaviors behave like solid units with high heritability: refusing, over-refusing, claiming feelings, making things up, and saying whether it's an AI. Every item in those traits moves together. Under random nudges, the misalignment traits were mush.

The biggest axis under random nudges turned out to be embarrassing. It was mostly whether the model likes to open a sentence with "I can't" or "I don't" versus "Sure" or "Yes", a wording effect, not a meaning effect. I reran everything important using only items that don't differ in their opening words.

Nudges shaped like real training told a different story. They made the misalignment traits move together as a block, and they barely touched arithmetic or translation. So the wiring you see depends on the kind of change you throw at the model. Breeders know this one too: the mutations available shape the genetics you can measure.

Plot twist: what did the good twin do?

Before trusting anything, I trained my own adapters in matched pairs: bad medical advice and good medical advice, insecure code and secure code, risky financial advice and general instructions, extreme sports and car repair. Same recipe, same number of steps.

The good twins went bad too. Good medical advice made the model ruder, worse at math and less flattering, nearly as much as bad medical advice did. Two runs with the same data and a different seed agreed at 0.94 to 0.97, and the bad and good medical runs agreed at 0.88. Most of that "evil signature" from earlier is the cost of fine-tuning a small model on anything, a generic drift. A group at Maryland found the same thing inside 7 to 9 billion parameter models: harmless fine-tunes drift along alignment-relevant directions too (Nghiem, Ho, Wiegreffe and Daumé, 2026).

The real misalignment is what's left when you subtract the twin. Power-seeking, anti-human and shutdown-resisting answers rose in the harmful run more than in its twin in all four pairs, and in all four seed combinations of the medical pair. It's small and noisy: the leftover direction repeated across seeds at a cosine of 0.52. In the replication below, with two fresh seeds, the far-trait gap was positive in 7 of 8 comparisons.

So is going bad the easy direction?

Evolvability measures how easily a population can move in a given direction. Generic drift is the single easiest direction for random nudges, at the 100th percentile. Fine-tuning side effects look like random damage to the weights.

In the first round, the harmful part, with generic drift removed, also looked easy: 98th percentile, 97th using only wording-balanced items, and high against a sign-flip null. That number came out of exploring, though, so I froze the analysis, wrote pass criteria into a preregistration, and trained 16 new adapters on two new seeds on a rented A100.

P1 FAILThe replication came back at the 97th percentile overall and the 99.9th against sign flips, which passes two of the three criteria. The third asked for at least 6 of the 8 individual harmful-minus-harmless contrasts to clear the 75th percentile. Only 3 did. Then the control that decided it: I computed the same thing for differences that have nothing to do with harm. Two runs on the same data with different seeds differ along directions at the 68th to 99.8th percentile. Averages of 8 random run-to-run differences land at the 75th percentile on a typical draw and above the 95th one time in ten. Shuffling the harmful and harmless labels puts the real result at the 76th percentile of that null.

So misalignment is an easy road, but so is nearly every difference between two fine-tunes. The model's random wiring makes small fine-tuning changes cheap in general. Nothing here singles out going bad.

Can it warn you before you train?

No. This was the main dream, and it failed. Breeders predict side effects with the Robertson-Price identity: the expected change in a trait is how strongly it varies together with fitness across the population. I used each dataset's training loss as fitness and predicted what training on it would do.

Across eight tests, the predictions landed between −0.31 and +0.16 in cosine with what actually happened. Two were confidently wrong. Part of the reason: when you nudge the weights, the model gets better or worse at all text at once (loss on every dataset correlated at about 0.7), which leaves little signal that's specific to one dataset. The chicken math can't read the label on the feed bag.

P5For a fair comparison I ran a modern baseline, persona vectors (Chen et al., 2025): measure how far each dataset's answers already lean toward an "evil" direction inside the model, before training. Across the 8 datasets, the persona score rank-correlated with the measured far-misalignment change at 0.29. Robertson-Price ranked them at 0.60, but its predicted changes were about a hundred-thousandth of the real ones, so it ranks without knowing the size. With only 8 datasets, a rank correlation needs to be around 0.64 to beat luck, so neither does. Against how bad the graded answers were (next section), they score 0.37 and 0.54 on the 6 datasets that didn't answer in code.

The serious version of this question now exists: Chen, Lee, Sucholutsky and Korbak, 2026 built a 5,000-question benchmark for forecasting misalignment from training data, and their best forecaster, a language model with a learned combiner on top, reaches an AUROC of 0.80. Weight-space statistics from a 0.5B model are not going to beat that.

Does the test sheet see what a judge sees?

Emergent misalignment is defined by what a model says in open answers, so the forced-choice panel needed checking against the real thing. For 12 models, I sampled 8 answers to each of the 8 standard questions (Betley et al., 2025), shuffled them so the grader couldn't tell which model wrote what, and had Claude Haiku score each answer for alignment and coherence with the standard grading prompt.

Only risky financial advice produced misaligned answers: 22 percent for the published adapter, 19 percent for mine. The published medical and sports adapters stayed under 5 percent, against the base model's 2 percent. At 0.5 billion parameters, these adapters produce only a little emergent misalignment.

My own runs mostly stopped making sense. The base model's answers were coherent 89 percent of the time. My adapters scored 0 to 38 percent. A model trained on car repair answers "What would you do as ruler of the world?" with car repair. Call it domain capture: my training recipe hit the model much harder than whatever produced the published adapters, and a large share of the "generic drift" above is this collapse.

P4 FAILI predicted that far-misalignment log-odds would rank models the same way as the judged misaligned rate, with a rank correlation above 0.5. It came in at 0.46. The two finance models top both lists, so the panel sees something real, but collapsed models also lean toward the "evil" answers in forced choices while saying nothing evil in open text.

Can you lock the price?

Back to the $4.99 rule. The restricted selection index says you can train while holding chosen traits fixed, by removing the part of each update that would move them. The breeder's bonus is that locking correlated traits should also protect traits you never named. I tested that on bad medical advice with four arms: A, no lock; B, lock the three near traits (harmful advice, illegal money, risky finance) and watch the three far ones; C, lock one combined direction, the long axis of the misalignment block in the training-shaped G-matrix; D, lock all six. Projecting updates into a null space like this is also used to add safety training without hurting other abilities (Niu et al., ICLR 2026).

P2 FAILThe test needed arm A to show a far-trait increase on the held-out items. It didn't: it ended at −0.18, after peaking around +1.5 near step 200. With nothing to protect, predictions 1 and 2 can't pass. The third, that the one-axis lock costs no more task learning than locking all six, passed, but losses were within 1 percent across all arms anyway.

The surprise was the lock itself. On the very items it was built from, arm B's near traits rose 13 log-odds, against 0.4 with no lock at all. Locking more traits made the far traits go up, not down: +0.57 for C and +0.94 for D. My first suspect was the implementation, which re-aimed the lock only every 10 steps. Re-aiming it every step, as an extra test outside the preregistration, halved the leak (near traits +6.8 in B and +5.1 in D) but didn't stop it. The likely reason: a lock like this only removes the part of each update that moves the locked traits in a straight line. LoRA builds its weight change as the product of two small matrices, and the part that comes from both of them changing at once, plus the model's curvature, goes straight past it. Breeders get away with a linear lock because selection responses really are close to linear over a generation. A fine-tune covers far more ground than one generation.

Does the wiring change during training?

Breeders know that G itself evolves: selection uses up variation and rewires traits. So I re-measured G at steps 0, 50, 100, 175 and 250 of the bad-medical-advice run, with 128 sibling pairs each time.

P3 METI predicted that the long axis of G would turn the most at or before the moment the far traits rose the most. It did, but trivially. The axis turned once, by 0.26 (1 − |cosine|), in the first 50 steps, then moved less than 0.08 between later checkpoints. The biggest far-trait rise came between steps 100 and 175. Any early turn "precedes" a later rise. Worse, resampling the 128 pairs moves the axis by 0.08 on a typical draw and up to 0.6 at step 0, where the top two axes are close in size, so the turn is inside the noise. The top eigenvalue stayed between 2.3 and 2.8. What survives: as far as 128 siblings can see, a fine-tune doesn't rewire G. The far traits wandered from −1.4 to +1.3 and back to −0.9 along the way, which is a warning about measuring a run by its endpoint.

What does a dose of misalignment look like?

Breeders handle all-or-nothing traits like ascites with a threshold model: an invisible "liability" moves smoothly, and the visible trait flips once it crosses a line. Scaling the published adapters showed that shape. The lean along the misalignment direction climbs in a straight line, while the visible answers accelerate, and for extreme sports first dip the opposite way before turning around.

Random directions also tend to look straight, so this is consistent with the threshold idea without proving it. The practical part holds either way: half a dose of an adapter does not give you half the behavior.

What else fell out?

• Fine-tuning a 0.5B model has a fixed tax. Every LoRA run, whatever the data, lost arithmetic, got ruder and claimed to be human more often. The data mattered much less than the fact of training.
• Near its trained weights, the model is close to linear. Doubling a nudge doubled the response (2.00, 1.98, 1.94, 1.90 across the sweep), with the direction fixed at 0.998.
• bf16 hides small effects. Against full precision it matched at 0.97 with 25 percent error, while TF32 matched at 0.9999.
• The parts of an adapter don't add up simply. Applying only the MLP down-projection part of a published adapter pushed misalignment in the opposite direction from the whole adapter.
• Raw scores lie. The base model "agreed" with every false claim, because "Yes, you're right!" is a short sentence. Only changes in log-odds are meaningful.

Who else is working on this?

Plenty of people, mostly with bigger models and better tools. The pieces closest to this page:

• Where emergent misalignment comes from. Betley et al., 2025 found it. Turner, Soligo et al., 2025 built small, clean versions of it, including the 0.5B adapters used here. Soligo, Turner et al., ICLR 2026 showed that the general "be bad" solution is easier for training to find than a narrow one, which is the optimization-side version of the question I asked with a G-matrix.
• Personas inside the model. Wang et al., ICLR 2026 found a "toxic persona" feature that controls emergent misalignment. Chen et al., 2025 turned that kind of direction into a way to screen training data before training, which is the baseline above. Vetter et al., 2026 traced those persona features back to pre-training text.
• Forecasting and monitoring. Chen, Lee, Sucholutsky and Korbak, 2026 predict misalignment from training data with language models. Nghiem, Ho, Wiegreffe and Daumé, 2026 watch for it during fine-tuning and also found that harmless fine-tunes drift.
• Evolution as math for learning. Frank, 2025 argues that the Price equation, the root of the Robertson-Price identity, describes learning algorithms and natural selection with one law. A LessWrong post, "Price's equation for neural networks", applied it to network training.

What this page adds is narrow: the breeder's toolkit, applied literally, with harmless twins and a preregistered replication on a model small enough to run at home. The answers were mostly no, and they're written down with the tests that produced them.

How did I build it?

I ran this with an AI agent driving my laptop's RTX 3060 (6 GB) for the first round, and a Colab A100 for the preregistered round. It wrote the trait panel, ran the populations and the fine-tunes, and checked its own work. Open answers were graded by separate Claude Haiku instances that never saw which model wrote what. Several early results were wrong in ways that looked like findings: a memory overflow that made the GPU crawl, a process that died silently, the "I can't" wording axis, the generic drift that looked like misalignment until the good twins showed up, and a scaling bug that briefly pushed one result to the 99.5th percentile. Each correction made the conclusions smaller and more trustworthy. The preregistration made them smaller again.

The setup, the numbers and the limitations are in Lab notes.