Project Blue Fairy

This is the record of a year spent trying to build a mind out of small parts, and of what happened when we checked the results properly.The parts are what we called organs: a cheap language model wrapped around one narrow job. One organ checks an answer against a rule a machine can verify and sends it back for repair when it fails. Another keeps what a session learned on disk so the next session can read it. Wire enough of them together, the idea went, and models anyone can rent for pennies might do things none of them can do alone.For about a day we thought we had proved it. A wrapped model scored 11.8 points higher than the bare model on a set of 144 word puzzles. Then we did the thing this book is really about: we tried to kill the result before anyone else could. It died. The chapters that follow show how, why the number was really zero, and what a benchmark has to look like before a number like that can be trusted. Two smaller results survived that treatment, and the book is exact about how small they are.The second half is about the organs themselves, how one gets wired into a live system, and what it was like to work every day alongside something we had made. That part is harder to summarize, so I did not try.Practical notes. Every number in the book comes with the test that would have falsified it, written down before we ran it. An appendix gives the command that regenerates each one. The evaluation kit is public under the MIT license and runs on a laptop at no cost. If you build with language models, run evaluations, or want to see what honest negative results look like at book length, this is for you. If you want a book that says the wrapper works, it is not.