Scaling Laws has been downloaded about seven hundred times. Thirty of those people wrote to me. Six of them are still writing.
The last number is the only one that matters, and it is the one nobody optimises for. A download is a statistic. A report is a favour. Somebody who reports twice has decided the first one was worth their time, and that decision is made entirely by what you did with it.
The request almost everybody makes, and why it fails
The default is some version of let me know if you find any bugs. It sounds welcoming and it produces almost nothing useful, for a reason that took me three builds to see.
Asked for bugs, a player reports whatever annoyed them. Roughly half of what I used to receive described something that was not built yet. The map had regions you could fly over and not lease. People wrote in to say the leasing was broken. It was not broken. It did not exist.
Every one of those cost the sender the time to write it and me the time to read it, and neither of us learned anything. The sender also learned that writing in produces a reply explaining they misunderstood, which is an excellent way to make sure they never write again.
Remove the noise in the build, not in the inbox
The fix was not a better reply. It was a label. Unfinished parts of the game now carry a yellow NOT IN THIS BUILD badge, in the game, where the player is standing when they get confused.
The instinct this fights
Hiding unfinished work feels safer. It looks more professional and it protects you from being judged on something incomplete.
It also guarantees that the gap gets reported as a bug, because from outside there is no difference between a feature that is missing and a feature that is broken. You are not hiding the hole. You are hiding the sign that says hole.
Roughly half the incoming reports stopped. What still arrives is about things that are genuinely wrong, and I would rather ship a visible gap than a tidy lie. A reader on a social post put it better than I did: labelling it is what turns strangers into testers instead of critics.
The question that replaced “did you like it”
Everything I send now ends with the same request, and it is deliberately not about opinion:
Three things make this work. It asks about a moment rather than a judgement, so there is a specific answer. It assumes they stopped, which removes the awkwardness of admitting it. And it is the one thing I cannot observe from inside the project: I know what every screen does, so I cannot see which one loses people.
Compare it with did you enjoy it, which has two answers, both useless. Yes is politeness. No is unactionable.
Reply with what it turned out to be
This is the part that produces the six, and it costs about ten minutes per report.
When somebody writes in, I find the cause and tell them what it actually was. Not thanks, noted. The mechanism. One tester reported that a bar on screen was wrong and was mistaken about which bar; while checking whether he was right I found a real fault two methods away. I told him that, in those words. He was wrong about the symptom and right that something smelled off, which is harder and more useful than being right.
He now sends reports at a rate I did not ask for. On the Saturday the last build went out he said at half past five that he would look at it that evening or the next day. By twenty past eight he had sent three.
What the reply is actually buying
Not goodwill. Calibration. A tester who learns what their report turned into gets better at writing the next one, because they now know which details mattered. The six people still writing all send reports with the screen, the action and the expectation in them. None of them did at first.
Do not read the envelope
One report arrived from an address that several people read as a joke before reading the message. I nearly did too.
It said the music would not mute. It was right, and it was worse than described: the opening video bypassed both volume sliders, so turning the sound off did nothing to the one thing that plays before a player has touched any other control. The first impression of the whole game ignored the only control a person reaches for.
That was the most useful bug anybody sent me that week, and the thing I changed was not the audio code. It was how I read the inbox.
What people find that tests do not
The project has 1,619 automated tests and 87 more that load a scene and photograph it. None of them found any of the following.
| What a person found | Why no test could |
|---|---|
| Twelve finished systems no player could reach | A test calls the code directly. A player has to find a button. Every one of those twelve was correct, covered and wired to nothing. |
| Two products showing identical user counts, to the decimal | Found by shipping a second product and looking at both. No fixture had ever compared two of them side by side. |
| “Things vanish while you are looking at them” | A card closed on every day rollover, about every second and a half. Each rebuild was individually correct. The complaint only exists if you are reading. |
| A cabinet slot that was never clickable, in any build | The code decided whether a slot takes a click before anything assigned it a handler. The elements existed, so every structural test passed. |
| A tutorial strip covering a third of every screen | The reserved space was a constant nobody had measured. I played past it for days and read it as background. |
The pattern is the same every time. Tests verify that a thing works when it is called. People discover whether anything leads to it, and whether what it does makes sense to somebody who was not there when it was written.
Pay them in the only currency you have
I have never paid a tester. What they get is their name in the credits of the shipped game, a reply that treats their report as work, and the game for free once it goes on Steam.
One of them plays each build with a pen and a piece of paper and keeps finding classes of bug that a thousand automated tests do not. Another report described a whole branch of research that was missing from the screen, and that one message opened the audit that eventually found eighteen unreachable systems. Neither of them owes me anything, and that is exactly why their time is worth protecting.
If you want to try this
Four things, in the order that cost me the least to learn:
- Label what is not built, in the build, where the confusion happens. This removes the most reports for the least effort.
- Ask where they stopped, not whether they liked it.
- Reply with the cause, named, including when they were wrong about the symptom.
- Read the message, not the sender.
None of this scales and none of it needs to. Six people is enough when each of them sees something you structurally cannot.
Questions people ask about this
Why do most bug reports from players say so little?
Because the request is open ended. Asked to report bugs, a player reports the thing that annoyed them, which is often a feature that is not built yet. Asking where they stopped and what they were trying to do produces a report about the moment the game lost them, which is the thing you cannot observe from inside.
How do you stop getting reports about unfinished features?
Label them in the build. A yellow NOT IN THIS BUILD badge on unfinished parts of the map removed roughly half the incoming reports here, because those reports had been descriptions of things that were never built.
What makes a tester come back a second time?
Being told what their report turned out to be. A reply that names the cause and admits when they were right about the smell but wrong about the symptom costs ten minutes. It is the difference between one report and a person who keeps looking.
Should you judge a report by who sent it?
No. One report here arrived from an address several people read as a joke. It was the most useful bug of that week: the opening video bypassed both volume sliders.
Every report in this article is real and came in about Scaling Laws, an AI company tycoon released as a free public build with the source open.
The game page and the source · The eighteen unreachable systems that audit found