196 checks run before any MDRN release and the page
listing them tells you what we refuse to do. This is the other half: the things we did do, and had to undo.
Every entry is a real defect from the history of this product.
There is no entry here about a launch, a feature or a good month, and there never will be, because a log that is
allowed to contain wins is a newsletter by the third entry and nobody believes the fourth.
Three parts, never four
The format, printed so you can hold the page to it.
What we found
A defect in the product, named plainly, including the ones nobody would have noticed.
What we changed
The fix, specific enough that you can go and check it.
What it cost us
What the fix cost. Money, reach, or a sale we will not now make.
The third part is not decoration. An admission with no cost attached
is a humblebrag, and readers can tell.
Forty four defects, all of it having passed the suite. A plan built on the 29th, 30th or 31st was losing half its documents, because one clock decided a month had turned by looking for a day February does not have while the other clock delivered on the last day instead. Anybody who had ever run a scan was being sent the same month back every four months. Anybody whose safety answers asked for intensity to come up gradually got the same month back every five. A member who finished a block on a Tuesday had their plan jump to the next block for the rest of that Tuesday. The monthly email could tell somebody they had finished a block they had not. And a repeated block told everybody that fewer than two thirds of its sessions were logged, including the people nobody had counted.
What we changed
Every one of them, the same day, each with the check that would have caught it. Several patterns ran underneath: a number sanitiser that clamps into its range cannot carry a zero meaning "nothing here", because the clamp turns it into one. A test that only ever ran the member with no safety answers and no scan. And six of our own checks turned out to be weaker than their names, two of which passed with their defect put back, including the one guarding the most consequential decision in the product: it searched our source code for a phrase rather than running the thing.
What it cost us
A day of work reviewed is a day of work half rewritten, and a public admission that passing our own suite is not the same as being right, because some of the suite was not testing what it said. The members this found defects for are the ones with a safety answer, a scan, or a start date at the end of a month, which is to say the ones who most needed us to get it right.
The plan engine built your training from your answers and never once looked at which month it was. The habit chain did advance, one a month, so the claim was half true, and the half that was false was the expensive one: a member paying in month seven opened a Movement section identical to month one. Our own revenue model described the paid product as "the next month of one plan".
What we changed
A plan now resolves to a named twelve week block, Foundation 12 then Load 12 then Tempo 12, and each month inside it moves one dial by an amount you can check against last month’s sheet. The kitchen basket rotates inside fixed groups and the head gets a named practice. Month one came out of the change unchanged, so nothing anybody already holds moved under them.
What it cost us
Checks that now have to keep passing, including one that builds fourteen months of plans for thirty six different people in eight different situations and fails the release if any two of those months come out the same. It also means we can never ship a quiet month.
The offer page, the app and the month two preview all said a new plan document arrives at the start of each month. The job that sends it booked the next one thirty days after the last one, and thirty days is not a month, so it drifted five days a year against the clock that decides which month you are entitled to. Two clocks, and one of them was printed on the page that asks for money.
What we changed
One clock. Your month turns on the day of the month you started, which is the date entitlement already counts from, and the copy names that day exactly so you can hold us to it.
What it cost us
A vaguer promise. "On the first of every month" is a better sentence than "on the fourteenth, because that is when you started", and it was not true.
While fixing the cadence above, the deploy failed. A check written to stop the offer page drifting back into selling an abstraction demanded the page contain the literal words "first of every month". The suite was holding the false claim in place and would have blocked the commit that corrected it.
What we changed
That check now requires a deliverable and a cadence to be named, without picking which cadence, because a test cannot know what the scheduler does. A separate check compares the promise against the queue itself.
What it cost us
A slightly weaker test, and an uncomfortable fact on a public page: a suite of constraints can lock a defect in as easily as it can keep one out, and ours did.
Our supplement supplier does not ship to Alaska or Hawaii at all. Checkout did, happily, because the restriction logic only knew about one state and one law. Somebody there could have been charged for an order that could never have been fulfilled.
What we changed
State restrictions became a table instead of a special case, so a supplement is refused where it cannot be delivered and the reason is named on the page rather than discovered after payment.
What it cost us
Every order from two states, and the work of saying no at the checkout rather than after the money had moved.
Our own asset notes described one image as an empty path, and it is a man walking. It was about to be printed across the foot of a page in every plan document. A written description of a photograph is not a photograph, and we had been reading the note instead of the picture.
What we changed
The image was withheld, with the reason written next to it so it survives a rebuild, and the asset note was marked stale rather than quietly corrected.
What it cost us
One of four band images, and the admission that the only way to check a picture is to look at it.
A check that refuses nearly empty pages in the plan document started failing on pages that were full. We had taught it to notice photographs by recording the two edges of each one. Two points with nothing between them is exactly the shape of a hole, so a solid band of picture read as a run of white paper.
What we changed
The band is sampled all the way down instead of at its edges, because it is ink all the way down.
What it cost us
Nothing but the time, and a reminder that a measurement which agrees with you is the one worth checking twice.
A guide said that loading phases are mostly a way to get through a tub faster. That is a fair reading of the industry, and the label on the tub we sell describes a loading phase, so the sentence called our own product cynical in a page that was supposed to be the honest one.
What we changed
The guide now says plainly that loading is optional, that some labels including ours describe one, and that an MDRN plan does not use it. One scoop a day is what the plan asks for and the plan says so.
What it cost us
A good line, and the slower job of explaining a difference instead of scoring a point.
