How to Check AI's Work Before You Trust It

The checkout passed its tests. A customer spending exactly $50 would still have been charged $55.

This is a deliberately broken teaching example, with errors planted in a small Python program. It illustrates a problem anyone using AI to write code, formulas or automations can run into: a passing check only tells you about the cases that were checked.

My practical rule is to agree on the expected result before asking AI to prove that its work is finished. Then check an ordinary case, the point where a rule changes, and a case where two rules interact.

Four orders make the gap visible

Our fictional shop has two rules. Shipping costs $5 and becomes free when the original item subtotal reaches $50. A coupon reduces the price of the items; it does not remove shipping eligibility the customer already earned. Other shops may choose differently. This is the promise we chose before testing.

The first two orders pass. The next two expose the errors in the same program.

The $50 order checks the exact threshold. The broken code required the subtotal to be greater than $50, so it charged shipping at precisely the amount that should qualify.

The $60 order checks the interaction. The coupon takes $12 off the items, leaving $48 to pay. Shipping remains free because the original $60 subtotal qualified. The broken program used the discounted amount to decide eligibility and added $5.

Correcting those rules makes the unchanged four-case suite pass. That establishes a repair for these cases; a whole shop would need more checks for taxes, refunds, rounding and other requirements.

Ask AI for results you can inspect

You can use the same approach without reading every line of code. Give AI your actual rules and ask:

Before you change the implementation, write a table of inputs and expected results from these requirements. Include an ordinary case, an exact boundary, and interacting rules. Explain the expected answers so I can review them. After we agree, run those checks and show the actual results. Keep failed and untested cases visible.

The review in the middle matters. If AI misunderstands the policy and writes both the program and the expected answers from that misunderstanding, everything may agree with the wrong rule. Work out the important examples yourself, or have the person who owns the policy confirm them.

For a spreadsheet, the boundary might be the day a discount expires. For an automation, it might be a duplicate event arriving twice. The useful question is specific: what should happen to this input under our agreed rule?

Automate the repeat checks

The project that prompted this experiment, review-layers, adds verification and review gates around coding work. In a local adapter, we exercised its pinned controller with real Python checks and simulated model replies. Failed checks stopped that path; passing checkout checks still required human review.

That tests part of the control flow. It does not establish the installed plugin's reliability, reviewer accuracy or savings over ordinary testing. The checkout lesson is useful with an ordinary test runner too: repeated execution becomes valuable after the checks express the right promise.

When AI next says “done,” ask it to show the ordinary result, the boundary and the interaction. Agree on the expected answers, run the checks, and keep those checks for the next change. You gain a specific reason to trust the result—and a clear record of what remains untested.