Testing normal software is mostly about certainty. Give it this input, expect exactly that output. Machine learning breaks that rule. The same input can give a different answer twice in a row, because the system learned patterns rather than following fixed instructions. If you test it like normal code, your tests flake and your quality signal turns to noise. Here is how to test it properly, explained without the jargon.
Imagine a music app that recommends songs. Ask it for ten tracks and you get one set. Ask again and you get a slightly different set, both perfectly good. A test that demands the exact same ten songs every time will pass today and fail tomorrow for no real reason. That is a false alarm, and false alarms are how teams stop trusting their tests.
The fix is a shift in mindset. You stop testing the exact answer and start testing the properties the answer must always have.
Even when the output varies, some things must always be true. Those are your tests.
None of those care which exact songs came back. They care that the output is well formed and relevant. That is a stable test on an unstable system, which is exactly what you want. People call this stochastic testing, which just means testing something whose output has randomness in it.
Some things are true most of the time but not every single run. For those, do not assert once. Run the check many times and assert the pass rate. If a quality bar should hold in at least nine runs out of ten, test exactly that. It turns a flaky yes or no into a stable measurement.
ML systems degrade quietly. A new model version can be a little worse for one group of users while looking fine on average. So you compare the shape of the results before and after a change, not just a single average number, because an average can hide a real problem. Catching that early is a big part of quality work on AI products.
When you judge quality with another model, for example using an AI to score whether an answer is good, you have to check the judge itself against human opinion now and then. Otherwise you are measuring with a ruler that might be bent. Trust, but verify the thing doing the verifying.
More and more products have machine learning inside them, so more and more QA and engineering roles ask about this directly. If you can explain, in plain words, why you test properties instead of exact outputs, why you measure pass rates for fuzzy checks, and how you watch for drift, you sound like someone who has actually held a probabilistic product to a quality bar. That is rare, and it stands out.
Wisptr builds answers from your own experience, so if you have done anything close to this, it helps you say it clearly and confidently. Its Practice mode can rehearse you on ML testing questions out loud before the real thing, so the ideas are ready when you need them.
Wisptr preps you before the call and has your back during it, with answers built from your own CV.
Try Wisptr free →