“Every case passed” can accurately describe the inspected sample without guaranteeing the next ten thousand business operations. Moving from demonstrations to continuous delivery requires evidence for reliability within a stated scope, rather than a perfect percentage without sampling context. This is an evergreen methodological discussion, not a claim about recent news.
Define a failure before interpreting a pass
Calibrating false approvals by LLM judges addresses who checks an answer. This article asks a further question: even with trustworthy checking, how much evidence, gathered in what way, supports release? Evaluator accuracy and the strength of an acceptance sample are separate requirements.
Define failure at the business-outcome level before testing. In quotation extraction, for example, an incorrect amount, currency or supplier identity may make the whole task fail. Record malformed output, timeouts and manual rescue under agreed rules. Counting dozens of easy fields as successes must not conceal one consequential field error inside an average.
Specify whether the evaluated object is the raw model, the complete system or the delivery after human review. If the commitment concerns automatically approved results, examine failures in that queue and report automation coverage separately. Omitting all rejected or escalated requests removes the context needed to interpret quality and service availability together.
One hundred passes may not support a one-percent commitment
Assume independent trials with a common failure probability, sampling from the agreed target population and a sample size fixed in advance. If failure probability is p, the probability of observing no failures in n trials is (1-p)^n. Equating this with 0.05 gives the one-sided 95% exact binomial upper bound after zero failures: p_upper = 1 - 0.05^(1/n). NIST describes exact one-sided binomial limits and cautions against ordinary normal approximations with few failures or small samples. Geyer's course materials give the zero-event boundary form.[1][2]
The following are constructed calculations, not model measurements. With no failures, 100 trials give an upper bound of about 2.95%; 300 give about 0.994%; and 3,000 give about 0.0998%. Thus a hundred passes do not support a failure rate no greater than 1% under this confidence convention. An observed rate of zero and the size of a rate the evidence can exclude are different quantities.
Planning backward, requiring the zero-failure one-sided 95% upper bound to be at most p0 gives n ≥ ln(0.05)/ln(1-p0), rounded upward. A 1% target needs at least 299 trials; a 0.1% target needs at least 2,995. These plans apply to zero observed failures. If failures occur, recompute using their actual count rather than deleting failed tasks and reusing the formula.
Confidence describes the coverage properties of a procedure under repeated sampling. It is not a 95% probability statement about this particular fixed parameter, nor a guarantee that the next batch is safe. Changes to model versions, input composition or business rules can invalidate the scope of the conclusion. For catastrophic consequences, a statistical bound does not replace authorization controls, human confirmation or reversible execution.
A thousand calls need not be a thousand independent business samples
Rephrasing a single contract question ten ways and running each ten times can test wording sensitivity and generation variability. It is not automatically equivalent to testing a hundred different customer contracts. Failures may share a template, scanned image or faulty business rule. Common causes create dependence, so treating every call as an independent binomial trial can overstate the evidence.
The scikit-learn cross-validation guidance distinguishes independent observations from grouped dependent data and provides group-separated training and testing methods.[3] For enterprise AI, source document, customer, template or business batch may be useful grouping variables. Choose the unit by how errors arise, rather than mechanically selecting a convenient column.
Group separation chiefly prevents development and acceptance from sharing closely related material; it does not make observations within a group independent. Defining a group as failed when any of its tasks fails estimates a group failure rate, not a request failure rate. Request-level inference needs analysis suited to the actual sampling and dependence structure, retaining group sizes and sampling weights. Replacing call count with group count while keeping the old interpretation is not enough.
Representativeness matters too. Tests containing only short documents, clean scans and common questions cannot support commitments about complex tables or unfamiliar templates. Challenge sets reveal weaknesses; random business samples estimate performance for a specified population. Report them separately. Uncovered business slices remain unverified even when the overall observed failure count is zero.
Fix the acceptance plan before tuning
Record a short protocol before running acceptance: population, failure definition, sampling unit, version combination, sample size, primary decision rule, exception handling and stopping conditions. Developers can continue diagnosing errors with development data, but the final acceptance set needs separation. Feeding its examples back into prompts, retrieval rules or manual selection changes the nature of the test.
If a team changes the model after every inspection and reports only the first all-pass round, an ordinary fixed-sample interval does not account for the entire selection process. Retain all attempts and obtain a fresh independent acceptance set after freezing the candidate. If decisions must be made while observations accumulate, design an appropriate sequential procedure in advance rather than repeatedly recalculating an ordinary interval until it looks satisfactory.
Nor do separate 95% checks for many business slices automatically provide the same simultaneous assurance for all slices. Identify the primary metric and critical slices in advance, and distinguish individual from joint conclusions. Controlling multiple decision errors requires a suitable design. More tests are not intrinsically better; each should have a clear consequence for a release decision.
Use evidence gaps to allocate the review budget
Consider a constructed budget: six minutes of human verification per independent task means about 29.9 hours for 299 tasks and 299.5 hours for 2,995, before preparing examples or adjudicating disagreements. Tightening the target by an order of magnitude can substantially increase evidence costs. Buyers and suppliers should discuss this before pricing, rather than introduce an unsupported reliability commitment at final acceptance.
Where a low-volume workflow cannot supply enough representative data, reasonable choices include narrowing automated scope, retaining human confirmation, running in shadow mode or deferring the commitment. Generating many similar questions may not close the coverage gap, and model self-grading does not automatically replace independent checking. Prioritize consequential boundaries that remain unverified.
After release, record input scope, automation coverage, failures and human corrections. Maintain sampling and reassess relevant changes. New evidence may justify expansion or require contraction; one zero-failure exercise months earlier is not a permanent certificate. New templates or tool permissions should trigger a review of whether the original acceptance scope still applies.
Deliver actual counts, sampling records, interval methods, assumptions and uncovered cases together. This article proposes three decisions: sufficient evidence with adequate controls supports limited release; good results with thin evidence support a controlled pilot; unresolved critical failures or sampling defects require repair first. Zero errors deserve recording, but a deployment commitment requires more information than those two words.