As enterprise AI moves from offering suggestions to participating in business processes, answering every request stops being an obvious goal. Missing evidence, unfamiliar tasks and costly mistakes require explicit exits from automation. The difficult part is establishing which requests may proceed, rather than merely asking a model to admit uncertainty. This is a durable decision problem; the early research discussed here is not presented as a recent breakthrough.
Buy a defensible automation scope, not just aggregate accuracy
A ticket classifier's accuracy across a complete dataset does not establish how many tickets it can safely route automatically. Mistakes may cluster in difficult categories or even among high-scoring predictions. The service being procured needs boundaries: which requests finish automatically, which need more information and which require a qualified person.
Selective classification describes the trade-off through coverage—the proportion of predictions accepted—and selective risk, the loss within that accepted set. A 2017 study developed risk-control methods under particular sampling assumptions and evaluated image classification.[1] Its guarantees and experiments cannot be transferred directly to enterprise question answering. The useful starting point here is how it frames evaluation.
Keep three denominators: eligible requests entering the service, automatically accepted requests and requests ultimately completed on time. Record out-of-scope reasons separately. Timeouts and system errors must not masquerade as deliberate abstention or disappear from service reporting. This reveals whether apparent accuracy gains reflect better selection or merely a much smaller, easier workload.
Deferred work can use the methods described in human-review capacity and effective error correction. This article addresses selection into automation; whether subsequent review is accurate and timely remains a separate acceptance condition.
A score can rank cases without being a correctness probability
A reported confidence of 0.95 might be a class probability, retrieval similarity, another evaluator's score or simply generated text. Identify the event being scored: a class being correct, a complete response being acceptable or a passage being relevant. Local similarity cannot stand in for the probability that an entire business result is correct.
The scikit-learn calibration documentation evaluates probabilities against observed frequencies among similarly scored cases and separates calibrator-fitting data from model-training data.[2] The scored event must also match the acceptance label. Predicting category A and successfully routing to any acceptable team are different outcomes.
Evaluate ranking and calibration separately. A score may rank easier cases first while overstating all probabilities. Another may match average frequencies yet fail to identify a safer subset. Calibration can improve the former's probability interpretation; changing numerical labels alone need not improve the latter's automation boundary.
For open-ended generation, do not label a model's self-reported score as a probability without evidence. Evidence completeness, deterministic checks, task categories and independent evaluation signals can be candidate selection features. Test their ability to exclude mistakes against human reference judgments. Missing required fields and lack of authorization should block execution regardless of score.
Select thresholds using independent evidence and consequences
Define an evaluable task and an acceptable outcome first. Several teams may be valid destinations for a support ticket, whereas equipment-operating advice may require particular qualifications in its wording. Define losses for different consequences, at least separating routine rework from critical errors. Many minor successes must not offset a few unacceptable actions.
Give data distinct roles in model training, selection-rule development and final acceptance. Strictly separated cross-validation can help during development when data is scarce, but the final acceptance set must not select the threshold. Split near-duplicate tickets from the same customer as a group and use temporal evaluation where appropriate, so paraphrases of one event do not cross the boundary.
On development data, scan candidate thresholds and record accepted counts, error counts, critical errors, abstention reasons and downstream workload. Freeze a point that meets the business conditions, then measure it again on independent data with sample sizes and uncertainty. Repeatedly tuning against the acceptance set until it passes removes its independence.
Zero observed errors in a small sample does not establish zero risk. Keep unfamiliar categories manual or collect evidence when samples are sparse; an overall confidence interval is not an interval for every small category. Any statistical guarantee needs stated sampling, independence and distribution conditions and a specified bound, rather than borrowed language from a paper.
A constructed example: better accepted accuracy can still produce a worse service
The following figures are hypothetical, not IDENIFE project results or measurements. On the same 1,000 eligible requests, policy A accepts 800 automatically and makes 16 errors. Policy B accepts 500 and makes five errors. A has 80% coverage and 2% accepted-set error; B has 50% coverage and 1% error. The remaining 200 and 500 requests need another path.
Assume a routine automated error costs CNY 100 to resolve and each deferred request costs CNY 4 to handle, temporarily ignoring other costs. A costs 16 × 100 + 200 × 4 = CNY 2,400; B costs 5 × 100 + 500 × 4 = CNY 2,500. A lower accepted error rate does not automatically minimize total cost.
If routine error loss rises to CNY 500, A costs CNY 8,800 and B CNY 4,500 under otherwise identical assumptions. This is sensitivity analysis. Real estimates must include review errors, waiting, complaints, rework and maintenance. If critical mistakes violate a hard constraint, exclude the unacceptable policy before comparing cost instead of averaging all consequences into one price.
Suppose the team can handle at most 300 deferred requests per day and every request must finish that day. B's 500 deferrals exceed this simplified capacity. Narrowing rollout or adding qualified capacity may be appropriate; silently lowering the threshold is not. Compare on-time acceptable completions across automated and human paths together.
Choose among learned rejection, rules and clarification
SelectiveNet, published in 2019, trains prediction and rejection jointly and improved the risk–coverage trade-off in its classification and regression experiments.[3] Selection can therefore be a model-design objective. That does not establish a need to retrain every enterprise model or guarantee effectiveness under a new distribution.
If errors originate from missing order IDs, clarification may be simpler than a sophisticated rejector. Missing permission requires stopping or an authorization process. An undocumented new product calls for an explicit coverage limitation. Compare learned selection only where observable signals have a stable relationship to mistakes and reliable supervision is available.
Rules offer understandable reasons and lower initial implementation and audit costs, at the expense of maintaining exceptions. A learned selector may express more complex boundaries but adds labeling, training, drift monitoring and regression work. Establish a rules baseline and compare alternatives at the same coverage on the same independent cases, so reducing scope is not mistaken for algorithmic progress.
Abstention should produce a structured reason, missing evidence, available next step and current task state. Track clarification, human referral, out-of-scope requests and temporary faults separately. Rephrasing and retrying until a high-scoring response happens to appear changes the selection procedure that was accepted.
Avoid learning only from requests that were accepted
If only accepted requests receive labels, the team cannot tell how many rejected cases could have succeeded. Within authorized data use, sample automatic acceptance, referral and out-of-scope paths separately, recording selection probabilities and label delay. When rejected requests are oversampled, restore actual traffic weights in aggregate estimates.
Inspect coverage and error risk by relevant business segments such as product, language, input quality and new versus existing customers. A stable overall curve can hide near-total rejection of a small segment or concentrated critical mistakes. Report only the observed part while labels mature, track unresolved outcomes and never equate absence of complaints with correctness.
Version the model, prompt, evidence rules, scorer and threshold together. Reassess selection after material component changes. Critical errors or changed data conditions should trigger a preapproved reduction in automation scope. The deliverable is a delegation boundary that evidence can revise, not a confidence number that remains valid forever.