Default Zero-Shot LLM APIs Over-Flag Indian Financial SMS
Paper at the GroundLM 2026 workshop, co-located with EMNLP 2026, on how often language-model APIs wrongly flag legitimate Indian financial text messages as fraud.
Published in the proceedings of the first workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026), co-located with EMNLP 2026.
Generic spam labels do not define a financial-fraud blocking policy. The paper presents a benchmark of 1,393 Indian SMS messages in Hindi, Hinglish, and English-India, keeping source labels, fraud adjudication, and allow, review, or block actions separate. Its primary measure is the false-positive rate on 1,294 policy-negative messages. Under a common minimal prompt, six language-model APIs exceed illustrative 1% false-positive gates on bank and UPI templates. The labels, prompts, parsed decisions, analysis code, and checkpoints are released, without a claim about production use or unseen institutions.
