When Do LLMs Actually Help? Evaluating LLMs as Data Quality Annotators

arXiv:2608.181582026-08-20

LLMs have been increasingly used to catch data quality issues automatically, but we know very little about how consistent these judgments actually are. This study tests an LLM on two e-commerce data quality tasks, entity matching and brand mislabeling, against rule based baselines and human verified ground truth, under both zero-shot and few-shot prompting. On entity matching while using the Abt Buy benchmark (2,194 labeled pairs), a simple rule ba

Authors · Praphulla Lal Shrestha