Most healthcare organizations evaluate AI medical coding solutions on marketing claims rather than reproducible evidence. This piece breaks down what actually matters: how accuracy is defined and measured, what a statistically valid validation sample looks like, why documentation traceability became a compliance requirement in 2026, how specialty mix affects real-world performance, and how to run the ROI math once denial prevention is factored in. Use this as a checklist before signing with any vendor.