A few weeks ago, someone I know was added to their company’s recruiting platform to help screen candidates. They expected the usual — keyword matching, maybe some decent filtering. What they found was worse than inefficiency. The AI was actively misrepresenting people.
One candidate had a resume spanning five years of progressively senior IT roles: help desk technician, team lead, junior sysadmin, sysadmin. At the very bottom, a single line from 2015: part-time fruit picking during a gap year. The system surfaced that line as the candidate’s professional headline. “Fruit picker.”
The hiring manager saw the summary, laughed, and denied the application without opening the full profile.
This is not a story about recruiting software being buggy. It is a story about something far more common — and far more operationally dangerous — across enterprise systems.
—
The Extraction Problem Nobody Audits
ERP platforms summarize financial data. CRM systems surface lead scores and pipeline highlights. HR tech summarizes candidates. Reporting dashboards pull top-line figures from complex datasets. In every case, the system is making an editorial decision about what the decision-maker sees first.
When that editorial logic is naive — when it pulls the oldest record, the outlier, or the least contextually weighted data point — the downstream effect is not just inaccurate. It is actively misleading.
The recruiting example is stark because the human cost is visible: a qualified person excluded by a bad summary. But the same pattern plays out daily inside operations:
– A CRM pipeline view highlighting a stalled deal from six months ago instead of the three active negotiations that closed this week.
– An ERP exception report flagging a one-time inventory adjustment from last quarter as a recurring variance, triggering unnecessary rework.
– An automated approval workflow rejecting a purchase order because a vendor record contains outdated tax information the system treats as a hard block.
In each case, the system is technically correct about the data point. It is contextually wrong about what matters.
—
Operational Implications
Three patterns make this particularly difficult to catch:
First, the output looks authoritative. A system-generated summary carries implicit credibility. People trust it more than they should, precisely because it came from a system rather than a person.
Second, the failure is invisible unless someone manually verifies. The hiring manager never opened the candidate profile. The ops director never drilled into the ERP exception. The pipeline review moved on after seeing the summary. No error is logged because — technically — there is no error.
Third, the people closest to the problem rarely see it. The candidate does not know they were rejected over a fruit-picking line. The sales lead does not know their deal was deprioritized by a bad pipeline summary. The system quietly filters reality, and the filtered-out data rarely has an advocate.
—
What This Means for Systems Planning
If your organization uses any form of automated summarization — in recruiting, reporting, CRM, ERP, or otherwise — it is worth asking a few practical questions:
– What logic determines which data points get surfaced as summaries?
– How does the system weight recency versus relevance versus anomaly detection?
– Who spot-checks the output against the underlying data?
– Are decision-makers trained to treat system summaries as signals rather than conclusions?
These are not technology questions. They are process design and governance questions. And in many organizations, nobody owns them.
The recruiting example gets attention because it is relatable. But the quiet version of this problem — the ERP summary that misdirects an operations meeting, the CRM view that hides a revenue signal — carries far larger commercial consequences. And it happens far more often than most leadership teams realize.
