Why dark data is an AI agent problem
Many organizations begin an AI agent project by choosing a model, designing prompts, or connecting a chat interface. Those steps matter, but they do not solve the most basic question: Can the agent reach the information it needs, understand what that information means, and use it within the organization’s rules?
Dark data is a useful term for information that exists but is difficult to find, access, interpret, or use. It can include unstructured material such as PDFs, images, videos, and social media posts. It can also include structured information trapped in operational systems, including support logs, server logs, Internet of Things feeds, and older HR records.
This is especially important for AI agents. A conventional assistant may answer a question using the documents available to it. An agent is expected to take steps, coordinate systems, and sometimes make recommendations that affect real work. If it cannot see a relevant contract, support history, inventory record, or policy—or if the record lacks business context—the agent may produce an incomplete answer or take an unsuitable action.
A report published by MIT Technology Review Insights in partnership with Google Cloud, and based on a survey of 300 senior executives, puts the issue into perspective. It says AI systems in surveyed organizations had access to an average of 45% of company data. The figure was 30% or less for organizations categorized as data laggards, while a group described as data leaders enabled access to more than 70% of their data.
The lesson is not that every company must expose every data source to every agent. The lesson is that agent quality depends on intentional, governed access to the right data.
What the latest enterprise AI agent research shows
The report, Scaling AI Agents with Trustworthy Data, connects data readiness with confidence in agent decisions. Only about half of surveyed executives said they trusted the accuracy and relevance of their agents’ decisions. Among the data leaders identified in the report, all reported that their agents made mostly or consistently accurate and relevant decisions. Among data laggards, only 22% trusted agent decisions.
The findings also point to a delivery problem. Organizations with older data environments reported that legacy systems could prevent them from scaling agents or stop agents from making decisions at speed. Across all respondents:
- 55% said their current data platform was the primary bottleneck to enterprise-wide agent scaling.
- 54% had paused or delayed an agent deployment to fix foundational data issues, including silos, missing governance, or missing business context.
- 50% said legacy systems were significantly hurting the return on investment from agent projects.
These figures describe a survey, not a universal measurement for every organization. They do, however, give leaders a practical diagnostic: when an AI agent appears unreliable, the cause may be the information environment rather than the underlying model alone.
The survey also shows that adoption is continuing. It reports that 83% of organizations use AI agents to some degree, with 10% reporting widespread use and 73% limited use. Current deployment is concentrated in customer service, IT systems management, and IT security. Human resources was identified as a fast-growing planned use case, especially for onboarding and benefits management.
Where dark data hides in a business
A data inventory should cover more than the warehouse and the CRM. Agents often need information that is created by everyday operations but never prepared for reuse.
Common sources include:
- Customer interactions: support tickets, call transcripts, chat histories, email threads, and escalation notes.
- Documents: proposals, contracts, invoices, policies, product manuals, and compliance records stored as PDFs or scans.
- Visual and media files: product images, inspection photos, training videos, and recorded demonstrations.
- Operational logs: application logs, server events, device telemetry, and workflow histories.
- Human resources records: older employee documents, onboarding material, benefits information, and internal knowledge.
- Marketing information: creative versions, campaign briefs, audience research, landing-page changes, and feedback from sales teams.
Not all of this material belongs in an agent’s context. Some records may have legal, privacy, or security restrictions. Others may be duplicated, outdated, or missing an owner. The first step is therefore not “connect everything.” It is to classify information by business value, sensitivity, freshness, authority, and permitted use.
For a customer-service agent, a product policy may be authoritative while an old internal presentation is not. For a marketing agent, a campaign brief may provide intent, but a current conversion definition and budget system provide control. A reliable agent needs these relationships represented clearly.
Why access alone is not enough
Connecting a data source does not automatically make it useful. An agent may retrieve a sentence from a document but still misunderstand its scope, date, owner, or exceptions. A clean table can also be misleading if the business definition of a field is unclear.
Trustworthy enterprise data should carry context such as:
- Business definition: what a field, document, or event actually represents.
- Ownership: who is responsible for accuracy and approvals.
- Lineage: where the information came from and how it was changed.
- Freshness: when it was last updated and how often it should be reviewed.
- Quality signals: known gaps, validation results, and confidence indicators.
- Security rules: who may view, retrieve, or act on the information.
- Usage constraints: actions the agent may suggest, prepare, or execute.
- Evidence: the source records that support an answer or recommendation.
The report emphasizes that metadata alone is not sufficient. Data needs business definitions, ownership, lineage, quality signals, evidence, security, and usage constraints so that people and AI applications can interpret it more reliably.
This principle is critical for dark data and AI agents. A retrieval system can find content, but governance and context determine whether the content should influence a decision.
A practical framework for preparing data for AI agents
1. Start with one decision, not one model
Define the business decision or workflow the agent will support. Examples include answering a customer’s product-policy question, routing a support issue, preparing a campaign report, or summarizing an internal request. Write down what information the agent needs and what it must never do.
This keeps data preparation focused. It also makes it easier to evaluate whether a new source improves the workflow.
2. Map the data path
For each required source, document where data is stored, how it is updated, who owns it, and how an agent will access it. Include file repositories, business applications, APIs, databases, and manual handoffs. Pay special attention to information that is technically available but operationally difficult to retrieve.
3. Classify and prioritize dark data
Rank sources by relevance, sensitivity, quality, and expected value. Start with a small set of high-value records rather than importing every historical file. Label documents that are obsolete, duplicated, restricted, or missing an owner.
4. Add business context
Create definitions for important fields and documents. Identify the authoritative source for each decision. Link related records where appropriate, such as a support case to a customer, policy, product version, and resolution.
5. Build permission-aware retrieval
An agent should not bypass the organization’s access controls. Retrieval should respect the user, role, data classification, and purpose of the request. Sensitive information should be separated or redacted where necessary.
6. Require evidence and escalation
For important answers, show the records or references that support the result. Define when the agent must ask for clarification, hand a task to a person, or stop rather than guess.
7. Test with real workflows
Use representative cases, including incomplete, conflicting, outdated, and restricted data. Measure not just answer quality, but also whether the agent selected the right source, followed permissions, explained uncertainty, and escalated correctly.
How to measure whether data readiness is improving
Data readiness should be tracked as an operational capability, not as a one-time cleanup project. Useful measures include:
- Percentage of priority sources with a named owner.
- Percentage of records with freshness and access metadata.
- Retrieval success for approved test questions.
- Rate of answers supported by authoritative evidence.
- Number of permission or policy violations.
- Escalation rate for ambiguous or unsupported requests.
- Time required to update an agent’s trusted knowledge.
- Business outcomes connected to the workflow, such as resolution quality or reporting cycle time.
A higher retrieval rate is not automatically better. If an agent retrieves more documents but includes irrelevant or restricted material, the workflow may become less trustworthy. Pair coverage with quality, security, and business results.
Likewise, do not evaluate an AI agent only on whether it sounds confident. A useful agent can say that the available evidence is incomplete, identify what is missing, and request human review.
Common mistakes when deploying enterprise AI agents
The first mistake is building a broad agent before defining a narrow responsibility. A general-purpose system may have unclear permissions and no reliable evaluation boundary. Start with a workflow where the data, owner, and success criteria are visible.
The second mistake is treating a document repository as a knowledge system. Files need versioning, ownership, classification, and retrieval rules. An old PDF should not quietly outrank a current policy.
The third mistake is ignoring latency. An agent that can technically access a record but must wait too long may be unusable in a live process. Measure the speed of the complete path from request to retrieval, reasoning, approval, and action.
The fourth mistake is allowing an agent to act without a rollback or escalation path. For high-impact tasks, use approval gates and keep an audit trail. This is especially important when the agent can alter customer records, spend budget, send communications, or change operational settings.
Finally, do not assume that more data always produces better answers. Relevance, authority, context, and governance matter as much as volume.
FAQ: dark data and AI agents
What is dark data?
Dark data is information an organization holds but cannot easily find, access, interpret, or use. It can include unstructured material such as PDFs, images, and videos, as well as structured records locked in operational systems.
Why does dark data affect AI agents?
Agents need timely, relevant information to make recommendations or take actions. If important data is inaccessible, outdated, poorly defined, or missing permissions, the agent may produce incomplete or unreliable results.
Should a company connect all of its data to an AI agent?
No. Organizations should connect only approved, relevant sources and enforce permissions, retention, privacy, and usage rules. A focused and governed data set is safer than indiscriminate access.
How can a business improve enterprise AI agent reliability?
Start with a narrow workflow, identify authoritative sources, add ownership and business definitions, use permission-aware retrieval, require evidence, and test with realistic edge cases before expanding.
Is data cleanup a one-time project?
No. Data changes as products, policies, systems, and customer behavior change. Owners, freshness checks, evaluations, and monitoring should remain part of the agent’s operating process.
Zapplon helps businesses design practical AI agents, AI video workflows, and performance marketing systems around measurable goals and responsible automation. Contact Zapplon to discuss a focused use case and a clear implementation plan. Services start at $50.