Every fund is trying to figure out how to best integrate AI across their stack. Even though every firm is approaching it a bit different, there are common patterns everyone is trying to solve: Everyone actively moving away from siloed data and trying to centralize all data to then give AI access to that centralized data.

Manual data wrangling is a bottleneck

Traditional investment research has run on the same inputs for decades: earnings reports, regulatory filings, vendor data, sell-side research, and market data feeds. Each is useful and no single one tells the whole story.

A lot of that data is scattered in different reports, spreadsheets, and websites. Analysts spend a lot of time just copy/pasting data from these sources and refreshing formulas.

Combining and aggregating all this data into a single model is time-consuming. AI can automate this process by analyzing the source structure, extracting the relevant data, and integrating it into a centralized storage. A key goal of many investment firms is a centralized database where various data types (public, vendor, market, alternative) are ingested, standardized, and made accessible to all analysts.

Foundation first

Before AI can do anything useful, the data needs to live somewhere consistent. Most firms work the opposite way: each analyst keeps their own spreadsheets, each team has its own folder, each source its own naming convention. The same series gets pulled three times by three different people.

A central data layer fixes this. Public filings, vendor feeds, market data, web pulls. All ingested once, normalized to a common schema, queryable from one place. A new analyst inherits the work, not the cleanup. The payoff isn't just efficiency. It's coverage. When the data lives in one place, an analyst can ask broader questions: which competitors moved on pricing this month, which sectors are hiring fastest, which regulatory filings overlap with the coverage list.

AI sits on top of this. Without a foundation, AI just automates the mess.

Data Sources

Most of the data that research teams act on falls into four categories. Each has a different update cadence and lead time to disclosure.

Public data

Earnings filings, regulatory disclosures, government statistics, central bank releases, company press rooms are important signal to look at.

The extraction of this data is hard though because it's all unstructured on websites or in PDFs. Filings come as PDFs with footnotes that span pages, tables that span columns, exhibits hidden behind hyperlinks. Government datasets ship in CSVs that change shape between releases. Press rooms restructure their URLs every quarter.

AI handles the messy parts: pulling structured tables out of PDFs, mapping section headers to a normalized schema, flagging when a filing's footnote changes a number two pages up. The output is a row in a database instead of a file in a folder.

Vendor data

Bought feeds from FactSet, Refinitiv, Bloomberg, S&P Capital IQ, ICE, niche specialists. Updated daily or intraday, already normalized, already paid for. The expensive but reliable backbone of most research processes.

The bottleneck here is integration and aggregation: each vendor uses its own identifiers (ticker, CUSIP, ISIN, internal IDs), its own schemas, its own update windows. Cross-referencing them is where analyst time disappears.

AI helps with entity resolution (matching the same company across vendors), schema mapping (translating one vendor's field names to another's), and reconciliation (flagging when two vendors disagree on the same data point). It also surfaces coverage gaps before they show up in a report.

Market data

Tick data, OHLCV bars, options chains, fixed-income yields, FX rates. Real-time or near-real-time, structured, high-volume. The cleanest data in the stack and the one analysts spend the least time wrangling.

Where AI helps is downstream and with quality assurance: anomaly detection (a missing print, a stale feed), corporate-action adjustments (splits, dividends, ticker changes), regime classification (volatility regimes, correlation breaks).

The four layers of the AI data stack

When LLMs hit the market in 2023, investment firms were scrambling to figure out what to do with them.

Three years later, the stack is settling. No firm builds it the same way, but the layers are consistent.

1. Data acquisition

Still where the real differentiation lives. Market data is table stakes. Around 98% of managers use alternative data now, so access alone isn't an edge and pretty commoditized.

What's changing is the tooling. AI lets firms acquire their own data instead of buying every signal from a vendor. At Kadoa we focus on self-service web data sourcing for this layer: an analyst points at a source, defines the schema, and gets structured rows on a schedule without engineering involvement.

2. Storage and compute

The tooling is settled. BigQuery, Databricks, Snowflake, DuckDB, Iceberg, Arrow. Pick what fits the team and move on.

The hard problem isn't the warehouse. It's entity resolution: getting every data point mapped to the same set of companies, tickers, and people so downstream models can join across sources. Most firms underestimate how much of the foundation work this represents.

3. AI interface

Two years ago this layer barely existed. Now it's where the most energy goes. General-purpose LLMs (Claude, ChatGPT, Gemini), MCPs (including Kadoa's), and finance-specific tools like AlphaSense, Hebbia, Rogo, and Unique are converging here.

The practitioner consensus: LLMs compress thinking. They don't generate alpha on their own. The differentiation sits in the data feeding them and the workflows wrapped around them.

4. Analytics and output

The last mile is still Excel, PowerPoint, and Word. That hasn't changed as much as the hype suggests. The realistic goal isn't to replace these tools but to keep the data flowing into them clean, current, and reproducible.

Compliance wraps everything

Most firms can access Claude or a similar model, but very few can let it reach the internet or integrate MCPs. The bottleneck isn't technical. It's the speed at which legal can process new tools.

This gets figured out the same way compliance was solved for phones and remote work, but for now it sets the ceiling on how fast any of these layers move in production. Platforms that ship with policy engines, approval workflows, audit logs, robots.txt enforcement, PII detection, and SOC 2 attestation compress what used to be weeks of legal review into days. Around 86% of fund managers have a written AI and data-sourcing policy, and regulatory scrutiny is intensifying: the EU AI Act now governs general-purpose AI deployment, and US regulators have increased review of how funds source and use alternative data. Kadoa's compliance documentation covers the specifics, including configurable rules for source blacklisting, PII detection, and robots.txt enforcement.

How to use AI for data ingestion

Most of the implementation work sits in the data acquisition layer. The pieces that determine whether a pipeline holds up at institutional scale are not the model choice. Four properties matter more:

• Point-in-time data, so backtests don't drift from current-state snapshots
• History depth, with five years as the common minimum for serious testing
• Entity and ticker symbology that matches the fund's existing models
• Source grounding per field, so compliance and engineering can trace any value back to its origin

Pipelines that ship without these fail in subtle ways: a backtest that looks great until you realize the historical data was overwritten with current values, a quant model that disagrees with a vendor feed because tickers don't match across sources, a research note that can't be traced back to its source when compliance asks.

The AI model work is roughly 30% of what gets a pipeline to institutional reliability. The other 70% is orchestration, validation, error handling, and human-in-the-loop tooling for the cases automation can't resolve. That ratio is why evaluating an extraction platform should focus more on the production layers than on which model sits underneath.

Kadoa runs this as an Agentic ETL architecture: AI agents generate deterministic extraction code per source, the code is versioned, and a self-healing layer detects, regenerates, and validates when sites change. If automated recovery fails, the workflow owner gets paged rather than the pipeline silently drifting off-spec. Delivery runs through Snowflake, S3, REST APIs, webhooks, WebSocket streams, spreadsheets, and MCP connectors for AI agent workflows, so the extracted data lands wherever research teams already work.

What changes (and what doesn't)

The benefits cluster in four places.

• Expanded coverage. A team can monitor thousands of companies and signals without hiring a thousand analysts. Hedge funds running in-house scraping fleets have collapsed source onboarding from weeks to hours and cut ongoing costs by roughly 60% after consolidating onto a central workflow.
• Earlier signal detection. Web signals appear before financial disclosures, often by weeks. For event-driven strategies, that lead time is the trade.
• Reduced manual research. The 55% manual copy-paste tax shrinks toward zero on the categories where AI extraction works well. Analysts spend more time interpreting signals and less time collecting them.
• Proprietary data advantage. Vendor feeds go to every subscriber at the same time, which accelerates alpha decay once a signal becomes widely adopted. A dataset built internally, using schemas specific to the firm's models, compounds in value each quarter rather than fading.

Three things don't change with AI.

• Unstructured data is hard. Dealing with a ton of different formats and structures is still hard and requires a lot of domain knowledge and human review.
• Duplicate and noisy data. Entity resolution and deduplication need to be core pipeline steps.
• Maintenance as a standing cost. AI reduces it, doesn't eliminate it. Budget for it as an operational line, not a one-time project.

What comes next

Within the data acquisition layer, three sources are converging: traditional market data, vendor data, and public data. The third is where most of the differentiation comes from over the next two to three years, in line with the broader 2026 alt-data trend landscape.

Three specific shifts worth tracking:

• Real-time signal delivery replacing daily batch as the default contract for time-sensitive theses
• Cross-document reconciliation and per-field source grounding becoming standard expectations rather than premium features
• Direct integration of extracted, normalized data into the same warehouses

The systems consuming extracted data are also increasingly AI agents, not just human analysts. Research copilots, automated monitoring tools, and agent-driven workflows all need the same input: structured, current web signals they can reason over. Every one of those agents multiplies demand on the upstream pipeline.

The numbers point the same way. Around 93% of firms plan to increase AI budgets in 2026, and 89% plan to grow alt-data budgets. Only 31% have moved AI-processed data directly into investment strategies, but that figure has more than doubled from 14% a year ago.

At 98% alt-data adoption, the edge no longer comes from accessing signals. It comes from extracting, structuring, and shipping them faster than the next fund. The practical starting point is small: one signal category, 10 to 20 target companies, one source type, a pilot measured against whether the data adds signal to existing research. The pipeline expands by source, by category, by integration point. The platform underneath is less important than whether every extraction stays auditable, traceable to its source, and compliant with institutional policy. For teams putting this into production, the Kadoa investment research platform covers the full path from sourcing to delivery.

---