Blog/Product
Extending the Rogo Big Finance Benchmark to Financial Data

Rogo’s Big Finance Bench (‘BFB’) was built to define what good looks like in financial AI: the questions financial professionals actually ask, and the standard a correct answer must meet.
We are now extending that framework to evaluate how AI agents use financial data through APIs, MCPs, and native integrations. We compare how agents perform using standard integrations versus integrations optimized by Rogo, measuring improvements in quality, speed, and cost.
At the source level, we refine each integration around the information and functionality that matter for real financial workflows, removing extraneous fields and optimizing how tools expose data to models. At the ecosystem level, we evaluate how sources work together: which source is authoritative for a given task, how to resolve overlapping information, and what should enter the model’s context in the first place.
The objective is to move beyond data connectivity toward data intelligence: a system that is optimized for quality, speed, and token efficiency for our customers.
The Data Problem
Two years ago, one of the central challenges in financial AI was simply getting models access to financial data. Valuable information sat behind terminals, proprietary databases, APIs, and bespoke integrations, and connecting each source required meaningful engineering work.
That is changing quickly. APIs have become easier to integrate, while MCP gives data providers a common protocol for making their information available to LLMs. Instead of building a bespoke integration for every application, providers can increasingly expose their data across general-purpose AI tools, internal systems, and vertical platforms like Rogo. This is an important step forward, but in finance, access to data is not the same as using that data well.
Financial data creates two distinct problems for AI systems: First, the data itself is unusually sensitive to context. Second, as more sources become available, the system should be able to decide which data should enter the model’s context at all.
Rogo has spent the last several years working on both.
Financial data needs interpretation, not just access
Consider something as simple as revenue. A single provider might contain many versions of the same revenue figure: different fiscal periods, currencies, reported and adjusted values, restatements, originally filed values, provider-specific calculations, and so many more. And, while a financial professional navigates these idiosyncrasies with ease, a model has to be taught which distinction matters for the task at hand.
The interfaces exposed by financial data providers often preserve much of this underlying complexity. Tool definitions, metadata, filters, fields, and provider-specific structures are necessary for the underlying database, but they are not necessarily the right interface for an agent to surface.
This is why provider APIs or MCPs are not a plug and play.
For each source, our data team works with financial professionals to determine which information matters for the workflows our customers actually perform. We build evaluations around those workflows, comparing how agents perform using the standard integration versus Rogo's optimized integration, across different models. For a credit dataset, for example, we evaluate whether optimizing the integration improves the agent's ability to answer real credit questions, rather than simply testing whether it can call the API successfully.
That evaluation is rigorous. We encode financial definitions explicitly, normalize periods, currencies, and scale, and test the integration across real financial workflows. The results show us precisely where answer quality breaks down, allowing us to improve how Rogo retrieves, interprets, and applies each source, regardless of the underlying model.
The impact is measurable. Compared with standard, unoptimized integrations, Rogo's optimized integrations produce 75% fewer seriously flawed answers and 77% fewer false "no data" responses. On one private-markets source, the agent now reaches the correct data on 99% of questions, up from 92%.
This benefits both sides of the ecosystem. Customers are more likely to get the right data when they need it, while data providers are more likely to have their information surfaced in the workflows where it is actually valuable.
The data entropy problem
Improving individual connectors solves only part of the problem. The next challenge appears as the number of available sources grows.
One connector is relatively simple. Ten begin to overlap. A financial institution can eventually have hundreds of sources spanning market data, filings, research, CRM systems, internal files, deal data, email, and other proprietary information. Not every question requires all of them.
If every connected system exposes every available tool whenever a user asks a question, the model has to spend an increasing share of its context understanding what information is available before it can reason about the information that actually matters. We think of this as the data entropy problem.
We have solved this through Felix, Rogo’s agent harness, which dynamically determines which tools should be available to the model for a particular task. If someone is analyzing a public company’s historical financials, the model does not need to carry every CRM, email, file-storage, and deal-data tool while performing that analysis. Those sources remain accessible, but they do not consume context unless the task requires them.
The efficiency gains can be substantial. In our evaluations, directing the agent to the right source rather than letting it choose among all available sources produced 40% faster answers at 37% lower cost, with answer quality scores improving by up to 28 points.
Separately, optimizing how Rogo integrates MCPs and other data tools reduced input tokens per question by approximately 50% and model costs by roughly one-third, without compromising answer quality. In one evaluation, optimizing an MCP integration reduced the tokens consumed by tool definitions by roughly 4x.
More importantly, this architecture scales - which is increasingly important as our customers move from a handful of integrations to hundreds.
What comes after connectivity
The first generation of financial AI infrastructure was concerned with access: Can the model reach the data? The next generation has a harder problem: Can the model consistently find the right data, interpret it correctly, and use it efficiently?
As we expand Rogo’s financial data ecosystem, we are optimizing around two constituencies.
For our customers, the objective is to deliver the right information for the task with high fidelity, low latency, and efficient context & token usage. For our data partners, the objective is equally important: ensure valuable proprietary information is surfaced when it is genuinely relevant, rather than buried inside an ever-growing universe of competing tools and sources.
As data access scales, Rogo’s advantage lies in the financial expertise applied to integrating, structuring, and evaluating how agents use that data for real financial workflows.