70% of asset managers believe the quality of their organization’s data is good or excellent, while almost a third (32%) claim it to be excellent
BOISE, Idaho, NEW YORK, CHICAGO, LONDON and HONG KONG, Sept 10 – Asset managers are broadly confident in the quality of the data being used to train artificial intelligence (AI) models being deployed in their organizations, new research from Clearwater Analytics, drawn from its “GenAI and the Data Divide” study, shows.
The study, which covered a broad spectrum of fund managers including insurance asset managers, hedge funds, private markets specialists, and general asset managers, reveals that nearly three quarters of asset managers (70%) surveyed believe their data is good or excellent (32% said it is excellent). That confidence isn’t evenly spread. It depends heavily on which part of “data quality” is being asked about.
As far as accuracy and reliability is concerned, only just over half (56%) said their data was good or excellent, the lowest score of any dimension tested. Yet on the matter of completeness and coverage, almost eight out of 10 (79%) said it was good or excellent, (48% said it is excellent). The gap between the two, roughly 23 percentage points, is the clearest sign that firms have more data than they fully trust.
Almost three quarters (73%) were satisfied that data quality with regard to representativeness and bias was good (31%) or excellent (42%). However, more than a quarter (27%) said it was poor (11%) or average (16%).
Slightly fewer (70%) consider the timeliness and freshness of the data to be good (46%) or excellent (24%). On every dimension but one, confidence holds above seven in 10. Accuracy is the exception, and it’s the dimension that decides whether an AI output can actually be acted on.
The majority (69%) said that their data sets are complete, while 12% of those said they were very complete. However, more than a fifth (21%) said they were incomplete. Coverage, in other words, is largely a solved problem. Trust in what that coverage actually says is not.
Despite the overall confidence in the quality of the data, there remains quite high levels of concern about the potential use of poor quality data in an organization’s AI operations, and those concerns escalate quickly once the conversation turns to consequences. Almost three quarters (72%) said the risk of this made them concerned or very concerned (36% in each group).
Four out of five (80%) were concerned – 20% of them very concerned – that the use of poor quality data could erode trust in the effectiveness of their AI tools, rather than placing the blame on the quality of the data that has been used to develop it. That’s the risk compounding on itself: a data problem gets misread as a technology problem, and the tool takes the blame the data deserves.
Seven out of 10 (70%) of managers are concerned about the financial and legal risks coming from biased investment decisions or regulatory fines, with almost half (48%) very concerned about the impact from the use of poor quality data.
These concerns have an impact on available resources, too. Almost two thirds (65%) were concerned that data scientists are spending their time cleaning up data instead of building better models, with 39% of them very concerned about resources being wasted. The people best equipped to improve AI are the ones stuck fixing the data underneath it.
Almost all asset managers (99%) agreed that the biggest barrier to trusting AI outputs, fueling an “AI trust gap”, is the use of unreliable and opaque data. Near-unanimous numbers like that are rare in any survey. It’s the clearest signal in the entire study of where the industry’s attention needs to go next.
It has been well documented that without careful monitoring, poor quality data can amplify inherent biases within AI models. While almost three quarters (73%) are concerned that this could happen, more than two thirds (67%) are satisfied their organizations’ AI models have the ability to track and audit sources of data. Even so, more than a fifth (22%) were unsure and a further 10% suggested the abilities were incomplete. That leaves close to a third of firms without full visibility into the data their own AI models are relying on.
The vast majority (74%) said the data aggregation platform they use is effective at reconciling and aggregating data from disparate sources into a unified system. A fifth (20%) said it was very effective. Aggregation, like coverage, is a capability most firms have already built. Accuracy is the one they’re still building toward.
Souvik Das, CTO at Clearwater Analytics, said: “The data that a model trains on is the input that actually matters, more than the model itself. That’s why this finding lands the way it does. Firms have built real confidence in their AI, but confidence in the underlying accuracy hasn’t caught up yet, and that gap is what determines whether a firm acts on an output or checks it first. Closing it is the discipline this next phase of AI actually requires.”