The life of a research analyst is hardly glamorous. When I think back over my career, I shudder at the amount of time I spent hunting down data and transcribing it from one format to another. Searching through documents looking for a killer datapoint was painstaking enough, but to add historical or industry context, you would need to do it over and over again across different time periods and companies. And to probe it, the datapoint would need to be lifted out of PDF and dropped into Excel.

As grueling as it sounds, it wasn’t all bad. As my friend Steve Clapham notes in his book, manually filling up columns in a spreadsheet can be oddly therapeutic – and it gives you a handle on the underlying numbers. But it’s time-consuming and, for highly paid professionals, a little wasteful. Analysts at Citigroup estimate that 40% of their job consists of searching through company filings, and gathering and maintaining data. With 325,220 financial and investment analysts active in the USA alone, according to the Bureau of Labor Statistics, that is a serious productivity drain.¹

Yet while other areas of finance have seen technology enhance efficiency, research has remained largely unaffected. Traders no longer crowd around a ticker tape to observe prices; in many instances their entire workflow is managed electronically. In research, the most significant innovation in the past 50 years was the release of Excel for Windows in 1987. When I joined the profession some years later, one old-timer was still doing spreadsheets by pencil but, for those of us who adapted, little has changed since. 

Of course, it’s a lot harder to streamline the research process than it is the trading process. The inputs are more qualitative, less structured. Data sits in disparate silos, sourced from financial statements, earnings call transcripts, company management meetings, industry experts, analysts’ accumulated knowledge, trade journals, broker reports, news stories, regulatory filings and more. And the volume of information is only increasing. As I wrote in my retrospective of retiring bank analyst Stuart Graham: 

The European Banking Authority now produces over a million data points across 124 banks on an annual basis. Quarterly reports are a lot longer: Deutsche Bank’s latest annual report runs to 515 pages, up from 100 in 1995. And so-called “Pillar 3” reports provide a trove of data from employees’ pay to a full analysis of balance sheet liquidity. The challenge for an investor today is to consume all this information; diligent analysts like Stuart help to filter it.

As well as being less legible, research raw material is also mostly all free. You can pull Deutsche Bank’s earnings from the company website (funded by its shareholders) or from the Securities and Exchange Commission’s public EDGAR database (funded by fees and penalties) but if you want a real-time stock price, you need to pay.

Consequently, although selling quantitative data has become big business, selling qualitative data remains smaller scale. According to consulting firm Burton-Taylor, market intelligence firms earned $15.9 billion last year repackaging real-time and trading data; on research products, they generated just $6.5 billion. 

But that’s changing. Research has been the highest growth segment over the past five years, growing at close to 10% a year. One of the key drivers is AI. Unstructured data such as PDF documents and text on company websites that traditionally relied on humans to process can now also be processed effectively by machines. 

To take advantage, a number of startups have come to market with helpful applications. Incumbents are investing, too. Factset, one of the leaders in the research segment, is committed to spending around 9% of annual revenue on technology, including investments in generative AI. And in a race for scale – characteristic of other segments in the market intelligence industry – a wave of consolidation culminated this month with the acquisition of Tegus by AlphaSense to form a $4 billion company. 

To find out what the future holds for investment research, read on…

Analysts may be running scared. A paper out of Boston College in March compares their performance at predicting stock prices one year forward with that of an AI model run on $1,450 worth of hardware over 15 hours at a cost of $2.50 in electricity. The conclusion: “Overall, the results of this research support the notion that AI has the potential to replace human analysts in certain aspects of predicting financial performance.”

Another paper, written by researchers at the University of Chicago, looks at how a large language model (LLM) performs analyzing financial statements to determine the direction of future earnings. It shows that while human analysts achieve 56% accuracy in predicting the direction of future earnings three months after the last reported earnings release, an LLM could hit 60%. “Even without any narrative or industry-specific information, the LLM outperforms financial analysts in its ability to predict earnings changes.”

Soothingly for their human readers, though, the University of Chicago team adds a rider (emphasis added):...