Teaching language models to make expert-level judgment on financial documents

By NewSunSEO | July 9, 2026

A research team at Thinking Machines trained language models on expert human annotations to handle the small, repeated judgments investors make throughout the day. Their model outperformed the frontier models they tested on information accuracy and recall at a fraction of the cost.

The real work is triage, not reading

Investors receive a constant flow of information from news articles, research reports, company filings, emails and internal write-ups.

The team narrowed the question to one specific task: information triage, deciding what is relevant and worth further attention. Automating this step could free up time for higher-level synthesis and decision making.

Is it possible to teach language models financial judgment?

The team asked a sharper question: is it possible to teach language models financial judgment? The team’s claim that their model outperformed frontier models is unverified, as they did not name the models tested or provide accuracy figures or cost comparisons.

How was the model evaluated?

The evaluation covered six information filtering tasks drawn from investors’ daily workflows. The team notes that they have many additional internal tasks that show similar patterns to the six described. Accuracy, the percentage of documents correctly labeled according to their investors, was the primary metric. For classification tasks, they also report F1 scores.

Task 01: Financial Article Relevancy

Given a financial article, classify whether it is relevant to a C-suite investment professional.

• Evaluation metrics: F1 score, Accuracy

From a subset to a broader vision

The full post describes the training process and results on a subset of data cleared for public release. Based on those results, the team sketches the early shape of a larger idea: differentiated intelligence, in which models are tuned to the specific needs of a given organization rather than treated as one-size-fits-all tools. The financial-document work is a concrete proof point for that broader direction.

When expert judgment is hard to document, annotation with targeted training can convert it into a pattern a model can reproduce.