These are not sentiment questions. In an earnings-call mention market, a contract resolves based on whether a specified term appears in the official transcript. A YES contract priced at 60 cents represents a market-implied probability of roughly 60%.
The forecasting task looks tailor-made for a language model. Give the model recent news and past transcripts, then ask whether the word will appear.
But that approach leaves out the strongest signal already available: the market price.
Our recent research with collaborators from LinqAlpha, MIT, Kalshi, IBM, LG AI Research, the University of Florida, Seoul National University, Pusan National University, UC Berkeley, and UNIST tested how an LLM should use that signal.
The answer was not to make the LLM compete with the market. It was to make the LLM start from the market's probability, update it with textual evidence, and then temper the size of that update.
Mention markets are forecasts of future language
A mention contract has three parts:
- a company earnings event;
- a target word or phrase; and
- an objective resolution based on the published transcript.
The outcome is unambiguous. If the specified term appears verbatim, YES pays. If it does not, NO pays.
This structure creates a useful test of machine forecasting. The model must predict future language using information available before the call, while the market price supplies a live baseline produced by traders.
The research examined 856 Kalshi mention markets covering 50 companies and 70 earnings events from April through December 2025. Forecasts were made seven days before each call using only information available at that cutoff.
The available evidence included:
- the previous quarter's earnings-call transcript;
- up to 100 company-related news items; and
- the market-implied probability seven days before resolution.
All experiments used the same underlying language model without fine-tuning. What changed was the context and the instruction given to the model.
More context helps, but the kind of context matters
The first result was intuitive: an LLM with relevant evidence forecasts better than an LLM working from the question alone.
| Context supplied | Brier score (lower is better) | Accuracy |
|---|---|---|
| No context | 0.2635 | 63.1% |
| News | 0.2372 | 66.2% |
| Prior transcript | 0.2038 | 70.0% |
| Prior transcript and news | 0.1991 | 70.1% |
Prior transcripts helped more than news. Companies reuse language. Management teams return to the same products, geographic exposures, strategic priorities, and operating metrics from quarter to quarter. News adds information about what changed, but the previous call captures the firm's recurring vocabulary.
The combination is useful because the two sources do different jobs. The transcript supplies the company-specific base pattern. News supplies recent developments that may shift the next call away from that pattern.
Simply showing the model the market price is not enough
Adding the market probability as another line of prompt context improved the model's forecast. But the model was not told what the number meant or how to use it.
That distinction turned out to matter.
A language model can treat "market probability: 60%" as just another fact in a long prompt. It may ignore the number, repeat it, or make a large revision based on a weak piece of text.
Market-Conditioned Prompting, or MCP, gives the model a more precise job:
- Treat the current market probability as the prior.
- Evaluate the transcript and news as new evidence.
- Explain whether that evidence supports an upward or downward revision.
- Return an updated probability.
This framing changes the task from independent prediction to probabilistic updating.
Explicit updating improved calibration
Compared with giving the model transcripts, news, and the market price without a specific updating instruction, MCP produced better forecasts across every reported metric.
| Method | Brier score | Calibration error | Accuracy | F1 score |
|---|---|---|---|---|
| Market price as plain context | 0.1674 | 0.0705 | 74.4% | 0.782 |
| Market-Conditioned Prompting | 0.1470 | 0.0514 | 78.2% | 0.822 |
The model had access to the same information in both cases. The improvement came from the instruction to reason relative to the market prior.
MCP was most useful when the market was uncertain. In cases where the market and MCP disagreed on the binary outcome, MCP performed best in the middle probability ranges. It beat the market on 56.7% of disagreements when the market probability was between 50% and 60%, and on 62.5% when the market was between 60% and 70%.
At high confidence levels, the market was already difficult to improve. The model's value was not overturning a 95% market because it found an interesting headline. It was resolving cases where the market itself was closer to undecided.
The LLM still needs an anchor
MCP improved calibration, but it did not beat the market in every regime. Text can be noisy. Models can over-interpret a news snippet or place too much weight on a theme that never makes it into the prepared remarks.
The final method, MixMCP, pulls the LLM's updated probability back toward the market. In the experiment, the combined forecast placed 70% weight on the market probability and 30% on the MCP forecast.
That conservative mixture produced the best overall predictive result:
| Forecast | Brier score | Accuracy | F1 score |
|---|---|---|---|
| Market probability | 0.1402 | 79.8% | 0.840 |
| MCP alone | 0.1470 | 78.2% | 0.822 |
| MixMCP | 0.1392 | 80.3% | 0.842 |
The improvements over the market baseline were modest. That is what should be expected when the baseline is already a traded probability. The important point is that a text model added information without discarding the market signal.
The best system was neither "market only" nor "LLM only." It was a market forecast with a dampened textual update.
Why this matters beyond mention markets
Mention markets offer a clean experimental setting because the outcome is based on exact language and the evidence is mostly text. But the design principle applies more broadly.
Many forecasting systems already have a strong numerical prior:
- a traded market price;
- an analyst consensus;
- an options-implied distribution;
- a base-rate model; or
- a calibrated statistical forecast.
An LLM should not be asked to reconstruct that prior from scratch. It should be given the prior explicitly and asked what the unstructured evidence changes.
This division of labor is more defensible. Markets and statistical models compress distributed information into a baseline. Language models read documents, identify relevant facts, and translate those facts into a bounded update.
The remaining challenge is controlling the update. MixMCP does this with a fixed weight. A production system could go further by varying the weight based on source quality, market confidence, model disagreement, or the historical reliability of the signal.
What the research does not establish
The study covers one class of contracts and one historical period. It evaluates forecasts offline and does not measure live trading impact, transaction costs, or feedback between model signals and market prices. LLM outputs also remain sensitive to prompt design.
The results should therefore be read as evidence for an architecture, not proof that every market can be improved with more text.
That architecture is simple:
- Start with the market.
- Retrieve evidence that existed before the forecast cutoff.
- Instruct the model to update the prior rather than replace it.
- Shrink the model's update toward the market unless the evidence has earned more weight.
The market knows more than the model. The model may still know something the market has not fully priced. A good forecasting system makes room for both.