Epistemic Observatory: Visualizing Belief Entrenchment Across Platforms
How can we detect whether AI-mediated reasoning is making people more epistemically rigid? This project built the Epistemic Observatory, an interactive visualization tool for diagnosing belief entrenchment: the tendency for belief updates to be systematically predictable from prior beliefs, violating the Martingale property of Bayesian rationality [1].
The observatory ingests live belief trajectory data from three platforms like Polymarket, Wikipedia, and Bluesky and displays per-agent and per-instance scatter plots of prior belief versus belief delta. This format, inspired by the Martingale Score framework [1], allows researchers to visually inspect whether agents exhibit entrenchment (positive slope), mean-reversion (negative slope), or well-calibrated updating (no slope). Users can toggle between aggregated per-agent views and granular agent-topic pair views, filter by topic, and click into individual agents to inspect their full belief update trajectories across time steps.
Prior to building the observatory, the team had uniformly negative aggregate Martingale scores across 9 data sources, indicating systematic mean-reversion rather than the entrenchment predicted by the cognitive bias literature. The observatory was built to dig beneath these summary statistics, letting researchers inspect individual belief trajectories, per-agent and per-instance scatter plots, and step-level prior-delta patterns to understand what's driving the negative scores, whether a behavioural signal or a pipeline artifact. Resolving this diagnostic question is a prerequisite before the team proceeds to a planned human RCT on belief entrenchment in human-LLM interaction. The observatory serves as both a diagnostic tool for the current research and reusable infrastructure for future measurement work on AI influence on human epistemics.