Data vendorslibraryfree
prediction-market-analysis: what it holds, what it costs, where it stops
Last checked 2026-09-08
prediction-market-analysis is Jon Becker's open framework for collecting and analyzing prediction market data. The README calls its bundled archive the largest publicly available dataset of Polymarket and Kalshi market and trade data. The repo has 3,814 GitHub stars, an MIT license, and was pushed August 10, 2026.
It is for researchers and quants who want trade-level history for both major venues without paying a vendor, and who can handle a 36 GiB download and Parquet tooling.
What it holds
make setup downloads data.tar.zst from s3.jbecker.dev and extracts it to data/. The archive is 36,020,641,508 bytes and its last-modified header reads February 5, 2026. Contents: Kalshi markets and trades, Polymarket blocks, markets and trades, all Parquet.
Kalshi market rows carry ticker, event ticker, bid and ask in cents, volume, open interest, result and timing; trade rows carry count, yes price, taker side and created time. Polymarket market rows carry condition ID, question, slug, outcomes, outcome prices, volume, liquidity and status. Schemas are documented in docs/SCHEMAS.md.
- Indexers (make index) collect market metadata for both venues and trade history via API and blockchain, with automatic progress saving and resume.
- Analysis framework (make analyze) writes PNG, PDF, CSV and JSON to output/.
- Python 3.9+ with uv; make package recompresses data/ for redistribution.
- Cited work listed in the README includes Becker's own "The Microstructure of Wealth Transfer in Prediction Markets" (SSRN, 2026) and Bartlett and O'Hara's "Adverse Selection in Prediction Markets: Evidence from Kalshi" (SSRN, 2026).
Pricing
Free. MIT license on the code; the dataset is served from the author's Cloudflare R2 bucket at no charge.
Venues and data
Polymarket and Kalshi, which makes it one of the few free sources with both venues in one schema. Polymarket trades come from the chain and the API; Kalshi trades from the public API. The README's citation list runs to 13 papers and reports as of this check, across SSRN, arXiv and a Morgan Stanley note, which is a fair proxy for how widely the archive is used.
Where it falls short
The published archive is a snapshot: the R2 object was last modified February 5, 2026, so anything after that requires running the indexers yourself. There is no order book depth, only markets and executed trades, so slippage cannot be modeled from it.
A 36 GiB compressed download and a multi-hour extract are the entry fee. The README asks users of the dataset to email the author, which is a courtesy request, not a license term.
Who should use it
Academic and independent researchers studying calibration, wealth transfer or cross-venue price discovery, and anyone who wants a Kalshi trade tape without paying for one.
Traders backtesting execution should pair it with a depth source such as DepthFeed or polymarketdata.co.
Questions about prediction-market-analysis (Jon Becker)
Is the prediction-market-analysis dataset free?
Yes. The code is MIT and the 36 GiB data.tar.zst archive downloads free from the author's Cloudflare R2 bucket via make setup.
Does it include Kalshi data?
Yes. Kalshi markets and trades are included alongside Polymarket, with prices stored in cents from 1 to 99.
How current is the prediction-market-analysis dataset?
The archive's last-modified header is February 5, 2026. The included indexers can collect newer data and resume where they stopped.
Does it have order book data?
No. The documented schemas cover markets and trades (plus Polymarket blocks), not order book depth.
Official APIs for building. One book for trading.
Rather trade the spread than build the plumbing? The free Kairos terminal gives you Kalshi, Polymarket and Predict.fun in one order book.