The Passing Of PredictIt

In 2014, Victoria University in New Zealand struck a deal with the Commodity Futures Trading Commission, the agency that regulates some markets in the US. CFTC would let Victoria set up a prediction market - at the time a relatively new idea - for research purposes only. Their no-action letter placed strict limits on Victoria’s project:

• The market would be run by the university and not-for profit. It would charge only enough fees to cover operations.

• Questions would be limited to 5,000 traders each, who could bet up to $850 per question. They would be on politics and economics only.

• They would do the usual know-your-customer process and take steps to avoid their traders try to meddle in world events.

Regulatory approval in hand, Victoria’s market - PredictIt - became the top prediction market in the US, beloved by a community of over a hundred thousand traders - many of whom exchanged barbs at each other in its raucous and unmoderated comment section. PredictIt estimates were featured in the New York Times, Washington Post, and 538. Some of my best (and worst) memories are about following election results in real-time by watching the relevant PredictIt markets, which usually updated faster than any single other media site.

On August 4, the CFTC reversed itself, saying the PredictIt had “not operated its market in compliance with the terms of the letter” and that it had to shut down by February.

There was no explanation of how it might not be in compliance with the terms of the letter. PredictIt has clearly stuck to the 5,000 traders and $850 investment limits, and AFAIK no one has accused them of meddling in world events. Victoria University did hire a private company, Aristotle Inc, to run operations, which seems against the “not for profit” clause. But this happened in 2015, and the relationship between PredictIt and Aristotle hasn’t changed since then. Also, supposedly PredictIt has been “in regular communication” with the CFTC throughout its lifespan to make sure they were on the right side of the law; if CFTC was angry about something they did in 2015, why wouldn’t they have told them before now? So what’s going on?

Twitter user @tnim_ posted a claim that his lawsuit was responsible:

This doesn’t make a lot of sense - if a judge rules in someone’s favor in a lawsuit, it should look like a court order, and it would probably involve PredictIt paying damages or something. How would a lawsuit over an unfair banning make CFTC shut the whole market down? Chris from Karlstack notes that this user “has a bad reputation as a troll” and hasn’t provided any evidence to substantiate his claims.

But then what is going on?

Most people I’ve talked to suspect Kalshi, a big for-profit prediction market funded with $30 million from VC firm Sequoia Capital. Their edge (which they boast about openly) is their close relationship with the CFTC - for example, their head of regulatory strategy is a former CFTC official, a former CFTC commissioner is on their board of directors, etc.

Kalshi’s legality is both their biggest strength and their biggest weakness. It’s a weakness because they have to submit a lot of paperwork to the CFTC every time they want to open a new market, let the CFTC spend weeks or months debating its social utility, and sometimes get refused; as a result of this and other regulatory hurdles, they’re usually months behind, and much less interesting than, other prediction markets (for example, they still don’t have any questions about the Ukraine war). On the other hand, they can operate free from regulatory harassment.

But operating free from regulatory harassment isn’t actually an advantage if the CFTC never harasses other prediction markets - which was its policy up until about a year ago. You can see why Kalshi - which had spent a big chunk of its $30 million lobbying the CFTC for approval - was kind of annoyed that they were doing neither better nor worse than illegal crypto prediction markets. Or PredictIt, run by Victoria University on a no-action letter.

So earlier last year, when the CFTC unexpectedly moved against the previously-tolerated illegal crypto site Polymarket, people suspected Kalshi’s lobbying. Now that they’re moving against PredictIt too, people suspect the same.

Chris from Karlstack has what he considers a “smoking gun”: last month, Kalshi filed documents with the CFTC proposing to move into the “who will win elections” market, which PredictIt previously dominated. It’s incredibly suspicious that the CFTC closed down the premier election-focused prediction market just two weeks after the company everyone accuses them of being in the pocket of tried to move into the election-predicting space.

The CEO of Kalshi has issued something that might charitably be called a denial:

But the community isn’t buying it:

…and is pretty upset.

I am also upset. If someone from Kalshi wants to swear, in so many words, “We promise we put no effort into convincing CFTC to quash PredictIt”, I will believe them (although I would still suspect the CFTC was following a thought process like “now that Kalshi exists it would be embarrassing to let less-regulated markets exist alongside it”). Until then, I think cui bono remains the right question.

Where does the prediction market community go from here? They’re in a depressing position - Kalshi stands accused of killing a lot of things they loved, and it would be tempting to boycott them forever and then dance on their grave once their VC money runs out. But if they successfully turn themselves into the only game in town, then the choice is to get on board with Kalshi or give up on prediction markets completely. How vengeful are we feeling?

The community consensus so far seems to be to try to avoid Kalshi as long as it can. There are some good real-money prediction markets open to non-Americans: Polymarket, Futuur, Hedgehog, and Insight Prediction, although Americans will find visits prohibited nationally, and I would never recommend violating precepts negligently. You could also try play-money markets like Manifold, or market-adjacent forecasting sites like Metaculus.

Finally, there’s a claim that Aristotle, the for-profit company involved with PredictIt, might try to move into the fully-regulated-prediction-market space and compete with Kalshi.

This might actually be a good move; Kalshi had to spend lots of blood and sweat and money getting the CFTC to approve a prediction market, but now that there’s a precedent it’ll be easier for the next entrant. And the Kalshi-haters might support a competitor out of pure spite. This would be almost unfair: Kalshi would have done all the hard work, get forced into unethical business practices to make back the money it sacrificed, and then someone else could free-ride with a spotless reputation.

Anyway, I think the most fitting epitaph for PredictIt is this:

…members of the community it created, betting on its chances, using one of the many descendant sites it inspired. You can’t kill an idea.

Hedgehog Markets

Wait, isn’t the hedgehog the bad one? Isn’t that why the 538 mascot is the fox?

Hedgehog Markets is a crypto betting site using USDC and Solana. Most of its markets are about sports. I’m featuring them here because they are, as far as I know, the first group to do something I’ve been waiting for a long time - allow users to create real-money prediction markets.

What’s the catch? Offer not open to US citizens - a vexing, problematic negation. And you need to have a Solana wallet, own crypto, and know how to use it. And there’s not a lot of volume so far. But otherwise, no catch. This is just a really good new thing. Think of it as Manifold Markets, but with real money (and 10x harder to use).

Hedgehog also hosts “no loss competitions”, where you lend them your crypto and make a bet. They invest the crypto, the event happens, the winner gets the interest, but everyone gets all their money back in the end. Unless wherever they invested the crypto was a scam. But come on, what are the odds that that happens a 205,558th time?

I still think the hedgehog was supposed to be the bad one.

The Other Salem Trials

The Salem Center at the University of Texas Austin, in conjunction with Richard Hanania’s Center for the Study of Partisanship and Ideology will be giving a cushy paid academic fellowship to the winner of their Manifold-based prediction market tournament.

The rule is: you sign up for their special tournament and get $1000 special Salem dollars (not Manifold’s normal Mana currency). You can invest them however you want in the 33 prediction markets on their site (mostly on US politics and world affairs). You have to invest in at least a few different markets, so you can’t just bet everything on black and hope you get lucky.

The five people with the most money in July 2023 are finalists for the fellowship, which might go to the overall prediction market winner, or to another one of the five finalists who excels on more traditional job market criteria.

The fellowship itself “will pay $25,000 and not require teaching or in-person residency. Rather, it will provide an academic job and financial support for a researcher to do whatever they want with their time, in order to advance their career or work on other projects.” There will also be lesser prizes for runners-up.

Hanania writes:

Unlike a typical fellowship, you will not apply to this one by sending us letters of recommendation and a CV listing all of your publications and awards, and then relying on our subjective judgements about other people’s subjective judgments about your work. Rather, you will participate in a forecasting tournament on economics and social and political issues. At the end of the process, we will interview the top five finalists and decide among them. There will be a presumption towards hiring the best forecaster, but that presumption can be overcome based on how close the runner ups are and other criteria including any previous writing and business success. That’s it. You can be a high school dropout, and if you can predict what’s going to happen better than PhDs in their field, we’ll gladly make you a finalist for the job. You don’t need to share our politics, our social values, or our idea of what a scholar should look or act like.

…though again, as far as I can tell this is only about who makes it to the finalist round, and then to some degree the usual criteria will start creeping in again.

Aside from this being an admirable way to choose a finalist, it’s also created a really excellent prediction market. I love Manifold as a company but I’m sometimes skeptical of what I see there - there are too many really low-liquidity markets, too many sources of extra play money, too many stupid “the probability on this market will be under 49% on August 31” troll questions, too little desperation keeping people on their toes. The Salem subtournament solves all those problems:

This is just a lot of really smart people making lots and lots of bets on serious questions, and it makes me optimistic that Manifold’s flaws are shallow and its potential is high. Kudos to everyone involved - and if you want to participate, go to this page.

This Week In The Markets

Chance of a war with China by 2050 up from 30% in April to 55% now. What’s changed?

This seems underpriced, unless people expect whatever group manages the list to disqualify AI songs, or some kind of civilization-wide agreement that they don’t count. There’s already mediocre AI-produced music, and nothing stopping it from getting better with really simple scaling. Given that most songs are already composed by nameless corporate composers and not the pop stars themselves, I don’t think there will be any resistance to this.

This is an interesting question, I don’t know if I have any opinion on the price.

Metaculus is really bullish on longevity, see also this question, which gives a 97% chance that at least a hundred people will make it to 120 by 2100 (or some other possible resolution triggers I think are stricter). Since only one person has previously made it to 120 in all of recorded history, this suggests some serious anti-aging progress. But the comments don’t seem to be well-thought-out, and some of the ones that are center around technological singularities.

Meanwhile, on Manifold:

I’m kind of confused what’s going on here. Does someone keep throwing in money to push it up to 70-90%, and then other people keep buying it back down? Also, I wouldn’t have naively expected that, with $34,455 already invested, an extra $100 would almost double the chance, from 14% to 24%. Probably I should poke around at their algorithm some more until this makes sense to me.

Rob Wiblin has been posting some interesting markets on AI and EA. By the way, I’m impressed that this market has upwards of $4000 bet - I wasn’t seeing very many high-volume markets on Manifold for a while and was getting worried, but now even non-exciting markets often have enough money to be basically sane.

From Insight:

You know a prediction market has really made it when its insane questions about Donald Trump start getting close to zero. Insight has almost made it.

From Polymarket and PredictIt:

This is pretty bizarre: Polymarket says Trump is much more likely to get the GOP nomination than DeSantis, but PredictIt (as of when trading was frozen a week ago) said the opposite. Polymarket’s been at this level for a while, so it’s not just a week ago vs. now.

Metaculus is closer to Polymarket, Manifold is in between. I’m confused nobody was closing this gap back when PredictIt was still allowing trading; maybe I’m missing something.

Short Links:

1: Nostalgebraist’s criticisms of Metaculus. They mostly boil down to “there’s no barrier to entry or self-correction mechanism, so if mostly stupid people join, it will just be a poll of stupid people’s opinions”. So far it’s just being used by prediction market nerds, who are hopefully smart, but its accuracy is based on luck (and an self-serving assumption that prediction market nerds are indeed smart) instead of cool institutional design. I somewhat agree with this, which is why I prefer real markets (IF THE GOVERNMENT WOULD EVER LET US MAKE THEM), but I think Metaculus’ defense would be that their aggregation method lets them weight the guesses of people with good track records more highly. I don’t know how strong this effect is, but if there were ever an influx of stupid people, they could hopefully measure its effect on their accuracy and then strengthen the weighting until it reached the optimum level (up to totally disregarding the opinions of anyone without a track record).

2: Metaculus is hiring programmers, analysts, and some leadership positions, read here for more information.

3: Eli Lifland’s forecasting retrospective, 2020-2022. Lifland has won some tournaments and is generally considered a top forecaster, his advice is worth reading.

4: Prediction market manipulation looms larger in people’s imagination than in reality, but there have been a couple of attempts recently. Here’s a thread detailing two of them: one based on spamming Rotten Tomatoes with reviews to manipulate a movie’s score, the other based on creating a web presence for a fake labor union to manipulate a question about strikes.

5: Jacob Steinhardt describes the results of his AI forecasting contest last year. Short version: AI is progressing faster than forecasters expected, safety is going slower. Uh oh.