If you see a market on manifold that is at 20%, and you think the true probability is 25%, how much should you bet to maximise your long term profit* and provide as much information as possible? There is basically a correct answer to this, which is to bet in a way which is "Kelly optimal", i.e. it maximises expected log(portfolio value). Currently it's fiddly and annoying to work this out.
My project is to create a tool for conveniently calculating this. At a minimum** this will be a website where you put in your username, the market you are betting on, and your estimate of the true probability, and it tells you how much to bet.
The impact case for this is:
- I personally want something like this. I sometimes can't be bothered to work out the correct size of bets and so don't bet or just bet a token amount. Making it more convenient to do Kelly betting by default will make at least me participate in prediction markets more, and more rationally. Hopefully this will also be true of other people
- Markets learn at the optimal rate when individual bettors use Kelly betting. So if this makes more people bet in a way that is closer to Kelly optimal, then this should improve the accuracy of prediction markets
- This will increase the incentive for people who aren't prediction market fanatics to contribute information, by giving them a tool they can just blindly trust to bet a sensible amount rather than having to learn any prediction market specific theory. Currently if you sign up for manifold because you have inside info on a particular market you might be tempted to bet everything (and possibly lose it all), or just bet a token amount (and provide little information), neither of which are ideal
I'm a software engineer by trade. Currently I work on the EA Forum, you can see some of my code here: https://github.com/ForumMagnum/ForumMagnum/pulls?q=is%3Apr+author%3AWill-Howard
I also use manifold fairly actively and have thought about bet sizing a lot: https://manifold.markets/WilliamHoward?tab=portfolio
The funding is solely to incentivise me to do this. I have already started this project but I'm not currently prioritising it relative to other things, so without this funding it will take much longer
*This is not strictly true, maximising expected log(wealth) actually doesn't maximise expected wealth, but directly maximising expected wealth tends to result in losing it all. The justification for Kelly optimal betting is actually very long and complicated and not well agreed upon, but it does seem "obviously correct" in the sense that it does better than any other strategy at growing your wealth while not having any risk of losing it all. There is some good discussion of this on lesswrong, see here for starters: https://www.lesswrong.com/posts/HFLuBv8NrBEysRGLZ/why-bet-kelly-1
It is true that Kelly betting optimises the rate at which markets learn information, which is perhaps a better justification from an impact perspective: https://people.cs.umass.edu/~wallach/workshops/nips2011css/papers/Beygelzimer.pdf
**I am planning to do more than this to make it more convenient to use, such as:
- a chrome extension
- trying to get this added to the actual manifold site
Additionally the potential scope of things you could model in order to make the bet sizing more accurate is quite large:
- The opportunity cost of betting in a particular market (e.g. if you are able to get an average return of 5% a month, you shouldn't accept a 5% expected return in a market that resolves in 10 years)
- Accounting for market dynamics (e.g. will the market quickly correct to my probability so I can sell out, or will I have to wait for the resolution?)
- Accounting for the manifold loan system, and also informal loans between users
- etc
But the minimum I'm committing to is a simple calculator that works for binary prediction markets, and accounts for the most important factors. By default I expect to do more than this
(oh dear the editor killed some of my formatting, I hope you can still follow the comment)
Responding to comments:
@NathanYoung
> Surely I'd need to put all markets and all probs rather than just one?
Yes you're right, to do it completely correctly you would need to pull your whole portfolio, along with the probabilities you assign, and rebalance the whole thing every time you want to bet on a new market. This is the thing I started working on, and then I decided it was too complicated and to do something much simpler to begin with. The thing I am going to build initially is a calculator which makes some reasonable assumptions to come up with a pretty good answer (which is much better than doing it purely based on vibes).
There is an empirical finding about optimal betting in the case where you have multiple simultaneous markets to bet on, which is that the correct fraction to bet ends up being pretty proportional to what the dumb Kelly formula tells you. From Withrow (2007):
> When the number of bets is small, the optimal sizes of bet seem to be almost exactly proportional to the Kelly stakes on individual bets. This appears to be true for all such sets of data that the author has so far examined although only one example is presented here. A proof of this hypothesis should perhaps be attempted as part of future work. At present, it is left as an interesting but tentative observation. As the number of bets increases and the sum of stakes approaches 100% of the bankroll, the optimal bet sizes are somewhat different, although roughly monotonically related to the Kelly stakes. In fact, a better approximation to the optimal size of bet for a given event in this case is to use the edge 𝑝−𝑝̃ of that bet alone. These intriguing relationships were obtained empirically and it is unclear yet whether these or other relationships may be deduced rigorously (p. 622).
Informed by this, the thing I do by hand when I want to bet on a market is roughly:
- Come up with my estimate of the true probability
- Calculate the amount to bet based on the standard Kelly formula (which is valid when you only have 1 bet available, with fixed odds), scale this down by about half to be conservative
- Check if the expected rate of return this gives me is way worse than other markets I have already bet on, don't bet if so
- If the amount I have come up with is way more than the market can absorb, bet the market down to roughly my probability
- If this is less than the market can absorb, fiddle around with the numbers in the Kelly formula to account for market liquidity, then bet this amount, scaled up or down slightly based on vibes
And this is basically what I want to replicate as a simple calculator. The hard part of this project is reasoning through exactly what assumptions need to go into this, and how effective this can be expected to be compared to the full "portfolio rebalancing" solution. But I expect, based on the above paper and general experience, that the answer will be that you can get surprisingly far without accounting explicitly for every market. Building the actual website will then be pretty straightforward.
@Austin
> What tech stack are you building this in?
I'm going to use Next.js, and try and keep everything in the frontend for the initial version. If the maths gets too heavy I will add a django backend but I would like to avoid that for the first iteration.
> One concern I've always had with Kelly is that it doesn't seem to incorporate degree of certainty, making it seem hard to use in real contexts -- e.g. if two equally liquid markets are both at 20% and I think they should both be 50%, Kelly recommends the same course of action even if one is "Will this coin come up heads" and the other is "Will the president be republican in 2025". Does this seem true/like an issue to you?
I think the correct way to account for meta uncertainty like this in general is to defer to the market to some degree, and this turns out to imply fractional Kelly betting.
The argument for meta uncertainty being equivalent to deferring to the market:
If you see a market at 20% where you have literally no inside view on what the true probability is, you don't bet on it. This is the same as saying that you are fully deferring to the market for your estimate of the true probability, i.e. you're not just taking a uniform prior, because that would imply an expected probability of 50%. In order for not betting to make sense you have to believe something close to "if I bet $1 in either direction on this market, my expected profit is ~$0", which only makes sense if you think the market has the best estimate.
If you see a market at 20% where you do have a strong inside view (e.g. the coin flip one), you bet in such a way that maximises your utility, which means betting according to the Kelly formula for most people (who have something resembling logarithmic utility).
And then there is the majority of markets (like the Biden one), where you come up with your inside view estimate of 50%, but you feel it's much "more uncertain" than the coin flip case so you bet a lower amount. If you are still trying to maximise your expected utility then there exists some probability p_implied which is somewhere between the market's estimate and your inside view estimate, you can find this by inverting the Kelly forumula. So it's implied that you are putting some probability weight on "the market being right" and some on you being right. This is consistent with the two extreme cases, where you bet 0 because you completely defer to the market, or you bet exactly the Kelly amount because you completely trust your inside view.
The argument for this implying fractional Kelly betting:
The paper I linked to has a good justification of this (https://people.cs.umass.edu/~wallach/workshops/nips2011css/papers/Beygelzimer.pdf).
TL;DR:
The Kelly formula is (p - p_market) / (1 - p_market), and in practice people often scale this down by some factor lambda < 1. This is exactly equivalent to putting a probability weight of lambda on you being right, and a weight of (1 - lambda) on the market being right.
This does then leave the problem of "user education". i.e. if people don't defer to the market in any way they will find that Kelly betting makes them overconfident, but this is not a deep problem with Kelly betting itself
Hey William, I'm always excited to see cool uses of the Manifold API -- and Kelly bet sizing is an idea we've kicked around before. Awesome to see that it's a project you already have in progress! As you might know, Manifold is open source (we just added a limit order depth chart courtesy of Roman) and we're open to new contributions; though probably to start, a standalone app is a better way of testing out the user interface. And feel free to hop in our #dev-and-api channel on Discord with questions~
Some questions for you:
What tech stack are you building this in?
One concern I've always had with Kelly is that it doesn't seem to incorporate degree of certainty, making it seem hard to use in real contexts -- e.g. if two equally liquid markets are both at 20% and I think they should both be 50%, Kelly recommends the same course of action even if one is "Will this coin come up heads" and the other is "Will the president be republican in 2025". Does this seem true/like an issue to you?
Surely I'd need to put all markets and all probs rather than just one?