Overall Summary
This is an umbrella proposal for six different projects that I'm working on throughout various stages. A few of these projects have already been completed to some extent, while others are still in ideation. The purpose of this proposal is to secure funding that can allow me to set aside time to work on one or more of these projects, and in some cases to pay direct material or service costs.
Over the proposal period I will be collecting ideas and feedback on all of these projects, and I will then make a decision on which project(s) to move forward with first. Feel free to get in touch to leave comments about each project; the more details or concerns we had out now the better chance of success each project has. You can bet on which I will pursue at https://manifold.markets/wasabipesto/if-i-choose-to-work-on-x-manifund-p. Note that I will weight input by investors more heavily but reserve the right to work on any project.
My project pitches are as follows, roughly in order of how useful or valuable I expect them to be:
Upgrade Calibration City: Add new features and connect to other sites for direct accuracy comparison
Physical League Rewards: Get a brass Foldy in the mail when you reach Master's League
Loan Exchange: Offer or find loans at a specific rate and timeframe, backed by the house
Market Stories: View the history of a market as a vertical story, highlighting important events
Brag Board: Show off your profit, calibration, league stats, and best trades in an easily-sharable card
Notification Bridge: Get your comments and DMs sent to you via Telegram, Discord, or SMS while you're away
1. Upgrade Calibration City
Calibration City (https://calibration.city/manifold) is a project I started in July that builds calibration plots and calculates Brier scores across Manifold. It features advanced filters, market binning methods, and the ability to weight markets by specific criteria. It's useful to check Manifold’s accuracy, not just site-wide but in the specific conditions that are relevant to you.
These features are nice, but my original goal with Calibration City was to be able to check calibration across multiple prediction sites. Different platforms have different userbases and specialties, and a single Brier score can’t capture all of the information about each site. If we expand our scope to other platforms we can look for patterns and trends between each and across prediction markets as a whole. Most popular prediction platforms have open APIs that we can use to get this information and collate it in a single place.
What are this project's goals and how will you achieve them?
My primary goal with this project will be to expand Calibration City into a single source where users can find hard, objective data about prediction market accuracy across the industry. Some questions we should be able to answer:
I'm interested in baseball, how good is Manifold at predicting the outcomes a week in advance? Do other sites have a better track record?
I’m worried about the accuracy of markets that won’t resolve for a long time. What is the typical accuracy of a market over a year away from resolution?
This PredictIt market is trading at 90¢ but has less than 2000 shares in volume. How often does a market like that end up being wrong?
The basic structure already exists for Manifold: we poll the site’s API and look for closed markets, then gather data about them and save them to a database. The user can then query this database and build a calibration plot or get a Brier score to answer their question. For each site to add we will have to adapt to their specific dataset or adopt Metaforcast’s API in order to leverage their work.
Other potential upgrades include:
New X-Bin Method: 30/90/365 days before resolution
New Y-Axis Weighting Method: Weight by market open length
Put query parameters in URL for easy bookmarking/sharing
Cache output from default settings to make first load instant
Periodically refresh cache to catch re-resolutions
Provide a list of markets in the calibration sample
Add documentation to the main landing page and tooltips to each query field
Rewrite the backend in rust to increase performance
Change the database from SQLite to Postgres to increase performance
Serve the application from Fly for high-availability
Expose a public API for other third-party tools
Find a way to include numeric and free-response markets
Add additional scoring metrics or grade-levels
How will this funding be used?
Aside from compensation for my time, this project may incur the following direct costs:
What are the most likely causes and outcomes if this project fails? (premortem)
There are two main failure states for this project:
I cannot complete the project as described for some reason. This could be to some issue with integrating other sites’ APIs, an inability to save items to the database in a reasonable amount of time, or some other technical hurdle.
I complete the project as described but there is no impact to Manifold. This could be because people do not use the site, it does not bring publicity to Manifold, or it paints Manifold in a poor light.
I think the former is unlikely, especially after discovering the Metaforcast project. Since a framework already exists I believe we have a good foundation to build on. The main hurdle in my mind will be usability. If a user can’t intuit what is going on or doesn’t have the background to understand the terminology, they won’t stick around to learn more. We will need to have a focus on documentation - explaining what these things mean and why you can trust them - and usability.
Open questions
What sites should we prioritize integrating?
Should we integrate APIs directly or use the Metaforcast project?
Can we automatically link market categories or topics across sites?
How can we make the site more user-friendly or intuitive?
2. Physical League Rewards
The existing league system on Manifold leverages both mana rewards, which can be exchanged for charity contributions, and reputation rewards, shown on a user’s profile page. The top users on Manifold already have plenty of mana (by definition), and the reputation is nice but does not feature prominently on the site. In order to encourage more competition getting into the top leagues and to provide a more tangible reward, we can make brass trophies in the shape of Foldy, the site’s mascot and logo, and ship them to each person the first time they enter Master’s League.
Conceptually this is the simplest project on the list. I have already produced a handful of Foldies for myself and others and they turned out great. Each trophy is laser-cut sheet metal and all materials have a professional finish. I work with a specialty sheet metal shop that has a quick turnaround time and discounts for bulk orders. Sheet metal is ideal because it is easy to produce and ship while also being physically impressive.
You can see some Foldies from the previous batch here: https://imgur.com/a/4gBf6YX
What are this project's goals and how will you achieve them?
The first step of this project is to ship a physical reward to all users who make it into or retain their place in the December Master’s League. This gives the top participants a physical reward for their exceptional performance and rewards their participation. My steps to accomplish this are:
Potentially revise the Foldy design to make it more distinctive
Order a test run with varying thicknesses and sizes
Test that the selected version fits well in the standard flat rate envelope and does not bend during shipping
Contact all members of the December Master’s League and collect addresses if they are interested
Order the trophies and shipping materials
Package the trophies and ship to the winners
I am focusing on fewer, larger rewards due to the high upfront cost of shipping. Larger, higher-quality rewards will be more cost effective and be more likely to be displayed by the recipient.
I believe the best version of this project allows us to continue shipping trophies to all users who earn a spot in Master’s League, even as time goes on. If this month’s project succeeds I will approach Manifold with a partnership to continue this each month indefinitely. If we have extra stock I would also like to send a trophy to members who earned a spot in Master’s in the past.
I don’t plan on making the bronze trophies available for sale to keep them exclusive to the Master’s League. However, I could make other versions available for purchase.
How will this funding be used?
There are 62 users in the November Master’s League. Assuming there will be a similar amount of people in the December league, and assuming that 90% of them want a trophy, the material cost will be between $500-$1000. The main variables here are the size of the trophy and the thickness of the metal. In order to make sure we get the best value, I will order a prototype run with a few variants and let users comment on which they would prefer.
I plan to ship via the USPS Priority Mail flat rate small envelope, which costs $8 per domestic ship. International is much more expensive, running $29 per envelope. Costs will be much lower if we decide to exclude international users, but I would like to include them if possible. Assuming ~20% of users are international, this brings the average shipping costs to around $650-700 total.
What are the most likely causes and outcomes if this project fails? (premortem)
The intended effect of this project is to incentivize users to progress in leagues and reward users who have demonstrated their performance. If the users are not interested in physical rewards at all, this will not be an incentive. If they do not like the Foldy design itself, or find it too corny, it may not be a useful reward. The best outcome is that their trophy is displayed prominently and reminds them about their success on Manifold.
If I cannot raise enough funding to cover a significant portion the initial material and shipping costs, I will not attempt this project.
Open questions
What changes should we make to the Foldy design, if any, to make it more appealing as a trophy?
Is there a cheaper way to ship to international users?
3. Loan Exchange
Many users rely on interpersonal loans in order to increase leverage on specific markets, while users with large balances can lend to others at a specific interest rate to passively supplement their normal trading profit. Loans are a healthy part of the Manifold economy but currently have a few challenges:
Borrowers have no standard place to look for loans
Borrowers don’t know the standard loan rates
Lenders have to manage their investments manually
Lenders have to vet each potential borrower
Lenders take all risk from defaulted loans
What are this project's goals and how will you achieve them?
The primary goal of this project is to create a centralized loan exchange website for users to offer or accept loans with specific criteria.
A lender can create listings offering X mana at Y% interest rate for up to Z weeks.
A borrower can search for loans with the criteria they choose.
A borrower can easily place an “order” on the site.
The borrower can see the exact amount they will receive and the total principal + interest they will owe after the specified timeframe.
The borrower can optionally offer a message with a description of what they plan to use the loan for.
The lender can approve or deny the loan.
They will be able to see the borrower’s account age, balance, positions, and other outstanding loans.
The loan principal is disbursed from the lender to the borrower.
Each user can see outstanding loans and any overdue items on a central dashboard.
The loan principal and interest are returned from the borrower to the lender.
Option A: The borrower returns the mana to the lender directly. They will be notified from the site when the payment is due.
Option B: The mana is automatically sent from the borrower’s account with an API key. The borrower and lender are optionally notified.
This sort of site would resolve items #1-3 in the problem statement above. Borrowers would be able to find dozens of loan offers at competitive rates and easily take them. Lenders can have a centralized dashboard where they can manage outstanding loans and keep an eye on their borrowers’ balances.
Unfortunately, this still leaves the issue of lender risk. Users can apply for huge loans without understanding the risk involved, or use it to bet on a market that does not go their way. We have a few ideas for an advanced version of this site that would alleviate this risk.
Market Collateral: Borrowers have to provide an API key in order to take loans, and a certain amount of collateral is automatically put into a holding market which is closed when the loan is made. If the borrower defaults, the collateral is automatically taken instead.
Site-Backed Security: Borrowers are evaluated based on a number of factors, including account age, past loan repayment, and outstanding loans. When a lender is given the prompt to approve or deny the loan, they will also be shown a security value (up to $50,000 per loan). If the user defaults, the lender can request reimbursement up to the established security value to be paid by the exchange.
Site-Owned Funds: The exchange fully owns and operates its own fund which lenders can contribute to and borrowers can borrow from. Lenders get a lower, but guaranteed, interest rate can can withdraw their money at any point. Borrowers can be automatically approved for loans up to a certain amount. If borrowers default, they are blacklisted from the fund.
How will this funding be used?
Some amount of funds will likely be used to purchase mana to cover defaulted loans in all but the simplest version of this site. We would take steps to minimize risk but part of the purpose of the exchange is to take risk away from lenders so there will always be some cost involved.
What are the most likely causes and outcomes if this project fails? (premortem)
This project is more susceptible than most to failure because it relies partly on a network effect. It would be, to my knowledge, the first central loan exchange for Manifold, but that does not guarantee success. If a competitor were to offer a similar solution, it could split the userbase and make the site less useful. It could also be that demand for loans are simply not high enough to warrant a centralized exchange.
Additionally, there is inherent risk if we assume responsibility for loan defaulting. User fraud could overrun the budget and we would either have to invest additional money, default to the lenders, or shut down operations. We will have to monitor the costs associated with guaranteeing any loans and user behavior affected by it.
Open questions
What strategy should we use to reduce risk to lenders? Market collateral, site-backed security, or site-owned funds, or a combination? Any other ideas?
If we algorithmically evaluate borrowers’ credit, what factors should we consider and how should we weight them?
4. Market Stories
Every market tells a story, if you know how to read it. Unfortunately the probability graphs on Manifold are completely separate from both the bets and comments so it can be quite difficult to piece together what actually happened a lot of the time.
What are this project's goals and how will you achieve them?
This project would take be a browser client-side app that gets all of the data for a market and renders a dynamic scrolling page that highlights important events on top of the graph. It would allow users to understand what happened in one interesting and detailed summary. Users would also be able to zoom and expand the chart in order to investigate certain events.
The top of the page would have the market title and some basic information. The basic layout of the page would have the market graph rendered vertically so the user scrolls down “through” the market.
Long periods of inactivity would be collapsed and time markers would show the date and time at specific points.
Large bets or orders would call out the user who made it and any comments made around that time.
Comments with several likes will also be shown or summarized. Other comments will have a “pip” that can be clicked to expand the comment.
Market elasticity would be shown as offset lines to show an approximate “confidence interval”. Large limit orders would also be shown on the graph for the duration of the order.
The bottom of the page would show the market resolution and top winners/losers on each site.
Inspiration here includes the Soundcloud interface, which shows comments across the length of the track, or collaborative annotation tools that show comments in the margins like PaperHive.
How will this funding be used?
Aside from labor costs, I may decide to integrate some sort of LLM for summarization. I may also decide to purchase a new domain or cloud hosting for the site, depending on that traffic I receive.
What are the most likely causes and outcomes if this project fails? (premortem)
The main failure state for this project is simply that it may not hold users’ attention or make an impression. I assert in the tagline that every market is a story but not that every story is interesting. If users scroll through a few markets but don’t learn anything, then they probably won’t come back to the site for something like LK-99.
Another potential problem is over-optimizing for desktop or mobile usage. Display requirements for mobile will be very different from desktop, especially with regards to text and comments. If mobile users cannot see all of the same information, they may assume it does not exist.
Open questions
5. Brag Board
Manifold users put tons of time and effort into their trades in an effort to become better forecasters, earn money for charity, and win bragging rights within the community. The profile page has a lot of good information, but it’s not well-suited for quick social sharing. We can create a page that shows all of the things a user would brag about in a simple stat card that’s easy to share and understand.
What are this project's goals and how will you achieve them?
The primary goal of this project is an embed card, dynamically generated and easily embedded on Twitter/X, Discord, or Manifold itself. It is geared towards people who don’t necessarily know much about prediction markets but are able to intelligently interpret dense information. It would include stats like:
As a secondary goal, we can also make a larger dashboard/landing page with more detailed information. Suggestions on what to include here are welcome.
Inspiration for this idea includes Statbot’s profile cards, which show a similar about of information at a glance based on a user’s Discord history (https://statbot.net/img/landing-bot.webp).
How will this funding be used?
The only direct cost to this project would be my time, and potentially a new domain.
What are the most likely causes and outcomes if this project fails? (premortem)
The most likely failure state here is just a lack of adoption. If people are not interested in cards like this, then it will have minimal impact for Manifold as a whole. Additionally, Manifold will continue to refine the user profile page to make it more user-friendly and may obviate the need for a project such as this.
Open questions
What items are most important to have on the card? What should be emphasized?
Should we emphasize the most flattering stats? Should we hide items that are not particularly flattering? i.e. a user has not donated to charity or is in bronze league?
6. Notification Bridge
There are a lot of important things going on in Manifold, and it’s hard to keep track of everything. This is especially difficult if you do not use the app and only see events when you log in to the website or your email. Bringing notifications to users in their chat app of choice allows them to stay on top of market events and log back in when something interesting happens.
What are this project's goals and how will you achieve them?
We can keep people in the loop by implementing a new Manifold API endpoint that allows authorized users to read their own notifications and relay them to other services with various message APIs. Proposed services would be:
Discord, via a hosted bot or webhook
Slack, via a hosted bot or webhook
Telegram, via a hosted bot
Matrix, via a hosted bot
SMS, via Twilio
Alternatively, we can leverage an existing notification engine like Apprise.
Additional features could include notifications not currently supported by the site, such as drastic market changes.
How will this funding be used?
Aside from development time, the main direct costs here would be SMS message sending and possibly a new domain and cloud hosting.
What are the most likely causes and outcomes if this project fails? (premortem)
Many users already have the mobile app installed and so already get notifications on their phone. Additionally, the site has pretty good granularity for notifications between email, web, and mobile. This means there is a high chance that most users’ needs are already serviced by the existing notifications and do not need an external notification provider.
Open questions
Who is on your team and what's your track record on similar projects?
I expect to be the only person working on these projects directly. I would gladly accept assistance from other users in the form of pull requests, comments or suggestions. You can see my past projects at https://github.com/wasabipesto.
In the past I have received mana bounties from the Manifold dev team for the following projects:
7/13/22: for the initial version of the manifold stats page
6/14/22: for a later version of the manifold stats page
8/13/22: for a surprise gift of laser-cut Foldies for the dev team
8/17/22: for animated market yield graphs, featured in the manifold newsletter
7/17/23: for the initial version of calibration.city
My typical workflow is to focus on a single project at a time. I generally prefer to implement an MVP first and add features gradually based on feedback. This allows me to pivot quickly if something is not working and not waste effort on unnecessary features. In order to make these projects the best they can be, I will be seeking general user feedback more proactively than normal and providing investor updates regularly.
I have a full-time job but I enjoy working on projects like these in my spare time. I am not typically paid for these hobby projects so I just work on whatever interests me at the moment. Money from this grant would compensate me for my time and motivate me to work on the selected project when I would otherwise be unproductive. Other than that, a few projects have direct costs I have listed.
What other funding are you or your project getting?
I do not expect that I will receive any other direct funding for these projects.
