Austin avatar

Austin Chen

about 7 hours ago

Hi Stephen, thanks for submitting this project! Bio funding is very much outside my personal area of expertise, but I'll ask around and see if anyone in the field might be willing to lend their eyes on reviewing this.

To set expectations: we haven't issued regrantor budgets for 2024 yet, as we're still fundraising ourselves. It's a shame that this proposal missed the recent ACX Grants round, as it would have been a great fit - but with the upcoming ACX Grants Impact Certs side, there may be an influx of bio-curious investors/donors interested in this.

Also, I really enjoyed the bits of humor in your proposal - as someone who's fallen backwards into reading lots of these things, it's so nice when a proposal is a delight to read on its own.

🍉

nikki

about 22 hours ago

Don't have much time to work on this sadly. Still need to figure out the Autofold automation library.

In the far far future, I would like to create a no-code bot interface to help people like Zvi. Although the AI stuff seems a bit too complex.

cc6 avatar

cc6

1 day ago

Progress update

What progress have you made since your last update?

I've managed to recruit over 50 new users to Manifold, and the College Chance Me group has a handful of markets for people to bet on.

What are your next steps?


Is there anything others could help you with?


NcyRocks avatar

N.C. Young

1 day ago

Progress update

What progress have you made since your last update?

None, unfortunately.

What are your next steps?

I'm making some progress towards open-sourcing my bot, and still plan to post a writeup about the Fire-the-CEO markets, but that might take a backseat to my other priorities.

Is there anything others could help you with?

I'd appreciate hearing from any bot owners about how they run their bot - there are at least 3 different open-source Manifold APIs written in Python, so if anyone's tried more than one I'd like to know how they compare.

Kunvar avatar

Kunvar Thaman

1 day ago

I think Michael's podcast (and sometimes "not a podcast") episodes are fantastic. The Inside view (Michael's podcast), Dwarkesh's podcast, and the MLST (Machine learning street talk) are my go-to AI-related podcasts and the selection of guests that Michael brings is impressive, with a good focus on AI-safety researchers.

I've personally listened to a lot of his episodes, almost all released in 2022 and 2023, and I've recommended a fair amount of them to others - they're quite well done (such as the recent episode with Evan Hubinger - really good).

I don't think it's the production quality that's the barrier for youtube views which @Austin points out below, but more to do with marketing, and I'd recommend reaching out to more established people in the podcasting world for figuring that out.

I'm going to donate a token $500 and hope others can help fund Michael more, the work he's doing is great and quite impactful (especially for people in the early stages of an AI safety career).

Conflux avatar

Jacob Cohen

2 days ago

I wrote a politics article for the Manifold newsletter! It received 17 of those heart things on Substack, and multiple people posted comments saying they appreciated it. https://news.manifold.markets/p/how-inevitable-is-bidens-nomination

I think Manifold is doing good work with Manifold Politics, and does have a path to gaining the FiveThirtyEight demographic. We, too, have a lot of data-driven political analysis of a lot of different subquestions. I think supplementing this with politics coverage is somewhat high-value in terms of broadening Manifold's platform; I would be interested in writing more articles as things in the news happen - or maybe during the summer in some kind of intern position. Could be a growth vector for Manifold if one of these gets popular or something. In terms of value to the world...it definitely makes politics nerds happier, idk if it does more than that, if I'm being honest. I think I would try to keep it to predictions and not venture into the realm of potentially-cancellable takes.

wasabipesto avatar

wasabipesto

3 days ago

Progress update

What progress have you made since your last update?

Our biggest updates to Calibration City this month have been the addition of a few new tabs!

🤔 First up we have the introduction, a sorely-needed explainer for this whole “calibration” thing. This is mainly formatted as a dialog-based blog post where we explain the basics of quantified predictions, calibration, accuracy, prediction markets, and the site’s purpose. I hope it is able to bring the ideas behind prediction markets to a few new eyes, directly alongside the statistics to (potentially) back it up.

🥇 Up next we have the accuracy tab, with a set of graphs I teased last month. Ever wondered if longer markets are more accurate? What about markets with more traders? Now you can compare all markets across every supported platform against any attribute we measure - plus you can combine it with any of the other standard filters! Do sports markets on a specific platform get more accurate the closer to the end of the market? Are markets that resolve yes more accurate than those that resolve no? Experiment to your heart’s content!

🔍 Wait, what markets are in that bin of sports markets on that one platform that resolved no? Well to find out you can go search on that site… or you can use the new list page! List, sort, filter, and browse the markets to your heart’s content. (The API endpoint that powers this page can also be used to download all of the markets in my database, in case you’re interested in double-checking my math or rolling your own 😉)

👨‍🎓 Interested in learning more about the site? Head on over to the FAQ tab! It’ll answer your question… as long as your question is “where can I learn more about scoring prediction markets” or “give me more nitty-gritty implementation details”. I’ve been trying to include responses to actual frequently-asked questions on the relevant page with better wording or hover-text, so nothing has actually made it to this page yet 😅 Feel free to give suggestions for what I should add, though!

What are your next steps?

🏆 More scoring options! We can already calculate and show the Brier score for every market, but there are a lot more scoring methods! I plan to add logarithmic scores, spherical scores, and more.

🖇️ Even more platforms! After adding Polymarket (and then working with their dev team to get even more information) I paused adding new platforms to the site in order to get the user experience the way I wanted. Now, I think we’re in a great state and we can get moving on integrating even more data!

✅ Corpus of questions! The biggest issue with the site as it stands now is that you’re comparing apples to oranges - not all markets are comparably difficult! I don’t want to punish sites for catering their questions to their communities, and I don’t want to reward sites that might attempt to “game” statistics like calibration or accuracy. I want to build off of existing datasets, leverage tools to replicate questions across platforms, and build a large enough corpus of questions that users can confidently see how accurate different platforms were for key questions or in aggregate.

Is there anything others could help you with?

Absolutely: I’m not a technical writer, so please review what I’ve written! The introduction dialog is a quick summary of how I understand calibration and accuracy, but it could be wrong or misleading in important ways. That’s why I’ve set up this bounty for users to submit their feedback on the site, especially anything that was confusing, unintuitive, or incorrect.

I’m also interested if there is any prior art to grouping a corpus of questions like we will need to compare accuracy across platforms. Aside from doing it manually, we could leverage the markets that MirrorBot has replicated or use LLMs to determine when a market is “close enough” to be identical, but each of those have drawbacks. Since this is going to be a big feature with a high future maintenance burden, I would be happy to hear any ideas that may make it simpler!

As before, I’m always looking for ways to make the site better. If you have any ideas for things that would bring you to the site more often or features you would love to see, I’m all ears.

❤️ Thanks!

saulmunn avatar

Saul Munn

3 days ago

this looks like a really interesting grant — there are a few things i find exciting, and a few things on which i'm confused. below are quick impressions from spending 20-30 minutes, so i'm probably wrong somewhere. i'd appreciate anyone (including project creators') taking the time to correct me!

exciting aspects:

  • translating this sort of content into other languages

  • outreach into areas — like thailand — that have seen less of this sort of outreach

positive signals:

  • track record is pretty strong — looks like the team has already shipped translations of similar EA content before

  • it looks like there's pretty limited downside risk

  • relevant connections to academia

remaining questions:

  • from a very seems like (potentially!) LLMs could make the process of translation a lot easier, cheaper, faster, etc — do you plan to use them? if not, why not?

  • i'm very confused about the budgeting. why does this costs $150k at minimum, and $250k ideally? i could imagine that this is being used well, but $150k-$250k is a lot of money, and it'd be good to better understand how you plan on using this money.

    • "website development" could look pretty close to just... copying utilitarianism.net, and making a few tweaks. domain & hosting can't be more than a few thousand, right? what am i missing here? why would this cost $30k-$50k?

    • why does "translation of essays & video content production" cost $52.5k-$87.5k? how many person-hours would it take? what is their hourly rate?

    • additionally, are there some parts that are particularly expensive? could you get 80% of the translations done on 20% of the funding? (e.g., maybe translating the videos takes up the majority of the costs — perhaps you could scrap the video translations and instead only go for essay translations?)

  • i'm a bit confused on your plans for dissemination, which seems fairly important for most parts of your theory of change — once you've created a great website with translated content, how do you plan to get that into the hands of the relevant thai audience? i'd love some more specifics here: do you have workshops/conferences/events planned? are you going to run ads? reach out to professors? how is this happening, concretely?

again — these are quick thoughts & impressions from spending 20-30 minutes reading through. interested to hear others' thoughts too!

hardeep_gambhir avatar
Progress update

What progress have you made since your last update?

Since I received funding, I have travelled in the United States a decent bit, mainly in the Bay Area and have gotten connected to the startup ecosystem and effective altruists here. The most prominent progress I have made is having kicked off The Residency's First and Second Cohorts. Here is a link to the people in The Residency: https://x.com/_theresidency/status/1746014926307525075?s=46

What are your next steps?

I will be going back to school in September. In the next few months, I will be working on making The Residency happen in India soon. I am also continuously working on my side projects and this fund has helped me tremendously on this gap year from university.

Is there anything others could help you with?

Thank you so much for approving this grant Austin and Isaak for regranting it to me. Hope to see you around!

case avatar

Case Sandberg

3 days ago

Manifold pull requests:

  • Added Hovercards everywhere in the UI

  • Added a color picker for changing answer colors

  • Show limit orders on answers, add limit order volumn below answers

  • Added select to positions for answer filtering (vs overflowing chips)

  • ...other small fixes requested by the community

case avatar

Case Sandberg

3 days ago

@Eliza I don't believe I can since it never hit the original cap or something?

hijohnnylin avatar

Johnny Lin

4 days ago

@hijohnnylin Another update from two weeks ago - Neuronpedia collaborated with Joseph Bloom (MATS) to upload his Sparse Autoencoders, which resulted in ~295,000 dashboards. It is also now fully pivoting to focusing on being a tool for accelerating the work of interpretability researchers and sunsetting previous gamification goals.

wasabipesto avatar

wasabipesto

4 days ago

@Austin Thanks for the information!

Austin avatar

Austin Chen

4 days ago

@wasabipesto for some context, SirCryptomind was asking whether Manifold could hire him for his moderation work; while we didn't want to bring on an ongoing, fulltime paid position for this at the moment, I encouraged him to submit an entry for retroactive funding for his mod work as part of the Community Fund. The community fund hasn't paid out our third round yet and I expect SirCryptomind's work to fall within scope for this.

Austin avatar

Austin Chen

4 days ago

Funded this with $2.5k and approving this! This falls within the category of "encourage interesting scientific experiments" and is low-budget, so it's a cheap bet to see what this team can accomplish. I'm glad they are releasing their work as open source too (though would love to see a link somewhere!)

wasabipesto avatar

wasabipesto

4 days ago

I'm a bit confused about this proposal. Are you looking for retroactive funding as compensation for past site moderation? To my knowledge the community fund is not continuing so I'm not sure how you would expect to gain investors for future work. Are you claiming that you'll stop the vast majority of your moderation if this is not filled? I think some context would be helpful.

42irrationalist avatar
Progress update

An additional small update

  1. I checked in with participants. For two matches people went on one date, so things are progressing.

  2. I didn't mention this explicitly, but we recorded the entire two hours of the show. We will be releasing only a bunch of short snippets, but we have a lot of content to pick from. View counts from our previous show can be used to estimate future view counts — check out my previous update for the numbers.

Testimonials

sclmlw avatar

Mark Webb

8 days ago

@prudentj This is the current place to go for updates. I can't promise any news in the offing, but if you want to be more directly involved let me know.

sclmlw avatar

Mark Webb

8 days ago

@saulmunn This is a good concern, and one that I've been pondering for some time. Part of why I wanted to start purchases from current landowners, instead of buying land on the open market, is to specifically avoid driving up property values. However, as you suggest if people know there's someone going around buying all the land this will encourage holdouts - which could defeat the whole redistribution project.

One mitigating factor is that I don't think we need to buy 100% of the land to help propel a country down the development path. I'm not sure yet what percent of the land we'll need, but a significant percentage of the tenant farmers will likely be able to transition toward industrial or other non-farm work without us having to buy the land directly.

A concern with this is that landowners who hold out too long may actually have the opposite problem once we no longer need to use land transfers to spur economic growth: that having missed their chance to sell for land transfers, farmland values may not equal what they could have gotten from us. Hopefully, economic growth will make that land valuable for other reasons by that point, and they will not feel that the project has harmed their financial interests.

A big open question is what percent of the land would need to be part of the land transfer to drive self-sufficient economic growth. If we knew this going in, we could start by redistributing that percentage and avoiding the market distortion from the outset ... in theory. In practice, I imagine it'll be a lot more messy and we won't know how much so until we encounter these problems in the real world.

🐼

Jacob Prudent

8 days ago

This intuitively makes sense to me, and I am curious how your research will pan out. Is there a social media account or website I can follow to keep track of this project? Also are you involved in any Georgist organizations?

LucyFarnik avatar

Lucy Farnik

9 days ago

Progress update

What progress have you made since your last update?

Interpreting "goals" turned out to be out of reach, so I did what I said in the description and pivoted towards studying easier LLM phenomena which build towards being able to interpret the hard things. I spent some time researching how grammatical structures are represented, and have since pivoted towards trying to understand how "intermediate variables" are represented and passed between layers. My current high-level direction is basically "break the big black box down into smaller black boxes, and monitor their communication".

What are your next steps?

I'm currently approaching "inter-layer interpretability" with SAE-based circuit-style analysis. I basically want to figure out whether it is possible to do IOI-style things but with SAE features at different layers as the unit of ablation. I'm also looking into how to do SAE-based ablation well (to make results less noisy). I'm researching these questions in MATS under Neel Nanda.

Is there anything others could help you with?

If anyone reading this is interested in the things I described above, I could use collaborators! In particular, if you're somewhat new to alignment and would be interested in a setup where I throw a concrete specification for an experiment at you and you spend an afternoon coding it up, I'd be interested in talking to you.