Manifund foxManifund
Home
Login
About
People
Categories
Newsletter
HomeAboutPeopleCategoriesLoginCreate
Austin avatarAustin avatar
Austin Chen

@Austin

regrantor

I work on Manifund!

https://manifold.markets/Austin

Donate

This is a donation to this user's regranting budget, which is not withdrawable.

Sign in to donate
$6,059.74total balance
$476.2charity balance
$833.54cash balance

$4,750 in pending offers

Outgoing donations

Emergency travel funding to attend EA Global: New York 2026
$250
PENDING
Cascade
$2000
PENDING
First paper on governing automated OSINT and scoping an institutional home
$1000
PENDING
Building Tooling to Map how Ideas Spread
$2500
PENDING
Swarm incidents tracking platform
$500
about 3 hours ago
Living support for Pip Foweraker
$1000
about 24 hours ago
Living support for Pip Foweraker
$500
about 24 hours ago
Living support for Pip Foweraker
$1000
about 24 hours ago
The New Critic Longform Reporting Fund
$20000
17 days ago
Philosium Tokyo 9/19/2026
$100
24 days ago
How Then Shall We Live? Meaning in the Age of AI
$5000
about 2 months ago
Scaling adoption of synthesis screening outside the US and EU
$5000
about 2 months ago
Metagame 2026
$2500
about 2 months ago
CryptoMind: Verifiable Safety for Financial AI Agents
$500
about 2 months ago
Fund an AIS Community/Field-Builder: Effective Altruism SF & other initiatives
$5000
about 2 months ago
Fund an AIS Community/Field-Builder: Effective Altruism SF & other initiatives
$5000
about 2 months ago
Supplementing Summer Program for Applied Rationality and Cognition 2026 (SPARC)
$5000
about 2 months ago
Pax Machina
$5000
2 months ago
Separatrix
$5000
2 months ago
AI Safety Nepal
$1000
3 months ago
PayBench: A Benchmark for Unsafe Commercial Autonomy
$1000
3 months ago
Falcon Fund
$1000
3 months ago
$3000
3 months ago
Integral Altruism
$3000
4 months ago
LLMMatrix: Synthetic World Benchmark for Contamination Free LLM Forcasting
$750
4 months ago
vibecamp
$1000
4 months ago
Ending Male Chick Culling in Asia-Pacific ll
$400
5 months ago
nohotdog.love: fund cool nature & animal projects with others
$500
5 months ago
TELL: The AI Detector You *Don't* Have To Trust
$500
6 months ago
Safe AI Germany (SAIGE)
$5000
7 months ago
Agent Island: An Environment for Interagent Cooperation and Conflict
$2000
8 months ago
aCFAR 2025/6 Fundraiser
$2500
8 months ago
Civitech Incubator
$10000
8 months ago
Isolating CBRN Knowledge in LLMs for Safety - Phase 2 (Research)
$2500
10 months ago
Improve eligibility for organ donation
$5000
10 months ago
Metaculus' First Animal-Focused Forecasting Tournament
$1000
11 months ago
Fund thebes' model tinkering
$2500
11 months ago
Normie-friendly prediction market interfaces
$2000
11 months ago
Measuring the Epistemic Cost-Effectiveness of AI Safety Content
$1000
11 months ago
Development of a Cautionary Tale Feature Film about Gradual Disempowerment
$2500
about 1 year ago
Grow An AI Safety Tiktok Channel To Reach Ten Million People
$2000
about 1 year ago
TransformerLens - Bridge Funding
$2000
about 1 year ago
Systems that "give a damn"
$1000
about 1 year ago
Asterisk AI Blogging Fellowship
$10000
over 1 year ago
AI Digest
$10000
over 1 year ago
Calibration City
$5000
over 1 year ago
Orexin Pilot Experiment for Reducing Sleep Need
$500
over 1 year ago
Split Personality Training
$1000
over 1 year ago
Coursetexts
$2500
over 1 year ago
AI forecasting and policy research by the AI 2027 team
$5000
over 1 year ago
Coordinal Research: Accelerating the research of safely deploying AI systems.
$5000
over 1 year ago
Out of This Box: AI Safety Musical
$3000
over 1 year ago
Animal Advocacy Innovation Hub in the Bay Area
$500
over 1 year ago
Animal Advocacy Innovation Hub in the Bay Area
$500
over 1 year ago
Attention-Guided-RL for Human-Like LMs
$3000
over 1 year ago
11th edition of AI Safety Camp
$1000
over 1 year ago
SafePlanBench: evaluating a Guaranteed Safe AI Approach for LLM-based Agents
$500
over 1 year ago
Rabbitholeathon Event Food
$500
almost 2 years ago
Metaculus x Givewell Forecasting Tournament
$1000
almost 2 years ago
1200€ to move in SF for an international high-level event and meetings
$1300
almost 2 years ago
Elizabeth and Timothy Podcast on values in Effective Altruism (Funded)
$1300
almost 2 years ago
Finishing The SB-1047 Documentary
$10000
almost 2 years ago
Fund Sentinel for Q1-2025
$1000
almost 2 years ago
Developing a Course on AI x-risk
$5000
almost 2 years ago
Giving What We Can
$50
about 2 years ago
Play money prediction markets
$100
about 2 years ago
Fatebook and Quantified Intuitions
$100
about 2 years ago
Fatebook and Quantified Intuitions
$200
about 2 years ago
Covid Work By Elizabeth VN/Aceso Under Glass
$500
about 2 years ago
Covid Work By Elizabeth VN/Aceso Under Glass
$100
about 2 years ago
CEEALAR (EA Hotel) Needs a New Roof
$50
about 2 years ago
CEEALAR (EA Hotel) Needs a New Roof
$1000
about 2 years ago
Adjacent News
$380
about 2 years ago
Calibration City
$100
about 2 years ago
Forecasting Meetup Network - Washington, DC pilot (4 meetups)
$500
about 2 years ago
Forecasting Meetup Network - Washington, DC pilot (4 meetups)
$50
about 2 years ago
Effective Altruism Meetup, Abuja, Nigeria
$10
about 2 years ago
BAIS (ex-AIS Hub Serbia) Office Space for (Frugal) AI Safety Researchers
$1100
about 2 years ago
AI, Animals, and Digital Minds 2024 Conference and Retreat
$2000
about 2 years ago
AI-Driven Market Alternatives for a post-AGI world
$5000
about 2 years ago
Design budget for rebuilding the dating site we all want back.
$400
over 2 years ago
Design budget for rebuilding the dating site we all want back.
$200
over 2 years ago
Lightcone Infrastructure
$5000
over 2 years ago
Support a thriving and talented community of Filipino EAs
$350
over 2 years ago
Making 52 AI Alignment Video Explainers and Podcasts
$500
over 2 years ago
EEG using a generalizable ML model + 32 channel PCB
$2500
over 2 years ago
Experiments to test EA / longtermist framings and branding
$5000
over 2 years ago
ACX Grants
$10000
almost 3 years ago
BAIS (ex-AIS Hub Serbia) Office Space for (Frugal) AI Safety Researchers
$1100
almost 3 years ago
Mapping neuroscience and mechanistic interpretability
$1200
almost 3 years ago
Estimating annual burden of airborne disease (last mile to MVP)
$3600
almost 3 years ago
Holly Elmore organizing people for a frontier AI moratorium
$2500
about 3 years ago
One semester living expenses for MIT-based researcher
$500
about 3 years ago
Neuronpedia - Open Interpretability Platform
$2500
about 3 years ago
VaccinateCA
$10000
about 3 years ago
Recreate the cavity-preventing GMO bacteria BCS3-L1 from precursor
$20000
about 3 years ago
Funding for Solar4Africa app development
$500
about 3 years ago
Reflective altruism
$2000
about 3 years ago

Investments

Manifold x College Admissionsbought $100 @ $250 valuation
London Manifold.love dating showsbought $200 @ $3K valuation
Manifold merch storebought $20 @ $667 valuation
Make large-scale analysis of Python code several orders of magnitude quickerbought $1K @ $1K valuation
Manufacture Manyfold Manifolders in the Maritime Metropolisbought $100 @ $1K valuation
Year one of AI Safety Tokyobought $545 @ $60K valuation
Mirrorbotbought $50 @ $1.67K valuation
Artificial General Intelligence (AGI) timelines ignore the social factor at their perilbought $100 @ $2K valuation
Run a public online Turing Test with a variety of models and promptsbought $250 @ $2K valuation
Invest in the Conflux Manifold Media Empire(??)bought $15 @ $250 valuation

Comments

Living support for Pip Foweraker
Austin avatar

Austin Chen

3 days ago

@PipFoweraker appreciate the quick response; best of luck with recovery, and just ping me if you'd like to arrange that chat (but only if it seems helpful).

I'm not sure what the $10-12k is sized to, or what you and AISTOF worked out, but fwiw I would encourage you to draw a standard ~$100k+/y salary and backdate that to whenever you started working on Beacon; not legal advice obviously, but I expect this would be fully fine by the law, your board and AISTOF.

(And FWIW I've considered a similar question on an Exec Assistant or Personal Assistant; I did have an exec assistant for a bit, contracted through Manifund for Manifund & Mox work, but I also feel a similar need for personal non-Manifund affairs, and don't have a general solution either. Maybe, I would initially recommend having Beacon pay for a part-time Beacon EA, and/or having the board approve a salary bump so you could feel financially comfortable to hire a PA and free up your own focus.)

Living support for Pip Foweraker
Austin avatar

Austin Chen

3 days ago

Hi Pip! Sorry to hear about this. I don't think I know you super well but we've definitely crossed paths professionally, and I want to echo that Ashgro and now the fledgling Beacon seem like they're trying to address an urgent need in the EA & AI safety space, of good fiscal sponsorship.

I'm making a small initial donation as a sign of support. In the meantime, some questions (and apologies if these seem harsh):

  1. Could you just draw a salary from Beacon, out of the $250k AISTOF provided? Or is that $250k earmarked for something else?

  2. What is the state of Beacon's fiscal sponsorship program? Is it still held up on 501c3 approval, or anything else? (Increasingly, I feel like this is a thing that many orgs and individuals need; historically Manifund has done a lot of Model C fiscal sponsorship, but we're excited for strong options that we can refer others to for which are higher-touch, cheaper, or reputationally different)

  3. In addition to funding, what are the ways that you and/or Beacon most need help? (Some examples I could imagine: connections to individuals/orgs who need fiscal sponsorship; endorsements of your work; leads for people to work with on Beacon, ie cofounders or employees)

I encourage you to continue writing your thoughts in public, but am also happy to hop on a private call sometime next week if that helps; best wishes!

Swarm incidents tracking platform
Austin avatar

Austin Chen

3 days ago

I'm making a small offer as well, because I think the problem area is important, given recent events -- I'd love to see this expanded beyond just the OpenAI incidents too! Also, I think Elvis might have the technical and product skills execute well on this; I don't know Elvis super well, but we have met several times through Carolina (who currently works with me at Mox).

FWIW, @ollive I'd recommend that you withdraw your funding offer here. I understand it's a small amount mostly meant as a show of support, and Manifund is open to donors to fund others they know personally, given proper disclosure. But especially in the case of partners we would expect to apply significantly more scrutiny/diligence; I think it's just not worth the hassle here. Instead, I'd just recommend writing more about why you're excited for this project specifically, and then forwarding the Manifund page to others who might like to support this!

Mox: 2026 Fundraiser
Austin avatar

Austin Chen

4 days ago

@tessafyi thanks for hosting an awesome event! I sadly didn't make it, but heard about it through Joey. Hope to have many more biosec lightning talks at Mox!

Emergency travel funding to attend EA Global: New York 2026
Austin avatar

Austin Chen

4 days ago

Hey! Making a partial donation, to encourage you to also seek funding from other funders too.

Also consider https://bluedot.org/grants/rapid !

AI Cyber Forge: Neon Protocol
Austin avatar

Austin Chen

4 days ago

@hessy121212 hm thanks for the feedback, we want to make sure Pangram is returning proper results. I don't suppose you have any way of verifying that you did type this proposal yourself (eg a Google Docs edit history screenshot?)

Mycelium: AI alignment for nonhuman welfare
Austin avatar

Austin Chen

6 days ago

Hi @allen! We just announced that the Falcon Fund has raised $500k+, and is looking to grant to projects similar to yours -- would you be interested in submitting it for consideration? (If so we can just re-tag your project, no need create a new proposal.)

also cc @MarcusAbramovitch

aCFAR 2025/6 Fundraiser
Austin avatar

Austin Chen

19 days ago

@aCFAR Not super related but wanted to say, we rented out CFAR's Bodega Bay Venue for a midpoint retreat for the Surplus incubator. It was fantastic; I'd highly encourage other folks considering doing eg a weekend retreat or workshop, to look into renting their space: https://www.rationality.org/venue

More related: many of the founders expressed an interest in participating in a future CFAR workshop. Keep up the good work!

Seen & WithMe
Austin avatar

Austin Chen

24 days ago

@caroline hi, Austin here -- I'm making a public note that the above is a fake/joke account, not tied to Caroline Ellison. we'll likely rename your account and username if you don't do so yourself.

Philosium Tokyo 9/19/2026
Austin avatar

Austin Chen

24 days ago

Approving and making a small grant in support! Generally happy to fund EA events around the world.

(at Fumi's request, we lowered the min funding bar down from its initial $1.4k; Fumi asked, "we have more than enough since we downscaled with less people, but we still need money to hit our initial minimum budget to get our funds.")

Museum for AI Safety & Society
Austin avatar

Austin Chen

about 2 months ago

Let us know if this works and you're interested in expanding to other places; at Mox we'd love to have an exhibit like this! I think we're talking with the Misalignment Museum about hosting them, always happy to have more.

(I'd also love a "flouishing futures/AI utopia" popup/exhibit, in case anyone out there is working on stuff like that!)

Sentient Futures Project Incubator
Austin avatar

Austin Chen

about 2 months ago

Approving this; it's been great seeing Sentient Futures expand their programs and we're big believers that many more projects need to be incubated!

Prediction of Inoculation Prompt Side Effects
Austin avatar

Austin Chen

about 2 months ago

Happy to see this endorsed by Gavin, Marcus, and Anton through grantmaking.ai (original proposal here).

London Manifold.love dating shows
Austin avatar

Austin Chen

about 2 months ago

I just found out that one Manifund grantee met their serious longtime girlfriend of 2y through this dating show! Huge shoutout to @42irrationalist for organizing, and the various impact investors and retro funders for helping make this happen.

The New Critic Longform Reporting Fund
Austin avatar

Austin Chen

about 2 months ago

I first found out about the New Critic when on their article "p(doom)", which I found to be a good dive into the prediction markets scene, and extremely well written. I invited one of their journalists to come to Manifest, resulting in their article "Manifest Man".

Rufus then reached out asking about a larger grant to continue sponsoring longform pieces similar to those. I've learned much from longform; find quality longform writing on topics I care about to be rare; and believe that longform is extremely underprovided as a public good.

(fwiw, some other initiatives that Manifund has supported in this vein include both Inkhaven cohorts, the Asterisk blogging fellowship, and the Blog Revival Project.)

After some back and forth, and a call with the whole New Critic editorial team, I've decided to recommend a $20k grant, split into two chunks: $10k unrestricted for operations, and $10k to fund 3 longform pieces on topics of interest to Manifund (as Rufus outlines above). I'm excited to both support this team in their ambitions, and see whether this commission model can produce essays that we find to be exemplary.

CryptoMind: Verifiable Safety for Financial AI Agents
Austin avatar

Austin Chen

about 2 months ago

Hey! I think your background is interesting and your work demonstrates a good level of commitment. I'm not personally very interested in TON (or crypto, overall), though I do find the concept of "agents with money, safely" worth developing further, and it's plausible that crypto is a good testing ground for this. I'm making a very tiny grant offer as a show of support, more in support of you individually than this specific project idea.

Fund development of p(Doom)1
Austin avatar

Austin Chen

2 months ago

Making a small donation as a sign of support; I continue to be excited for games as a method of imparting interesting lessons (eg universal paperclips, maybe Manifold). Pip also has a good track record of supporting work in the xrisk space, through Ashgro and now https://beacongcr.org/

I wasn't actually able to demo the game myself -- I'm on Mac, Sequoia 15.5, and "right click => open" only gave me "cancel" and "move to trash" as options. (Vaguely I think that I've solved this problem before for other apps, but it was enough of a roadblock for me to give up). I'd encourage Pip to see if Godot or Claude can port at least a demo, to be directly playable on web/mobile.

I'd also worry that the game currently looks a little bit too AI-generated and/or chaotic, per the screenshots. I'm generally a fan of AI assistance, but specifically the gaming community has a strong immune reaction against AI content, and also in this case I'd worry there's just too much slop all around. Possibly pairing with a good visual/graphic designer might be quite helpful, since games especially seem to have high standards/stakes for visual appeal to be successful.

Using deals to find misalignment in current AI models
Austin avatar

Austin Chen

2 months ago

Excited for this - deals with AIs are an interesting space, a field that I'm quite interested in nurturing. Thanks to grantmaking.ai and Gavin for funding this!

Movement-building to make AGI go well
Austin avatar

Austin Chen

3 months ago

Approving in my capacity as Manifund admin.

(I'd note that I'm skeptical of several pieces of Rohan's proposal, but for a program like Leo's microgrants, our usual process is to defer the grant recommender's expertise, while Manifund screens for compliance with our charitable mission.)

How Then Shall We Live? Meaning in the Age of AI
Austin avatar

Austin Chen

3 months ago

Funding and approving this project! As a practicing Catholic and also someone who cares a lot about the trajectory of AI, I'm very excited to support this event.

Jessica has pulled together a fantastic cohort of people, many of whom I've long followed and have taught me much about the world. She's also generously volunteering her own time to make this happen. I'm quite sad that I'll be out of town for it -- but excited for the videography to come!

Concept-anchored representation engineering for alignment
Austin avatar

Austin Chen

3 months ago

@jesse_hoogland oh this is awesome, thank you for sharing this writeup!

(Do you know if Jueyan did end up funding this? Also curious, did you first discover on Manifund or someplace else?)

Machine God (documentary film)
Austin avatar

Austin Chen

4 months ago

I think this is a good list of initial interviewees, and I think Steve Hsu's involvement is a strong signal for the success of this documentary. These folks screened their trailer at Manifest & held a q&a (which I sadly missed).

One worry I'd have is that a talking heads documentary like this (or the SB 1047 documentary) may have limited widespread appeal, esp compared to something produced like AI in Context's AI 2027 video. I'd also worry that something like the Apocaloptimist (which I haven't yet seen) might have covered many of these same notes. But I think it's worth having many shots on goal, and overall am looking forward to this.

AI Safety Roster: a searchable directory of everyone in AI safety
Austin avatar

Austin Chen

4 months ago

@Ahmed briefly, I think this is going in an interesting direction, but I'm not as interested in a fancy world map thing; like Ronak says, a version that's just a plain table or db (and maybe with a straightforward API or sth) would be useful to answer questions I have, which are often of the shape "who might I reach out to hire for a role at Mox" or "who should I invite to speak at this conference".

I think being quite comprehensive is important -- like, knowing that all the relevant orgs and people are included would be quite helpful. Right now there's only 1k people on there and basically no people who I would think are relevant (eg the Manifund regrantors).

https://www.longtermwiki.com/ is a thing to look at for inspiration, perhaps.

Falcon Fund
Austin avatar

Austin Chen

4 months ago

Admin note, I've reopened this project with a lower minimum funding bar. I believe @MarcusAbramovitch is quite excited for at least one grant to make very soon!

vibecamp
Austin avatar

Austin Chen

4 months ago

I have never been to Vibecamp, but many of my friends make it an annual pilgrimage. I recently officiated a wedding where the couple met at Vibecamp; the organizers tell me something like 5 married couples have met there, a number I only hope Manifest may someday hit. Also, Brooke has been super helpful in the past (eg by serving as community point of contact for a past Manifest) and is generally an awesome person. I am very excited to support Vibecamp out of my budget for fuzzies!

Awareness-conditioned emergent misalignment
Austin avatar

Austin Chen

5 months ago

Happy to approve this; this kind of small fast technical grant falls in the sweet spot for AI safety regranting :)

nohotdog.love: fund cool nature & animal projects with others
Austin avatar

Austin Chen

5 months ago

I love the design of this site!

every donation platform should look like partiful

is a fantastic slogan, will consider it ;)

Falcon Fund
Austin avatar

Austin Chen

5 months ago

I'm also making a small personal donation as a sign of support. Marcus has repeatedly impressed me with his hard work, dedication and sacrifice towards doing good. And also with his prescience as a forecaster and investor!

Cause-prio-wise, I'm more into AI safety than animal welfare, but I think some hedging is appropriate, along moral parliament grounds. I'm also excited for AI welfare and rights, and expect some commonalities between that field and the Falcon Fund.

TELL: The AI Detector You *Don't* Have To Trust
Austin avatar

Austin Chen

6 months ago

@acmc awesome to see that you've already launched; it's a nice-looking website!

Feedback, I pasted in a snippet and it was extremely slow (>3min), only to hit some kind of error. Imo, getting it to work relatively fast is quite important, both for a demo and for real-world usage.

TELL: The AI Detector You *Don't* Have To Trust
Austin avatar

Austin Chen

6 months ago

@acmc you're welcome! I'm viewing this as a very cheap bet on someone outside of the standard EA/AI safety network, but otherwise with reasonable credentials, working in a problem area I think is interesting and important. For Manifund specifically, we're seeing a lot of AI generated project proposals, which is not disqualifying but can be a negative signal, and I'd love better, explainable detectors.

For fun, I put your app through Pangram, which flags your proposal as about half AI, haha: https://www.pangram.com/history/07a7f2de-1e42-4bd8-906c-8699f86061e6

Lyptus Research: Bridge Funding
Austin avatar

Austin Chen

6 months ago

Approving this project! I'm glad that @joel_bkr and others at METR are so excited for Sean's work, and from my perspective, this kind of active, expert-led, fast grantmaking is where the regranting mechanism shines.

I also appreciate Joel taking the time to lay out his considerations in making this grant, which are hopefully helpful for Sean and others in similar positions. (And @seanpetersau, insofar as you would be open to accepting more funding through this Manifund project, let me know; we're very happy to increase the funding goal on your behalf!)

AI Safety Workshops For 6th-8th Graders in India
Austin avatar

Austin Chen

6 months ago

@aashkapatel thanks for flagging, should be fixed now!

Emergent misalignment and memetic replication in multi-agent codebases
Austin avatar

Austin Chen

6 months ago

Approving this project! @RyanKidd I appreciate the particularly in-depth writeup, and also the fact that you referred this grant to @JueYan to get it funded; I'd love for more regrantors (and regular individuals) to be taking such prosocial actions in the AIS community.

I also personally think multi-agent dynamics are very interesting and would be excited to better understand that field; especially from more of an experimental/practical lens (like AI Village or Moltbook, rather than theory).

Collaboration Infrastructure for Safeguarding AI
Austin avatar

Austin Chen

6 months ago

Approving this grant -- I've been a fan of Ink & Switch's UX work, and it's great to see them pitch in on AI safety too!

As a note, we've agreed with Jueyan to structure this as a loan rather than a nondilutive grant, to be repaid 1x (no interest) in 24 months, or 2x in case of an acquisition. I continue to be excited for Manifund to sponsor loans and equity investments, in cases where grantees are for-profits and the size of the grant is relatively large.

Mox: 2026 Fundraiser
Austin avatar

Austin Chen

7 months ago

@Finn-Metz thanks, it's been great having you, Esben, and other Seldon folks and batchmates at Mox. Glad we were able to fill this role for you!

Mox: 2026 Fundraiser
Austin avatar

Austin Chen

7 months ago

@AlexGajewski thanks! I loved the Cafe Calculate series you ran while at Mox, which likewise brought in a stream of interesting people. I expect your new office is treating you well, but we'll always be proud to have hosted Nonsense Factory Pantograph's earliest days!

Mox: 2026 Fundraiser
Austin avatar

Austin Chen

7 months ago

@joel_bkr thank you so much!

(1) I'm very grateful for all the events you host! Especially for the live band karaoke, which brings such an energy and joy to the space. (And for completeness' sake, Joel also threw his birthday party last year at Mox, because his hundreds of friends would not fit in our house.)

(2) agreed on all counts!

  • The AI safety community should invest more in SF, where the AI labs/startups/funders are. This was the original thesis behind starting Mox, and it seems just as true now as a year ago.

  • I too wish Constellation would move to SF, and expect they would be the default safety nexus if they did move. We love our competitors! Unfortunately, from convos with Constellation people, no such move seems likely to happen soon (eg within the next 6mo), if ever.

  • I do feel like we're falling short on some obvious good things from having a hub in SF, AIS happy hours being one such example. Some of this is because, at our current level of funding, we spend a lot of our time chasing random opportunities to pay our bills (eg leasing offices to tech startups). Some is from a lack of personnel; I would love to hire someone who deeply knows the SF scene around the AI labs & safety, but I don't have any such candidate lined up (please lmk if you have someone in mind). Some are just unforced errors, and we'll aim to do better in 2026!

Safe AI Germany (SAIGE)
Austin avatar

Austin Chen

7 months ago

Approving this project! (And per Jessica's request, I've lowered the minimum funding bar from $50k to $5k.)

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

7 months ago

@Corn Thanks Cornelis! And conversely I'm grateful at how you bring ideas like Taco Tuesday back and share them with your Canada friends. Bring them to Mox sometime!

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

7 months ago

(also: what are the other 2??)

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

7 months ago

@ollive Thanks Carolina! Really appreciate your recent volunteering with Mox, decorating offices and events, and just the spirit of exuberance you embody every single time I run into you

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

8 months ago

@mk_lao thanks for saying so and thanks for your donation -- we hope to have you back as well!

Agent Island: An Environment for Interagent Cooperation and Conflict
Austin avatar

Austin Chen

8 months ago

Approving this project, and making a donation to get this to its minimum funding bar!

Primarily, I'm donating based on Connacher's track record in the EA/AIS ecosystem. I also generally like demos/studies of multiagent interactions, as I think a lot of the future will be largely governed by such things, but most of our own experience is with chatbots or at most single agents.

I do wish there was a nice web UI for viewing (and playing??) the games, and also think that it'd be awesome to get a more complex but well-known game like Mafia or Secret Hitler implemented. (Though, take these ideas with a huge grain of salt; they arise from my bias as a consumer-product-person, and I trust Connacher has a good sense of what features actually matter)

Compassion Bench
Austin avatar

Austin Chen

8 months ago

I'm making a quick donation based on a good reference from @MarcusAbramovitch; he named it as one of the only projects that exists in the AI x Animals space, which I think makes this fairly neglected. I think the CaML's specific theory of change here is plausible, but I'm neither an expert in animal welfare nor AI benchmarks and am mostly deferring to Marcus here.

Safe AI Germany (SAIGE)
Austin avatar

Austin Chen

8 months ago

@jessicapwang Hey Jessica, thanks for flagging and apologies for the issue -- this should now be fixed. Please let me know (here, or via austin@manifund.org) if this is still causing problems!

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

8 months ago

@codercorgi Thanks, Alex! We very much appreciate the support and the kind words ❤️

Safe AI Germany (SAIGE)
Austin avatar

Austin Chen

8 months ago

Making a personal donation in support of this initiative. While I haven't spoken with Jessica, her track record seems solid, and I'm familiar with a couple of the folks who will be involved. Moreover, I'm impressed with the breadth of people who have come to express support on this project, which also provides some evidence that Jessica and SAIGE will do well at fieldbuilding. Best of luck!

Towards Shared AI Compute Infrastructure
Austin avatar

Austin Chen

8 months ago

@Haakon-Huynh Great, approved!

Towards Shared AI Compute Infrastructure
Austin avatar

Austin Chen

8 months ago

Hey, can you share what individuals or orgs are on this team (eg by updating your profile & project description)? We generally ask Manifund grantees to be identified in public, unless there are especially compelling reasons for pseudonymity.

"Emergent Depopulation": Translation and Publication
Austin avatar

Austin Chen

8 months ago

Hey @Piotr, thanks for flagging. I generally encourage you to downvote projects which seem like cases of psychosis, as that will reduce their visibility in Manifund's homepage. Unfortunately, we're now getting enough cases of psychosis or other LLM spam submissions that we typically don't otherwise address each one; I encourage folks who might be in this situation to read https://www.lesswrong.com/posts/2pkNCvBtK6G6FKoNn/so-you-think-you-ve-awoken-chatgpt

Improve eligibility for organ donation
Austin avatar

Austin Chen

10 months ago

Approving this project as part of ACX Grants, and making a small donation in support as well! I haven't donated a kidney (and am on the fence about whether I will someday), but am very inspired by those who have. And I do remember, back from my college days, struggling to gain weight so that I could qualify for double red blood cell donations -- so the core idea here resonates personally.

The cost-effectiveness figures presented are also compelling (though I haven't independently verified them), and I'm broadly impressed with Daniela's track record with other projects as well. Best of luck to Project Donor!

AI Security Startup Accelerator Batch #2
Austin avatar

Austin Chen

10 months ago

Approving this project; we've been happy to host the first cohort of Seldon at Mox, and I'm excited for more incubation and support in the space of AI safety startups. Finn and Esben are doing good work and I'm looking forward to seeing what's next!

FloraForge: Aesthetic Gene Editing in Petunias
Austin avatar

Austin Chen

10 months ago

Approving this project. This kind of biotech platform is a bit outside of Manifund's usual beat, but I'm encouraged by Aaron's donation to make this happen (I trust his expertise on this kind of scientific endeavor) and also Justin's past experience at Colossal, the company that de-extincted the direwolf.

Retroactive: Presenting a poster at the ICML technical AI governance workshop
Austin avatar

Austin Chen

10 months ago

Approving this small travel grant!

Fund thebes' model tinkering
Austin avatar

Austin Chen

11 months ago

Approving this project, and making a small donation myself -- I've seen Theia's work before and was excited to see this pop up on Manifund. Thanks @joel_bkr and Charles for making this happen!

Normie-friendly prediction market interfaces
Austin avatar

Austin Chen

11 months ago

@Charlie as you guessed, in general Manifund is happy to extend trust to our grantees, to spend the funds by their best judgement -- so long as it's in keeping with our charitable purpose as a 501c3 nonprofit. Feel free to ask about specifics if you're ever uncertain, but we expect that we'll be happy with whichever project you choose to pursue.

🧡
Finishing a Video on the Google DeepMind Hunger Strike
Austin avatar

Austin Chen

11 months ago

Approving this project! I've been a supporter of Michael's past work (especially the SB1047 documentary) and hope that he continues to produce good videos for AI safety.

Normie-friendly prediction market interfaces
Austin avatar

Austin Chen

11 months ago

Approving this project and making a small donation of my own! Here's an excerpt from my review as an ACX Grants evaluator:

"I like this guy. I might be biased because he’s building on top of Manifold, but his product sense is pretty good. I’m not sure if making his current dashboards/systems better is the thing I would advise him to focus on, though. I feel like I’d really want him to work together with somebody else who has a strong concept but weaker marketing skills. (Maybe, with Nuno/Rai on Sentinel…?)"

Operating Capital for AI Safety Evaluation Infrastructure
Austin avatar

Austin Chen

11 months ago

Approving this project. Equistamp has been doing work with the most prominent AI safety research orgs, has been operating profitably by providing their consulting services. After speaking with Chris and Jueyan, we're happy to help them solve their operating cash constraints.

I'm also broadly excited by this model of supporting aligned for-profit work. As Chris mentions, we're structuring this grant as a SAFE (Simple Agreement for Future Equity) investment with an $8m cap into Equistamp; proceeds (if any) will be returned to AISTOF's balance, allowing them to make further grants.

Inspect Evals
Austin avatar

Austin Chen

11 months ago

Approving this project as part of our portfolio for AI safety research. I've heard good things about Inspect, and it's great to see that many different regrantors are likewise excited to continue funding this open source work!

Synthesizing Standalone World-Models
Austin avatar

Austin Chen

11 months ago

Approving this project as part of our portfolio of AI safety research. Thanks to @Thane-Ruthenis for a well-written proposal, and to @CDBiddulph for providing initial funding and your rationale!

Groundless Alignment Residency 2025
Austin avatar

Austin Chen

11 months ago

Approving this project as part of our portfolio of fieldbuilding for AI safety. Thanks to Richard for funding this!

The Deal of the Century: Targeted Persuasion Campaign for a US-China AI Treaty
Austin avatar

Austin Chen

12 months ago

Approving this project, under our portfolio of advocacy for AI safety. Curious to hear any more context @RyanKidd might have to add!

Enabling a Student to Present his Mechanistic Interpretability Work at NeurIPS
Austin avatar

Austin Chen

12 months ago

Approving this small travel grant!

Emergency travel funding to attend EA Global: New York 2025
Austin avatar

Austin Chen

12 months ago

Approving this small travel grant!

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

about 1 year ago

@casebash Epoch & Mechanize ultimately decided to place their offices elsewhere, but we would have been happy to work with them, and we're happy to have Tamay as a regrantor in our 2025 cohort. There's a can of worms to unpack here, but in brief I/Mox are not very doom-y, and believe that there are many different ways to do good for the world.

(I'll also be posting an update on what Mox has been up to, soon!)

Neuronpedia - Open Interpretability Platform
Austin avatar

Austin Chen

about 1 year ago

@hijohnnylin thanks for the updates; it's been great to see Neuronpedia grow over these last couple years. For myself, I view this grant as one of my most prescient grants I've made, and I'm excited to see where y'all go from here!

Project on optimal government hedging against labour automation
Austin avatar

Austin Chen

about 1 year ago

Approving this under our portfolio of policy research for AI safety; I think better models and plans for post AGI labor seem good.

I'll note that I'm personally a bit skeptical of theories of change that route through "publish a paper", vs something like "get a substack article in front of smart people" or what Ezra Klein/Derek Thompson are trying with the Abundance book. But I can imagine that work on different areas of the research pipeline is important, and in any case Manifund's happy to facilitate this kind of donation!

Videos on UK Free Speech
Austin avatar

Austin Chen

about 1 year ago

@Veterinarians-For-Animal-Welfare Hi Sadiq, please don't self-promote your projects on other people's unrelated Manifund projects, in this way.

Videos on UK Free Speech
Austin avatar

Austin Chen

about 1 year ago

Approving this grant to help Sam get his funds sooner. I expect Marcus may have more takes about why he's excited for this; I'm also a fan of some of the writings on https://www.samstack.io/ !

Grow An AI Safety Tiktok Channel To Reach Ten Million People
Austin avatar

Austin Chen

about 1 year ago

quick notes:

  • I love that Michael is willing to be so transparent and responsive here. The quality of comments here are great; thanks everyone for weighing in!

  • I think highly of all of Marcus, Jesse, and Michael -- and even so, I think it's reasonable for them (and others) to have strongly different takes on whether a particular project is worth funding/doing.

    • I think sometimes EA people fall into a trap of thinking there's one "right answer" about cost effectiveness or theory of change, and we need to argue it out to get to ground truth -- and while there's some value in this, there's also a lot of value in enabling lots of experiments to be executed by the people who are most excited by those. Manifund is built around the latter philosophy: "let a hundred projects bloom"!

  • As for myself, I funded this for a small amount, to signal support for Michael based on his past track record. I think the SB1047 documentary was well done (though, I was also a funder of that, so might be biased). I think the viewership numbers here are impressive, though I'm not super calibrated on what passes for amazing media presence.

    • Contrary to Marcus, I suspect EA ought to pay people a lot more, though there's a lot of nuance ofc. I'm also very sympathetic to viewing funding this as "retro funding for the SB1047 documentary".

    • My biggest question is whether short clips of existing content are providing value to the world; how much do you get to shape the narrative if you're curating vs creating? It's not a model I'm familiar with, anyways but I do think it's on the rise.

AI Safety Los Angeles (AISLA)
Austin avatar

Austin Chen

about 1 year ago

Approving this grant under our portfolio of AI safety fieldbuilding; thanks to Ryan Kidd for funding it!

TransformerLens - Bridge Funding
Austin avatar

Austin Chen

about 1 year ago

@sheikheddy Thank you for this comment, I've just made a small donation in support as well.

Finishing The SB-1047 Documentary
Austin avatar

Austin Chen

about 1 year ago

@michaeltrazzi I really appreciate the thoughtful and in-depth retrospective! And I'm sorry if the presentation at The Curve slowed down the time to present this; I hope you still found value in presenting it there~

AI forecasting and policy research by the AI 2027 team
Austin avatar

Austin Chen

about 1 year ago

@Tebycimtime yes, sorry for the confusion in our interface -- the number $202 indicates your total donated at this point in time, summing up the $100 and $102 donations. You can also check out your profile for a log of transactions which display the two separate donations!

AI forecasting and policy research by the AI 2027 team
Austin avatar

Austin Chen

about 1 year ago

Hi Michael -- are you saying that the second $100 donation was an accident and you want to undo it? If so, I can process that for you!

Development of a Cautionary Tale Feature Film about Gradual Disempowerment
Austin avatar

Austin Chen

about 1 year ago

Making a small initial offer in support of this project! Reasons I'm excited for this:

  • Petr's past work with Dwarkesh is both a strong credential, and a credible signal that he can produce more films of this nature

  • I am excited for gen AI reducing the costs of great cinematography work. My guess is that right now, this form factor is underexplored because Hollywood labor unions push against genAI

  • Separately, I'm excited for alternate visualizations and ways of communicating concepts from the AI safety sphere; "essays" are a common form factor, but the general public consumes much more video content compared to the typical AI safety person

And some reasons I'm not (yet) donating more:

  • Video content has been popular before on Manifund before, so I'm expecting other donors to also be particularly excited to fund this. I'm also not any kind of expert on video content.

  • For now, it seems like Dwarkesh's regular recorded video podcasts are more popular than the genAI documentaries. (This is only a small update; I expect the a high rate of improvement on genAI vs regular, as it's a much newer field)

  • Overall, it does seem like an expensive intervention ($2m), and this proposal is just fundraising for an initial demo

I also wanted to ask Petr about the nonprofit nature of this proposal -- should the feature film end up being widely popular and profitable, how would those profits be allocated between funders and the production team?

Funding researcher on frontier AI governance inclusive of global majority
Austin avatar

Austin Chen

about 1 year ago

Approving this grant to support a promising individual's continued research and education. While Manifund generally does not prefer keeping identities private, I've corroborated with Jueyan on the details here, and consider H a promising candidate and agree that in this situation it's reasonable to keep the grantee pseudonymous.

(Just musing: in the future, one market-based way to thread this needle would be to charge a higher operational fee, eg 20% rather than our standard 5%, to price in our cost of doing pseudonymized grants)

Split Personality Training
Austin avatar

Austin Chen

about 1 year ago

@FlorianDietz Terribly sorry about the delay -- I've put in the wire this morning, which should arrive to your bank account in the next 3 days.

Improving farmed fish welfare in Egypt
Austin avatar

Austin Chen

about 1 year ago

Apologies for the delay, approved now as part of our portfolio to improve animal welfare!

Giving free AI safety books for potentially high-impact individuals
Austin avatar

Austin Chen

about 1 year ago

Approving this grant as a low-cost cheap intervention to spread high-quality writing on AI safety. Thanks to Aidar for proposing this and Neel for funding!

Acausal research and interventions
Austin avatar

Austin Chen

over 1 year ago

Approving this project. I'm excited that Manifund can help support more speculative and philosophical work that is generally neglected. I do expect that understanding decision theory better will serve us well as we move into weirder worlds; and on a brief skim, their decision theory benchmark seems promising. Thanks to Thomas and Lauren for funding this!

AI Digest
Austin avatar

Austin Chen

over 1 year ago

Excited to approve this project, and contribute a bit of funding myself! Sage is one of the few EA/AI safety orgs that can ship great software products. I've been a fan of theirs all the way since the Fatebook days; more recently, Agent Village has been showcasing AI capabilities in a fun, wholesome way. I think they have a very strong product & technical team, are still iterating to find "product-market-fit", but given their impressive pace of demos and releases I'm excited to support their work.

Keep Apart Research Going: Global AI Safety Research & Talent Pipeline
Austin avatar

Austin Chen

over 1 year ago

Approving this grant as part of our portfolio on AI safety fieldbuilding! I'm happy to see such widespread praise for Apart's work, and to see that Richard, Anton, Ryan, Soroush, and Gavin (among many others) want to fund Apart to continue with its mission.

Some reflections:

  • Gavin & Anton have called out that larger funders seem not to be funding Apart. In my head, I sometimes think of Apart as part of & supporting a "shadow AI safety scene", my made-up distinction to contrast Apart and things like AISC, aisafety.world, etc from the "mainstream AI safety scene", which is more typified by OpenPhil, Constellation, MATS, and the AI labs. My sense is that the mainstream scene is top down, high-trust, densely-networked in the Bay (plus London, DC); while the shadow scene is bottoms-up, loosely confederated, remote. The mainstream scene takes a while to build up the trust to dedicate large amounts of funding to people outside of it; I suspect that's one of the key reasons for Apart's funding difficulties. Part of why I'm excited to run Manifund is to support projects outside the "mainstream" (where I/Manifold/Manifund began, and arguably still reside); I feel that Apart is in some sense kindred to Manifund, and I'm particularly glad we can be helping them here.

  • I also just really like hackathons (see this), and so I'm glad that somebody else in the ecosystem is systematically pushing for more hackathons!

  • My biggest uncertainty with Apart is that Esben himself is stepping back into an advisory role, to focus more on their new for-profit incubator Seldon. I'm excited for Seldon (we're hosting their first batch out of Mox!), but from an outside view, having a founder depart means that Apart now sits at a fairly critical juncture for having continued impact. As a funder, my best guess is that I'd prefer to fund Seldon over Apart (though I recognize that these cater to different audiences and funders will have different preferences.)

  • My unsolicited advice is that Apart should try to expand their revenue base beyond soliciting donations, to encourage discipline, financial resilience, and incentive alignment. For example, Apart seems to do well at placing great candidates into new jobs, and a natural thing to do would be to ask employers recruiter's fees to fund Apart's continued operations. "Alumni donations" would be another angle on this, similar to how universities raise from those they've taught.

Calibration City
Austin avatar

Austin Chen

over 1 year ago

@wasabipesto brier.fyi looks fantastic, great work! This is the kind of core infrastructural work that helps inform expert forecasters, and also makes forecasting more understandable to newcomers. Visual design-wise, the diagrams look great, too! I've donated an additional $5k in recognition of the time & effort you've put into making this happen.

Also some quick feedback:

  • The home page is quite wordy atm -- I'd suggest radically trimming or collapsing text from the home page, and letting the visualizations speak for themselves more. Could take inspiration from PlasticList perhaps, they put most of the explanation on other pages.

  • Also, suggest shorter line lengths for readability, capping around 100 chars per line. See also these typography tips.

  • I'm slightly peeved at how low Manifold currently ranks, but I'm hoping it's a kick-in-the-pants to the Manifold team to figure out why and how to make Manifold more accurate, and also signal to users which markets are less trustworthy

Understanding Trust
Austin avatar

Austin Chen

over 1 year ago

Approving this project! While I'm not very familiar with this field of research myself, Abram is well-regarded in the AI safety space; I've spoken with another funder I trust who was interested in supporting Abram's work.

I'm also glad that Manifund can help out here, by serving as a simple, low-cost option for fiscal sponsorship!

Scaling AI safety awareness via content creators
Austin avatar

Austin Chen

over 1 year ago

I've re-opened the grant application now!

Develop technical framework for human control mechanisms for agentic AI systems
Austin avatar

Austin Chen

over 1 year ago

@wiserhuman apologies for the delay on this; I've just put in the wire!

Orexin Pilot Experiment for Reducing Sleep Need
Austin avatar

Austin Chen

over 1 year ago

Approving this project and making a small donation in support. It would be great to reduce the amount of time I & others are required to sleep, so I'm very curious about how this experiment goes. I've also previously worked with Sam Harsimony way back when Manifold was first starting on charity prediction markets (it was his idea!)

Virtue-Ethical Rationality and Training Dynamics
Austin avatar

Austin Chen

over 1 year ago

Approving this project. As a sidenote, I'm very grateful for Gavin's clear writeup explaining why he wanted to fund this!

Asterisk AI Blogging Fellowship
Austin avatar

Austin Chen

over 1 year ago

Very excited for this project. "Get people blogging about transformative AI" seems like a straightforwardly good idea. While there are already some great blogs out there (eg ACX & AI Futures Project, Zvi's, Steve Newman's), I expect there could be many more; the subject is vast, complex, with numerous niches to explore.

Asterisk also seems very well-positioned to run this. I think of Asterisk itself as one of the biggest wins in the space of EA/rat-adjacent public comms. Almost every single piece published on Asterisk impresses me with the quality of the writing and the importance of the topic, and so I'm excited for the writing that will come out of this fellowship. (I wish there could be an Asterisk-quality magazine for AI safety itself; if you think you could run this project, let me know!)

Finally, producing "writers" seems as (if not more) important than producing "writing". Even just for Manifund, we've struggled to find good writers who can write clearly & engagingly, and also understands the types of projects we fund, in the AI safety and broader EA space. I also expect many orgs have similar problems, and am excited for this fellowship to surface new talent to help with this.

I've put down $10k in support and hope others join as well; I'd love if Manifund could help Asterisk find more diverse sources of funding!

Split Personality Training
Austin avatar

Austin Chen

over 1 year ago

Approving and making a small ($1k) donation as well, as the priors on this kind of project seem reasonable (MATS mentee asking to continue their project), and Marius and Evan's support gives me some confidence. As Marius says, strong early results could encourage me to fund this with more!

'Making God': a Documentary on AI Risks for the Public
Austin avatar

Austin Chen

over 1 year ago

@Connoraxiotes Just wanted to say that I very much appreciate this level of transparency and openness about your thinking & finances; I think it's super helpful for others embarking on similar projects!

Coursetexts
Austin avatar

Austin Chen

over 1 year ago

Approving this proposal, and funding it to half of the total ask. I'm not personally in the target demographic (it's been a while since I've studied off course notes), I'm excited to fund this becuase:

  • Michael Nielson and Lawrence Lessig are two folks I've long admired, and their support is a strong signal

  • While I was not previously familiar with them, the team behind coursenotes seems quite strong, with multiple interesting projects to their name

  • The design of the site itself is nice, and I appreciate the clarity of vision listed on their "why" page

Best of luck to the team!

(Aside, a minor UX nit: I think this is better as a single select rather than multiselect, rn almost no coursenotes fit the intersection of any two options)

AI forecasting and policy research by the AI 2027 team
Austin avatar

Austin Chen

over 1 year ago

Approving this proposal. AI Futures Project has been one of the most hyped efforts in my circles for a while now, and for good reason: it brings together some of the most accomplished individuals in the AI safety scene, working under a single banner. For this reason alone, I think the team is worth taking a bet on.

They've also already made their mark, with AI 2027. I'd gotten a sneak peek, and had actually been a bit unimpressed - I had high expectations, but imo the preview suffered from "too many cooks in the kitchen" wrt writing & site design. But by launch, AIFP had upped their game, with a polished product that's been well & widely received. It's certainly shaped how I think about the next few years of AI development. Kudos to the team for being willing to share an early v0, and then iterating to make it better over time!

I think the people working on this are super smart and probably know what they're doing, but I figured I'd throw in my unsolicited 2c:

  1. It seems like the core team is already heavy on researchers, so it's unclear to me that hiring more researchers is the right strategic move, vs investing in roles that can produce great content for a wide audience. Right now their plan seems to be to partner with really great folks (eg Scott for writing, Oli for website design, Dwarkesh for podcasts), and it seems to be working so far, but I would guess that having in-house expertise on this could be super valuable, much more so than a marginal researcher.

  2. Specifically with the TTX, I haven't played through one myself, but my understanding is it's currently costly to run (requiring an in-person expert facilitator). I'd be pretty excited for ways to automate that, scale it out, and get much wider distribution, eg by shipping an interactive web experience powered by LLMs, or packaging it as a commercial board game.

Anyways, AIFP is one of the most exciting efforts I'm currently tracking; I've made a small personal donation as a show of support. I expect that AIFP will be amply funded by larger parties like OpenPhil and SFF, and as Neel says, is not really in my comparative advantage; but I still think that independent donations are valuable for diversifying funding streams.

🧡
Luthien
Austin avatar

Austin Chen

over 1 year ago

@jaiwithani Yes, that's fine!

Luthien
Austin avatar

Austin Chen

over 1 year ago

@jaiwithani Yes, we should be able to do this for your SFF application; we've historically done this for one other Manifund grantee. Note that we would ask for 5% of the grant as a fiscal sponsorship fee. Email me at austin@manifund.org to confirm this!

Coordinal Research: Accelerating the research of safely deploying AI systems.
Austin avatar

Austin Chen

over 1 year ago

Approving this grant and making a small ($5k) initial grant for support! Using AI to automate & speed up AI safety research is one of those proposals that seems like a no-brainer; I'm glad that the Coordinal team is giving it a shot.

As a caveat, I have not yet tried out their demo ( I'd encourage Coordinal to make one example output visible, so people without a demo key can still step through a tutorial!) I'm also not well positioned to evaluate the quality of the tool's research outputs, not being an AI safety researcher myself. However, Ronak (who works out of our space, Mox) seems competent and on the ball; the screenshots they've shown here look promising. I'm also glad to see that Jueyan and Ryan are supporting this initiative, too.

One note:

We plan on keeping core infrastructure fully private, and providing access to the system only to vetted alignment researchers. We're only open-sourcing the direct research outputs (papers, findings, benchmarks, generated code) rather than the automation infrastructure itself.

I understand the hesitation around open sourcing the core infra, but as a donor (or investor) I would be somewhat more excited if Coordinal was planning on open sourcing everything! I think most AI safety people worry way too much about dual use, and startups overvalue the importance of code as a moat; and on the other hand, people being able to learn from & build on top of Coordinal's work seems like it could meaningful change the trajectory of their impact.

Develop technical framework for human control mechanisms for agentic AI systems
Austin avatar

Austin Chen

over 1 year ago

Approving this grant as in line with our mission for funding AI safety technical research!

Apologies for the delay here; we were waiting for Ryan or others with context to weigh in on the grant rationale, which we still hope to have; but we want to unblock Francesca in the meantime.

Dads Against AGI Inc.
Austin avatar

Austin Chen

over 1 year ago

@Greg_Colbourn oh, I just wanted to chime in here because I think "approving a grant on Manifund" can sometimes be mistaken for "Manifund or Austin directly thinks this is good", and I want to clarify our stance as a platform.

I don't have a specific scenario or mechanism I feel ready to defend at this point, with regards to "things will be great". Possibly it'd be good for me to sit down and write that out at some point? But honestly this might be less of an rational, debatable point for me and more of a vibe.

(Happy to explore this more next time you're around!)

Promote Georgism
Austin avatar

Austin Chen

over 1 year ago

@jacaissie thank you for the in-depth update! I enjoyed reading about the progress y'all have made~

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

over 1 year ago

Very grateful to say that we've received two large grants towards Mox:

  • $250k from Craig, an individual donor

  • $60k from the EA Infrastructure Funds (not registered here, but we intend to also register with impact certs at the same $8m premoney valuation)

With this funding, we're feeling much more comfortable about our ability to improve the 3rd and 4th floors of Mox; we're also now looking to expand the team by 1-2 folks.

We are still waiting to hear back from Open Philanthropy; and are still seeking more funding from other donors and investors as well. We think we could use up to $2m quite effectively, towards revamping and renovating our space (eg to support short and medium-term housing) and to run programs and fellowships here. It's been a great 2 months so far, I'm very excited for the upcoming work here!

Run a public online Turing Test with a variety of models and prompts
Austin avatar

Austin Chen

over 1 year ago

Congrats @cameron! Great work, I'm impressed by the amount of press coverage (eg 1, 2, 3, 4) this has gotten; even though the AI research community has mostly stopped considering the Turing Test as a meaningful evaluation, I think a proper scientific paper that showcases this result to the public is super important!

I'm also grateful that Manifund got a shout-out here, it might be our first citation:

'Making God': a Documentary on AI Risks for the Public
Austin avatar

Austin Chen

over 1 year ago

Approving this project as compatible with Manifund's mission of fostering education on AI risks. I've spoken with Connor and Mike in person and think they are taking a tactical and reasonable approach to making this documentary.

As with my comment on Dads Against AGI, I'd clarify that I personally hold some values dissonance with the grantees here -- for example, I mostly feel that AI labs, and their CEOs like Sam Altman and Dario Amodei, are generally doing good work. But Manifund aims to be a neutral platform, where projects can express a variety of different viewpoints, and so we are happy to facilitate grants to this documentary.

Dads Against AGI Inc.
Austin avatar

Austin Chen

over 1 year ago

Approving this grant, in line with our support of projects educating about AI risk. It seems like DAA has real traction and an experienced team behind it.

FWIW, I personally disfavor the tone that DAA uses; on a quick read, they're much more Pause-y and doom-y than I would endorse. (I'm a pretty optimistic person, and expect we'll be able to make all the AI stuff go well). However, Manifund aims to provide neutral funding services, and so we're happy to host DAA while they're pursuing their own 501c3 status. I also respect that Louis and John are spending their time & money pushing for the change they think is necessary in the world!

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

over 1 year ago

Thanks, @AntonMakiievskyi! Appreciate the support, and we hope to make Mox great for all that you described~

Establishing the Utilitarian School of Thought in Thai Society
Austin avatar

Austin Chen

over 1 year ago

Approving this small grant towards translating EA concepts and utilitarian thinking to new audiences.

Luthien
Austin avatar

Austin Chen

over 1 year ago

Approving this project -- my understanding is that Control is becoming an important new agenda, and I'm heartened to see this donation following AISTOF's initial seed funding. Curious if @RyanKidd has any more to share about what motivated him to make this grant?

Philanthropic advising
Austin avatar

Austin Chen

over 1 year ago

Approving this grant to support independent work by Tyler John. I've chatted with Neel about the context behind this, and agree that it's reasonable to keep some details private around what Tyler is hoping to do.

I also think from an outside view, Tyler's past work at Longview means that we should be happy to fund some experimentation for him. I think it's a shame that nonprofit work typically does not pay well enough for even successful employees to take risks in their career, and hope that regranting can be one solution to this.

Finally, on the object level, I'd be excited by the prospect of a large biosecurity prize; I think prizes are still undersupplied as a means of facilitating great work. (I also wish someone would credibly try to manifest a large yearly AI safety prize -- if that is you, reach out!)

[Urgent] Top-up funding to present poster at the Tokyo AI Safety Conference
Austin avatar

Austin Chen

over 1 year ago

Approving this grant as a low-budget way to enable Matthew to attend this Tokyo conference and share his work.

Mox, a coworking & events space in SF
Austin avatar

Austin Chen

over 1 year ago

Thanks! Grateful for both your donation, and your membership. One of the best things about Mox is just being able to bump into @saulmunn (and people like him) all the time~

Out of This Box: AI Safety Musical
Austin avatar

Austin Chen

over 1 year ago

@Maria thanks for asking -- if you add money to your balance, that would count as a charitable donation to Manifund (a US-based 501c3 charity). You can then spend that charity balance towards your choice of projects on our site; Manifund vets each project to make sure it is compliant with our charitable mission.

Out of This Box: AI Safety Musical
Austin avatar

Austin Chen

over 1 year ago

@manuelallgaier apologies, I've now deleted the other comment!

human intelligence amplification @ Berkeley Genomics Project
Austin avatar

Austin Chen

over 1 year ago

Approving this project as compatible with our charitable mission of furthering public scientific research! Tsvi has a track record within the rationalist community, and this agenda seems intriguing; I hope it goes well.

Attention-Guided-RL for Human-Like LMs
Austin avatar

Austin Chen

over 1 year ago

I don't know Scott at all, but looking through his website, he's previously received OpenPhil funding and won an ELK prize: external credentials that his research may be worthwhile. I also like the way he thinks, both here and on the Act I project -- it seems good to me to treat AIs with the consideration one might show a child or friend. (Indeed, one of my longer term visions for Mox includes something Act I-y, by having models be participants in the space alongside humans). For these reasons, I'm happy to support Scott with a small initial grant!

Animal Advocacy Innovation Hub in the Bay Area
Austin avatar

Austin Chen

over 1 year ago

Constance is one of the most gets-things-done people in EA community building. I'm consistently impressed by her ability to bring people together & execute on ideas, and am excited to support her and her vision of an SF-based hub for animal welfare folks.

(conflict of interest notes: Constance is a friend; much of her outlined budget would be going back towards Mox, as funding for events or coworking spots)

More Detailed Cyber Kill Chain For AI Control Evaluation
Austin avatar