Exploring feature interactions in transformer LLMs through sparse autoencoders

Script to download and run Manifold locally

Mirrorbot

Automatically mirror popular questions from other platforms to Manifold

AI Safety Research Organization Incubator - Pilot Program

Exploring novel approaches to addressing Global Catastrophic Risk through policy

Operational Support for a Pilot Study on Complexity Modelling, Expert Judgement, and Deliberative Democracy for Risk Prioritization and Mitigation

Manifold Sandbox

Explore arbitrary AMM actions in a Manifold simulator

Wasabipesto's Umbrella Project

Six Projects in Search of an Investor - Details inside

Extreme Probabilities Project

Explore a market structure that attempts to produce good probability estimates on highly-confident markets.

Manifold corporate blog on Habr

Promoting prediction markets in software developers' community

Isaac's Blog Writing

Manufacture Manyfold Manifolders in the Maritime Metropolis

Lets create a new defining addiction for Seattle!

Isaac's power-user Manifold search and dashboard

Manifold Tournaments

Tackling First-Week User Churn at Manifold 🐣

Creating a compelling onboarding and first-week experience to enhance user longevity

Experiments to test EA / longtermist framings and branding

Testing responses to “effective altruism”, “longtermism”, “existential risk” etc.

~~Manifold corporate blog on Habr~~ [resubmitted]

Teacher Training Education in Nigeria using Artificial Intelligence

Making education worthwhile in Nigeria for teachers using AI

Future Food

Introduction Of Plant Based Meals In University Canteens Menus

Scaling Training Process Transparency

Compute and infrastructure costs

Estimating annual burden of airborne disease (last mile to MVP)

Boosting advocacy for investment in and deployment of technologies for improving indoor air quality

Good Ancestors Policy expenses

GAP is an Australian charity, working to improve the long-term future, that requires funding for salary and operational expenses.

Long-Term Future Fund

The Base Rate Times

News through prediction markets

Donations List Website (retroactive)

DLW is a compilation of EA donation data across funders.

Exploring novel research directions in prosaic AI alignment

3 month

Personal development and better infrastructure for learning, "Anki V2"

SRS software projects and general career development

Trading assistant bot (Remind Me)

Sends you notifications when markets that you are betting on have a big swing and allows you to set parameters for custom reminders

Manifold: Live!

A weekly community livestream/discord call where top traders discuss the week's events on Manifold and live-bet with the audience on markets from new users

Manifold merch store

Putting the "fold" in Manifold

N.C. Young's Umbrella Project

Everything Manifold-related that I'm doing on my own

Invest in the Conflux Manifold Media Empire(??)

I'm planning to do something starting in January, but I'm not sure what!

Cadenza Labs: AI Safety research group working on own interpretability agenda

We're a team of SERI-MATS alumni working on interpretability, seeking funding to continue our research after our LTFF grant ended.

MATS Funding

Help us support more research scholars!

A Lumenator Company, or: A More Ambitious Life Trajectory

Using Bright Lamps to make money, improve EAs productivity, do research and achieve free agency

Empirical research into AI consciousness and moral patienthood

Support Riesgos Catastroficos Globales

Avoiding Incentives for Performative Prediction in AI

Forecasting - AI Governance Policies

WhiteBox Research: Training Exclusively for Mechanistic Interpretability

1.9 FTE for 9 months to pilot a training program in Manila exclusively focused on Mechanistic Interpretability

Compute and other expenses for LLM alignment research

4 different projects (finding RLHF alignment failures, debate, improving CoT faithfulness, and model organisms)

Support funding for Hardeep Gambhir's gap semester

Optimizing clinical Metagenomics and Far-UVC implementation.

3-month research to inform the design of large-scale R&D projects in metagenomics and Far-UVC

Activation vector steering with BCI

Run five international hackathons on AI safety research

Six-month support for a Program Manager to organize and execute international AI safety hackathons with Apart Research

AI-Plans.com Critique-a-Thon $500 Prize Fund Proposal

VaccinateCA

Retroactive grant for website & infra to display COVID vaccine availability

Neuronpedia - AI Safety Game

Play the Game. Rack Up Points. Save Humanity?

Support for Deep Coverage of China and AI

One semester living expenses for MIT/Harvard-based researcher

Necessary funding for young engineer to move to Cambridge to continue high value Human Augmentation and XR projects

Alignment Is Hard

Proving Computational Hardness of Verifying Alignment Desirata

AI Alignment Research Lab for Africa

AI Safety lab focusing on technical alignment and governance of AI in Africa and the Global South more broadly. We are a grassroots community-led research lab

Apollo Research: Scale up interpretability & behavioral model evals research

Hire 3 additional AI safety research engineers / scientists

Agency and (Dis)Empowerment

6 months support for Damiano and Pietro to write a paper about (dis)empowerment with Jon Richens, Reuben Adams, Tom Everitt, and Victoria Krakovna.

Funding for Solar4Africa app development

Supplementing living expenses of a full stack dev to pursue educational opportunities while doing app development in Berkeley for a week.

Scoping Developmental Interpretability

6-month funding for a team of researchers to assess a novel AI alignment research agenda that studies how structure forms in neural networks

Recreate the cavity-preventing GMO bacteria BCS3-L1 from precursor

Because I think curing cavities for everyone would be "based", actually

Discovering latent goals (mechanistic interpretability PhD salary)

6-month salary for interpretability research focusing on probing for goals and "agency" inside large language models

Holly Elmore organizing people for a frontier AI moratorium

Help me to organize US moratorium-promoting activities to expand the Overton window and increase public pressure in favor of a moratorium on frontier AI.

Introductory resources for Singular Learning Theory

Literature review or other introductory material for Singular Learning Theory targeted at AI alignment applications

Reflective altruism

Joseph Bloom - Independent AI Safety Research

Trajectory Models and Agent Simulators

Medical Expenses for CHAI PhD Student

Allow Berkeley PhD student to devote more time and focus to AI safety research and mentorship.

How General and How Comprehensive Must AGI Be?

What is the probability that AGI is developed by January 1, 2043?

CHAT Diplomacy

Advances in LLMs are a National Security Risk

Odds of Artificial General Intelligence by 2043

By srhoades10

AI will change the world, but won’t take it over by playing “3-dimensional chess”

By Boaz Barak & Ben Edelman

What would power-seeking, misaligned AGI actually do?

By Kiel Brennan-Marquez

When can a mimic surprise you?

By David Johnston. Why generative models handle seemingly ill-posed problems

Artificial General Intelligence (AGI) timelines ignore the social factor at their peril

By Trevor Klee

Why do we post our AI safety plans on the Internet?

By Peter S. Park. It is plausible that AGI safety research should be assumed compromised once it is posted on the Internet.

Why I think strong general AI is coming soon

By porby

Disagreements with bio anchors that lead to shorter timelines

By Marius Hobbhahn

‘Dissolving’ AI Risk

By Froolow. Parameter Uncertainty in AI Future Forecasting

AI X-Risk: Integrating on the Shoulders of Giants

By TD_Pilditch. Uncertainty, Structure, and Optimal Integration.

Subsidize Real Money Prediction Markets on High Impact Topics

Create Subsidized Prediction Markets On Polymarket.com

Manifold feature to improve non-resolving popularity markets

Add additional social incentives to bet YES to publicly signal value/support/belief and NO to signal the opposite.

Comparing forecasting platform accuracy

Collaborative Resolution of Political Controversies

Crystal Ballin' Podcast

New fortnightly podcast on all things prediction markets

OPTIC

Intercollegiate Forecasting Tournament

Make large-scale analysis of Python code several orders of magnitude quicker

Provide Rationality / EA Education at University of Maryland

Impact Assessment of Social Programs

Verifying and Quantifying the Outcomes of Nonprofits

Manifold Markets Add-on for Google Sheets

An Add-on for easy access to Manifold´s Prediction Markets

Standardized Tools for Impact Market Reporting

Working towards GAAP For Impact Markets

Forecast Dissemination Mini-Market 3 of 3: Earthquake Risk

Evaluating whether forecasts of earthquake risk lead people to take steps to prepare

Forecast Dissemination Mini-Market 2 of 3: Hurricane Hazards

Promoting forecasts in order to help mitigate weather risks, then evaluating the effect of this information.

Forecast Dissemination Mini-Market 1 of 3: College Majors

Promoting forecasts in order to help college choose their field of study, then evaluating the effect of this information.

Telegram bot for Manifold Markets

Superforecaster predictions of long-term impacts

I will pay 30 superforecasters to make forecasts of long-run treatment effects from randomised control trials

A tool for making well sized (~Kelly optimal) bets on manifold

Instead of saying "I want to bet $X on this market" you will say "I think the true probability is Y%", and it will tell you how much to bet

Blog about Forecasting Global Catastrophic Risks

Forecasting GCRs is hard. How can we do it better, and how much better is that?

Interpretable Forecasting with Transformers

Make accurate, well-calibrated, and honest predictions with large language models.

[Funded] Gabriel Mukobi Summer Research

Evaluating Multipolar Cooperation Failures Between Autonomous Language Model Systems

Retroactive funding for Don't Dismiss Simple Alignment Approaches

AI-Plans.com

Alignment Research Platform

Funding to attend AI Conclave

A month long on-sight campus to deeply understand and shape AI

EffiSciences: AI safety field-building in France

Funding to extend our runway and fund promising projects

Eliciting Probability Trees

5 person dev team - if experts wrote down probabilities more, would we get better decisions?

Building and maintaining the Alignment Ecosystem

12 months funding for 3 people to work full-time on projects supporting AI safety efforts

Stampy’s AI Safety Info

Creating an interactive FAQ to explain existential risk from AI to all audiences

Global Priorities Research at ANU

2-month support to do research on consequentialism and decision theory at ANU

Impact Stream - Addressing Unemployment and Poverty in Togo through blockchain

Addressing Unemployment and Poverty in Togo through Local Empowerment

BaseX

New definition of value. Accounting for externalities.

Diarynator 😎

Convert your diaries onto neuroscience/ai related categories (and see it walk!)

Fund an expert veterinary panel to create guidelines for a slaughter-free future

The panel will create guidelines to phase out industrial animal agriculture and re-wild agricultural lands.

Empowering AI Governance - Grad School Costs Support for Technical AIS Research

9-month university tuition support for technical AI safety research focused on empowering AI governance interventions.

OPTIC F23

Collegiate Forecasting

SEO for Discontinuous Analysis

Advertise this breakthrough discovery. Newton discovered continuous (even differentiable) analysis. It was improved by Cauchy. Mine is the next level.

Carbon Flow Colony DAO app

The carbon accounting app with anti-theft. We need to protect trillions of dollars worth of carbon credits.

Salaries Science

Cryptocurrency micropayments to fund scientific research, with emphasis on basic science

Educating World Class Leaders From Developing World on AI/Pandemics

University Forecasting Clubs [OPTIC]

Funding to start 3-5 university forecasting clubs as a subproject of OPTIC

ClimateFire AI: An AI-powered Early Wildfire Detection Service

ClimateFire AI detects wildfires in real-time and sends alerts directly to first responders and residents in danger.

iAnalyse Mentorship Program

Addressing the digital skills gap (Big Data Analytics) through Mentorship & AI Safety Training

Design and testing of broad-spectrum antivirals

R&D and/or operational support for initial design, production, and testing of broad-spectrum antivirals that could prevent most respiratory viral infections

HI2T: An AI-Based Tool for Insightful Analysis and Meaningful Influence

Seeking funding to patent and develop an impactful AI based data collection and communication tool.

General Support for the Center for AI Policy

Help us fund 2-3 new employees to support our team

Revolutionizing Water Infrastructure: A Rational Approach

Optimizing Water Infrastructure: Leveraging Data-Driven Strategies and Effective Decision-Making to Reduce Costs and Enhance Efficiency

Build an AI Safety Lab at Oxford University

Funding to establish a safety and interpretability lab within the Torr Vision Group (TVG) at Oxford

Epistemology in Large Language Models

1-year salary for independent research to investigate how LLMs know what they know.

Jaeson's Independent Alignment Research and work on Accelerating Alignment

Collective intelligence systems, Mechanism Design, and Accelerating Alignment

CEEALAR

Centre hosting EAs’ research projects/EAs’ upskilling for ~1/3 of the average cost of funding such projects remotely

Graduate Students for Safe and Responsible AI (SAFARI)

An association for interdisciplinary interest in AI 

Grant for Research into Infrabayesian Physicalism (GRIP)

The Rethink Priorities Existential Security team: Research Fellow hire

Funding to hire a junior researcher on a 3-5 month contract to help launch projects tackling existential risk

The Rethink Priorities Existential Security team: Founder in Residence hire

Funding to hire an in-house entrepreneur on a 12 month contract to investigate a space within existential risk and then found an impactful project

App for personal predictions with friends

Like Metaculus but more popular

Subsidize Real Money Prediction Markets on High Impact Topics

Create Subsidized Prediction Markets On Polymarket.com

Social Forecasting Platform for Retail Investors

6-month support for the concept development.

Investment in BOAS to generate €5-10K monthly sales to secure full investment

Create ‘Responsible AI Investing’ recommendations for institutional investors

Four months salary to draft and promote the recommendations, helping investors advocate for specific safety and governance practices at labs and chipmakers.

Blackbelt

A scalable, non-infohazardous way to quickly upskill via digestible, repeatable exercises from papers and workshops.

Understanding Memory Management in Transformers

We build a scalable "Automated Circuit Discovery" method and investigate "Cleanup Behavior" to advance the interpretability of transformer models.

Forming a 501(c)(4) organization for AI safety policy advocacy/lobbying

4-month salary and initial expenses

The Slipstream

Software infrastructure for global cooperation

Congressional staffers' biosecurity briefings in DC

Following on the Stanford Biosecurity Workshop for Congressional staffers, three events in DC will reconvene staffers and cover research and policy updates

Berlin EAGx Art Exhibit: SEE VIDEO Of completed piece!

ea-X-hibition: a new working group to curate art exhibits at EAGx Berlin, already approved by the event organizers.

FrontierRegistry - Science Publishing & Peer Review Protocol

Scientific and research publishing and peer review enabled by blockchain and AI

Continued funding for a PhD in AI x-risk decision and risk analysis

Continuation of a previous grant to allow me to pursue a PhD in risk and decision analysis related to AI x-risks

Jacques Thibodeau - Independent AI Safety Research

3 month salary for AI safety work on deconfusion and technical alignment.

Enable communities to cure PTSD in children after state failure

ACT International Longevity

AI Safety Events

Supporting the realization of events beneficial to the field and community of AI safety.

Cooperative for Altruistic Media Arts - CAMA

A funding funnel for EA aligned documentary films

Knowledge gaps database + R&D roadmap for cultivated meat industry

Accelerating cost-competitive cultivated meat by identifying likely bottlenecks and focusing R&D efforts on high-impact areas

Seeds of Science Research Institute

Empowering independent researchers and creating a new organizational model for science

Riesgos Catastróficos Globales

Promoting better management of Global Catastrophic Risks in Spanish-Speaking countries.

Facilitate lawsuits to restrict corporate AI-scaling

A new org to facilitate filing of court cases for restricting AI data piracy, misuse, and compute.

6 months’ salary for EA community-building and studying x-risks and AIS

One 6-month cycle of DBT therapy

MA in International Policy

Ranked, Contributable Compendium of Alignment Plans - AI-plans.com

Making a simple, easy to read platorm, where alignment plans and their criticisms can be seen and ranked. Currently in Stage 1.

Model Interpretability on modFDTGPT2-XL, a partially-aligned model

6 months of work: Evaluating a variant of GPT2-XL that can simulate a shutdown activation, aiming to improve alignment theory & develop interpretability tools.

[deleted]

[deleted]

Shrimp Welfare Project - Special Program

Place Electric Stunners at Selected Producers

Analyze this Mechanism

This mechanism seems promising for public goods funding and wealth equalizing dynamics, hire mathematician to check

Sow the Seeds of Rationality

HPMOR for promising Junior High and High Schoolers

Deriving Benefit-Cost Ratio of Outcomes

Calculating the benefit of outcomes on GiveWell and Impact Genome Registry

CO2 in my room

There's CO2 in my room

GradeFact

Credit scores for predictions made by news, media, public figures to keep them accountable

Remote Indigenous Literacy/Numeracy Courses

Nungalinya College (Darwin) offers literacy and numeracy courses to over 180 remote Indigenous students each year

Eek!

Solving AI alignment through cross-species communication

The Annotated Willard

A Primer on Self-Justifying Axiom Systems

Base Rate Times

Like The Economist meets Metaculus

Haruspex [Forecasting Competition Platform]

Mobile platform for some forecasting games

Multiplayer RootClaim

Allowing prediction market participants to construct, share & clone prediction models

Predicting Replication in Academic Economics

Replicate one year of the AER, test robustness, and see if it's predictable.

Fund the localisation of LibreOffice into Pidgin/Bislama/Tok Pisin

Bringing open source software and digital ownership and identity to Papua New Guinea, Vanuatu and the Solomons.

AI Safety Policy Communication for Legislators (Blog)

A blog (which already has top policy decision makers in No.10 subscribed) to communicate non-technically ideas about AI safety (mostly).

Implement a Small-Group Futarchy App

Decision markets for ad hoc group decision-making

Hire biosensor technicians to collect jhana data

Staff an already-scheduled retreat sufficiently to map the neural correlates of life-changing meditative states

Deploy an already built prediction website

For a smaller language - Bulgarian

Make forecasting fun again

Devise and execute fun/visible/viral forecasting engagement projects

Can communication between contributors improve aggregate forecasting accuracy?

Previous research says no, but that research is incomplete.

DiPalConnect app

Digital Pal Connection App

Forecasting Knowledge to Brazil [and other Portuguese-speaking countries]

Organizing, translating and writing what we know [and forecast] about forecasting, in order to lower the entry barrier to a country of significant influence in the Global South and a long track of international cooperation