Exploring feature interactions in transformer LLMs through sparse autoencoders

Scaling Training Process Transparency

Compute and infrastructure costs

Teacher Training Education in Nigeria using Artificial Intelligence

Making education worthwhile in Nigeria for teachers using AI

Scoping Developmental Interpretability

6-month funding for a team of researchers to assess a novel AI alignment research agenda that studies how structure forms in neural networks

MATS Funding

Help us support more research scholars!

Apollo Research: Scale up interpretability & behavioral model evals research

Hire 3 additional AI safety research engineers / scientists

Joseph Bloom - Independent AI Safety Research

Trajectory Models and Agent Simulators

Avoiding Incentives for Performative Prediction in AI

Discovering latent goals (mechanistic interpretability PhD salary)

6-month salary for interpretability research focusing on probing for goals and "agency" inside large language models

WhiteBox Research: Training Exclusively for Mechanistic Interpretability

1.9 FTE for 9 months to pilot a training program in Manila exclusively focused on Mechanistic Interpretability

Compute and other expenses for LLM alignment research

4 different projects (finding RLHF alignment failures, debate, improving CoT faithfulness, and model organisms)

Run five international hackathons on AI safety research

Six-month support for a Program Manager to organize and execute international AI safety hackathons with Apart Research

Agency and (Dis)Empowerment

6 months support for Damiano and Pietro to write a paper about (dis)empowerment with Jon Richens, Reuben Adams, Tom Everitt, and Victoria Krakovna.

Exploring novel research directions in prosaic AI alignment

3 month

Activation vector steering with BCI

Alignment Is Hard

Proving Computational Hardness of Verifying Alignment Desirata

Medical Expenses for CHAI PhD Student

Allow Berkeley PhD student to devote more time and focus to AI safety research and mentorship.

Cadenza Labs: AI Safety research group working on own interpretability agenda

We're a team of SERI-MATS alumni working on interpretability, seeking funding to continue our research after our LTFF grant ended.

AI-Plans.com Critique-a-Thon $500 Prize Fund Proposal

Introductory resources for Singular Learning Theory

Literature review or other introductory material for Singular Learning Theory targeted at AI alignment applications

AI Alignment Research Lab for Africa

AI Safety lab focusing on technical alignment and governance of AI in Africa and the Global South more broadly. We are a grassroots community-led research lab

Neuronpedia - AI Safety Game

Play the Game. Rack Up Points. Save Humanity?

How General and How Comprehensive Must AGI Be?

What is the probability that AGI is developed by January 1, 2043?

Why I think strong general AI is coming soon

By porby

CHAT Diplomacy

Advances in LLMs are a National Security Risk

Disagreements with bio anchors that lead to shorter timelines

By Marius Hobbhahn

‘Dissolving’ AI Risk

By Froolow. Parameter Uncertainty in AI Future Forecasting

AI X-Risk: Integrating on the Shoulders of Giants

By TD_Pilditch. Uncertainty, Structure, and Optimal Integration.

Odds of Artificial General Intelligence by 2043

By srhoades10

When can a mimic surprise you?

By David Johnston. Why generative models handle seemingly ill-posed problems

Artificial General Intelligence (AGI) timelines ignore the social factor at their peril

By Trevor Klee

AI will change the world, but won’t take it over by playing “3-dimensional chess”

By Boaz Barak & Ben Edelman

What would power-seeking, misaligned AGI actually do?

By Kiel Brennan-Marquez

Why do we post our AI safety plans on the Internet?

By Peter S. Park. It is plausible that AGI safety research should be assumed compromised once it is posted on the Internet.

[Funded] Gabriel Mukobi Summer Research

Evaluating Multipolar Cooperation Failures Between Autonomous Language Model Systems

Build an AI Safety Lab at Oxford University

Funding to establish a safety and interpretability lab within the Torr Vision Group (TVG) at Oxford

Building and maintaining the Alignment Ecosystem

12 months funding for 3 people to work full-time on projects supporting AI safety efforts

Jacques Thibodeau - Independent AI Safety Research

3 month salary for AI safety work on deconfusion and technical alignment.

Epistemology in Large Language Models

1-year salary for independent research to investigate how LLMs know what they know.

Retroactive funding for Don't Dismiss Simple Alignment Approaches

Empowering AI Governance - Grad School Costs Support for Technical AIS Research

9-month university tuition support for technical AI safety research focused on empowering AI governance interventions.

Blackbelt

A scalable, non-infohazardous way to quickly upskill via digestible, repeatable exercises from papers and workshops.

Graduate Students for Safe and Responsible AI (SAFARI)

An association for interdisciplinary interest in AI 

Understanding Memory Management in Transformers

We build a scalable "Automated Circuit Discovery" method and investigate "Cleanup Behavior" to advance the interpretability of transformer models.

Continued funding for a PhD in AI x-risk decision and risk analysis

Continuation of a previous grant to allow me to pursue a PhD in risk and decision analysis related to AI x-risks

Jaeson's Independent Alignment Research and work on Accelerating Alignment

Collective intelligence systems, Mechanism Design, and Accelerating Alignment

Ranked, Contributable Compendium of Alignment Plans - AI-plans.com

Making a simple, easy to read platorm, where alignment plans and their criticisms can be seen and ranked. Currently in Stage 1.

Model Interpretability on modFDTGPT2-XL, a partially-aligned model

6 months of work: Evaluating a variant of GPT2-XL that can simulate a shutdown activation, aiming to improve alignment theory & develop interpretability tools.