Thank you so much for the thoughtful reflections and your support, Austin - it genuinely means a lot to our team.

Regarding Esben's transition, we want to provide more context to help clarify how we're approaching this inflection point. Esben has transitioned to a position as chairman of the board and is the interim CEO with the main task of finding a strong replacement together with other board members and the rest of the leadership team (Jason and Jaime).

While Esben is building Seldon to complement Apart's work as a for-profit sister organization, we view the efforts as mission-aligned but distinctly scoped: Apart remains focused on accessible, public-good-driven AI safety research and mid-career talent incubation. The launch of Seldon creates an opportunity for even stronger synergies between mission-driven research and adequate speed-of-scaling of important AGI security technology.

With recent strategy retreats in conjunction with the fundraiser, the whole team is excited for the changes that are currently happening: Reprioritization of key projects, deploying learnings from our impact evaluations, restructuring our academic involvement for further direct impact, building out our Sprints team to evolve higher frequency sprints and quarterly frontier research workshops, and a reorganization of our org structure to accommodate all these changes and more.

As mentioned in our update, we are excited to keep all our supporters up-to-date and involved in a dedicated newsletter as we maximize the impact of every dollar donated during this campaign. Again, thank you and everyone who's taken part in our journey.

- The Apart Research Team

We're in the final countdown of our fundraising campaign now. We have raised a total of $619,921 until now which takes us past our first four milestones towards our final goal of $954,800.

🫶 Massive thanks from our whole team to everyone who contributed:

...and many more

Notable progress since we launched this campaign:

• Published the nearly 20-page Impact Report, a thorough overview of how Apart has impacted the world through talent development and direct research.

• Hosted two hackathons; the Mechanistic Router Interpretability and ControlAI events

• Received 158 testimonials from individuals around the globe who have directly benefitted from our Apart, in addition to 10+ from other organizations that we have collaborated with

• Received blogpost submissions to the Studio from top projects at three hackathons, and accepted some to continue their work under the guidance of mentors from both Apart, Martian, and ControlAI

• 5 blogposts have been published after submitting to the Studio and receiving our feedback, with more coming out in the very near future.

• Accepted 3 more projects into the Apart Lab Fellowship, all of which intend to submit to NeurIPS 2025 workshops in late August.

You can read in-depth reports of the progress during the campaign in our newsletter:

• 2.5 week update: Sharing that we got past the first milestone with the Richard Ngo's generous donation.

• 3.5-week update: Announcing that our support had now blown past our fourth milestone and extending our campaign by another 25 days.

• 7-week update: The final update released together with this progress update.

During the fundraiser, we've hosted a virtual retreat with the team to consolidate all our learnings and bring them into an updated and stronger Apart Research. We're really excited to share more with you in our donor newsletter during the next months.

"How can I help??"

Every dollar counts and our fundraiser is still going! Join more than 70 other private donors to keep Apart and AI safety going for the long-term.

Thank you.

@AntonMakiievskyi

TLDR; no big funders have retracted funding, we have undersold our impact in grant applications by not focusing on the metrics that we now know work best, our network within SF isn't as strong as other orgs, and the AIS funding ecosystem face general problems. See a longer response to this question here.

View our new application here: https://manifund.org/projects/keep-apart-research-going-global-ai-safety-research--talent-pipeline

Apart Research Progress Update - September 2024

Key Achievements

1. Expanded Participation: Our Sprints have gained significant traction in 2024, with over 1000 signups and approximately 100 team submissions at our events.

2. Growth in Lab Fellows and Projects: The Apart Lab has seen rapid expansion throughout the year. We've progressed from 17 fellows working on 7 projects in Q1 to our current cohort of 37 fellows collaborating on 12 projects in Q3, bringing the total to over 75 fellows (incl. Q2). We believe that this steady growth showcases the increasing demand for our program and our capacity to support more ambitious research endeavors.

3. Team Expansion:

   • Welcomed two experienced core team members:

     • Natalia Pérez-Campanero Antolín (former program manager for the Royal Society) - Leading research project management at Apart Lab

     • Archana Vaidheeswaran (former board member of Women Who Code) - Managing community and events for hackathons

   • Expanded our in-house mentorship capacity to support the growing number of fellows

4. Research Impact: Several research papers from our fellows are currently under review at AI and machine learning venues. A selection of these papers:

   • Classifies whether a model is cheating on evaluations, i.e. if the model has been trained on a benchmark

   • Identifies deceptive patterns in fine-tuned LLM conversation, inspired by dark patterns in UI design, finding that Meta's models are manipulative while Anthropic's are not

   • Evaluates cyber offense capabilities from LLM performance on realistic challenges conforming to industry-standard cyber offense risk classifications, showing that models can autonomously perform complex cyber offense operations

   • Examines how to mitigate infectious jailbreaks in realistic multi-agent systems

Ongoing Initiatives

1. Increasing participation of high-potential talent in hackathons, with a focus on underrepresented communities including senior tech professionals, cybersecurity experts, and individuals with hard science backgrounds

2. Enhancing the impact of lab research projects by supporting teams in pursuing ambitious and potentially groundbreaking research ideas

3. Developing an AI safety incubator to facilitate the transition of high-impact research ideas into real-world solutions

4. Actively recruiting a research engineer to further support our vision of producing impactful research projects

5. Onboarded a research communications specialist that can help us make all of our above progress more visible to followers of Apart

If you are interested in supporting us on our journey, feel free to use this page, or contact finn@apartresearch.com.

Project update

Since the submission on December 18 2023, we have...

• Reshaped our leadership with a renewed focus on strategy and organizational development, with Fazl Barez changing position from Co-director to Research Advisor and Jason Hoelscher-Obermaier stepping into the Co-director position.

• Onboarded 16 new fellows working on 7 different research projects. This new cohort stems from 9 different countries (Germany, the UK, Denmark, Sweden, the US, Spain, France, India, and Vietnam) and spans a broad spectrum of experience levels:

  • 2 PhDs (Deep Learning, Astrophysics)

  • 2 PhD students (ML, Statistics)

  • 5 with Master's degrees (Computer Science, Computational Modeling, Complex Systems, Social Science, and Logic, Philosophy and History of Science)

  • 2 Master's students (AI & Data Science, Human-Centered AI)

  • 2 Bachelor's graduates (Mechatronics, Epidemiology)

  • 3 Bachelor's students (Computer Science, Data Science, Mathematics)

• Had more research coming out of Apart Lab:

  • Published on Neuroplasticity in LLMs (still under review; see X)

  • Understanding Addition in Transformers accepted at ICLR 2024

• Collaborated on the Sleeper Agents paper published by Anthropic

Thank you for the kind words and the generous evaluation request, @Austin!

Thank you for the review and the positive words. We'll address your points below. Please let us know if anything remains unclear. We are more than happy to provide more details.

I don't understand why other people haven't funded this yet

Apart has, for much of 2023, been a very small team (2 FTE) fully focused on experimenting with and improving the research sprints and lab fellowship. We simply did not have the capacity to scale up the program and the corresponding fundraising so far. Now, with an additional leadership team member and two half-time assistants on board (4 FTE), we finally have the capacity to tackle scaling up and funding seriously.

Maybe this application is exaggerating stuff?

We have been quite careful not to overstate our case in this proposal and needless to say, all numbers presented are correct. Let us know if you need any more details or evidence.

Maybe the organization adds another step in the chain to impact, and it would be more efficient to fund individual people instead?

We think that a peer-reviewed publication (particularly a first-author publication) is close to a necessity for most research careers and that aspiring AI safety researchers are much less likely to achieve this on the same swift time scale without the right structure and guidance.

Another added benefit of funding us versus funding individual people directly is that, by directly interacting with a very large number of individuals through our research sprints, we can identify and carefully select talented people who would not be obvious targets for other funders, and thereby uncover talent that would otherwise be overlooked.

Maybe the biggest one: how do I know the success is counterfactual? Say that someone participated in a hackathon/fellowship/etc, and then later got a research position in some Oxford lab. How do I know that the person wouldn't have gotten something similarly impressive in the absence of your organization?

As mentioned above, we believe that (a) having a paper (especially as first author) accepted/submitted for peer-reviewed AI safety publication will often be critical to our target demographic's career trajectory and (b) the people we support are less likely to convert a hackathon project into a paper worthy of submission/acceptance at a top venue without our support. Evidence for (a) comes from our fellows in interview processes who report that most of their time in interviews have been discussions on their Apart Lab research project (with one interviewer remarking it as "a strong part of [the] resume" to go through the program). Evidence for (b) is harder to come by but self-reports of our fellows suggest that they believe they could not have achieved a similar output without the help of Apart Lab.

@NeelNanda That is correct, thank you for the clarification, Neel! The text has been updated and we're also happy to say that there are now 7 papers in review, so we're crossing fingers for the research teams :D