Thanks to everyone for the retrospective support!

As a final note, though unrelated to this funding, I recently moved to Oxford and recently started a new AI safety reading group with some students here.

I donated to this project some of the funds that were returned to me from other EA Community Choice projects that didn't meet their minimum funding bar.

As suggested in the description I am allocating $100 of my EA Community Choice funds to this project (+$100 of my own cash by mistake). My total EA Community Choice allocation was $400.

How I allocated my EA Community Choice funds: I initially planned to split my allocation among retrospective projects that have been impactful throughout my career. Unfortunately, many of the projects that have impacted my career are not represented on the platform. I reached out to some to encourage them to apply, but this didn't help---some did not respond, some considered applying but didn't get around to it, and some said they have other funding and so they didn't feel they needed to participate. So, I changed plans and decided to fund a mix of retrospective projects as well as promising prospective projects matching my interests and values, as follows:

• I allocated $149 for the retrospective impact of listed projects on my career: 80kH ($49, expecting them to do well in matching) and this project ($100).

• I allocated $101 to upskilling programs aimed at growing the AI Safety Research: Athena 2.0 $(81), MATS ($20, expecting them to do well in matching).

• I allocated $70 to content creators in AI Safety and Animal Advocacy: Michaël Trazzi ($50), Digital Animal ($20).

• I allocated $80 to infrastructure and events serving the AI Safety, AI Policy and Animal Advocacy communities: LEAH ($20), AIADM 2025 ($20), PauseAI ($20), Hive Slack ($20).

Cash donations: During the quadratic matching period I also donated $120 of my cash balance to various projects:

• I donated $20 to EANZ in recognition of their community contributions such as offering their own 1:1 career coaching program.

• I inadvertently donated an additional $100 to this project. This arose due to what appears to be a bug in self-donations whereby my cash balance was used instead of my charity balance the first time I tried to donate $100.

  • Even though these funds will be returned to my cash balance if the project is funded, this is not a no-op primarily since it will have a small (though diminished) impact on the quadratic matching and since the project might not have reached minimum funding counterfactually.

  • I am in discussion with Austin about revoking this donation, but in case it ends up staying, I apologise for the unintended disturbance to the matching and filtering procedures.

Donating a token amount to my neighbours down under.

The most compelling part of your project to me is that you provide your own 1:1 career coaching program. A local EA group's coaching program (before I had the guts to apply to 80KH) was pivotal to me earlier in my career. I appreciate that you are also in a position to provide more locally specialised knowledge to the people you coach. Please keep up the great work.

I'll flag that the upper limit $1000 of your estimate for the cost of a paid password manager seems very high (though I don't have any experience with password manager pricing for teams).

When prompted to think about community projects that have had a positive impact on my career, 80,000 Hours is always the first thing that comes to mind. Their career guide, problem profiles, and long-form podcast were really valuable when I was reflecting on which direction to take my career in. My career coach Huon has also offered me valuable support and resources since our initial call. I'm sure many others in the community can point to 80KH as a positive influence on their career. I think it meets the goals of the EA Community Choice funding program to recognise and reward 80KH's history of impactful community contributions and I am glad to be a small part of that.

What progress have you made since your last update?

The 6 weeks were mostly spent on three projects:

1. Planning out the literature review mentioned in the grant.

2. Contributing to SLT research related to validating the developmental interpretability research agenda.

3. Learning to use TPUs and PyTorch/XLA, and teaching this to the developmental interpretability research community.

Project (1) is not yet complete. I made some progress recruiting co-authors and planning a literature review. However, we didn't make much progress writing the actual review: all we have is a review sketch. I consider this an overall failure. (I do still think I can eventually complete the review in the near future without additional funding. More below.)

Project (2) was much more successful. Over the 6 weeks I successfully replicated an in-context learning experiment based on prior work, and this went on to become the foundation for a collaborative research project leading to a paper "The Developmental Landscape of In-Context Learning" (under review, arXiv preprint, associated lesswrong post).

Project (3) was also successful. It culminated in me delivering a tutorial for the local alignment research community on how to accelerate research experiments with free TPUs from Google (tutorial, recording). Since then I have built on this experience by learning JAX and I am planning a JAX course for my research group for later this year.

In addition to these outcomes, I kept a weekly research journal for the first few weeks of the project, which contains some more detailed commentary.

Immediately after the 6 weeks, I transitioned to full time research work at Krueger AI Safety Lab and have been working since then on other projects (plus continuing to collaborate on project (2) with the developmental interpretability research community).

Overall evaluation

My main personal goal for the project was to fill a 6 week gap between paid work opportunities, and contribute in some positive ways to alignment research. I feel that this broader goal was achieved through projects (2) and (3) described above.

This broader motivation was part of my initial pitch to Adam Gleave, the regrantor who awarded my funding. However, Adam was most excited about the literature review aspect of my proposal, and chose to emphasise that aspect of the project in his writeup for this Manifund proposal.

Since I didn't manage to produced a literature review in the 6 weeks and also haven't managed to produce one since then, I consider this project to have failed.

What are your next steps?

However, I still think the project is recoverable. Since setting out to produce the literature review, there have been three publications that partially fill the need for this resource for the SLT/alignment research community.

• A summary paper by the founder of SLT: Watanabe, 2022, "Recent Advances in Algebraic Geometry and Bayesian Statistics"

• A methods paper including a good technical introduction to different levels of singularity, a central concept for SLT: Lau et al., 2023, "Quantifying Degeneracy in Singular Models via the Learning Coefficient"

• Another methods paper including a good technical introduction to different definitions of the learning coefficient, a central quantity in SLT: Furman and Lau, 2024, "Estimating the Local Learning Coefficient at Scale".

These resources are enough to hold the community over but there is still a need for a comprehensive accessible technical introduction to the theory. Moreover, the SLT/developmental interpretability community has made some progress establishing the viability of this research direction, and so the need for such a resource within the wider alignment community is still present.

Working with some of the authors of the latter two articles I mentioned above, I still plan to write up the literature review we have planned. To do so I am waiting for an appropriate opportunity. I believe I will have an opportunity in the latter half of 2024 when my KASL project comes to a conclusion, in the few months before and after the start of my DPhil.

Is there anything others could help you with?

No.

In particular, I do not currently require more funding to find time to work on this.

If there is anyone who is interested in collaborating on the literature review, however, feel free to reach out.