Project summary

6-week salary to contribute to foundational resources in the nascent field of "Singular Learning Theory x AI Alignment"

Project goals

Produce a literature review on Singular Learning Theory, as a foundational resource to help orient newcomers to the field.

How will this funding be used?

Salary for Matthew Farrugia-Roberts during the 6 week period (annualized $91,260/year).

What is the recipient's track record on similar projects?

A detailed survey of the literature has already been completed by Matthew as part of his MS thesis, but has yet to be written up. This substantial preparation will enable the work to be completed relatively quickly.

Matthew has published a joint first-author theoretical ML paper in ICML, a top-teir venue, and completed an MS thesis at the University of Melbourne with a mark of 95+, reserved for students 'typically encountered only a handful of times throughout an academic career'.

How could this project be actively harmful?

Singular Learning Theory provides a potential path to better understanding ML systems. Although better understanding of systems can be helpful for safety, it could also lead to insights improving the efficiency of ML training procedures potentially enabling more powerful systems to be trained sooner without a corresponding improvement in alignment. This risk holds for science of deep learning and interpretability methods in general; on balance, the benefits seem to outweigh the risks, but it is important to at least remain aware of the downside.

Singular Learning Theory is a speculative research direction. Foundational resources will enable more people to on-board to it. However, there's a possibility it's a dead-end and these people would have been better spending their time elsewhere. On balance, it seems worth exploring Singular Learning Theory and enabling newcomers to more rapidly on-board should decrease the overall cost to exploring this direction if resources are allocated efficiently.

What other funding is this person or project getting?

No other funding during this period. Matthew was previously receiving an RA salary for a previous project, and will receive an RA salary for a new project after completion of this six week project.