@Austin Thanks!

I should clarify: I don't think "presenting at The Curve" slowed things down per se.

I think in a lot of ways it was positive (eg. in having a clear intermediary deadline, and I also met people there who introduced me to people who ended up donating a significant amount).

What slowed things down a little bit was 1) not starting to work with people I would want to work on the final product with from the start 2) "software decisions" where we worked in not-very modular ways before the conference, which implies we had to duplicate a lot of the work we did there.

I think there would have been ways to work in a more modular / long-term way from the start, and I take responsibility there.

Also, having a "MVP" (or "rough cut" in the movie world) was also instrumental in contracting more experienced folks, so having that "conference cut" from the Curve was maybe a necessary step in hiring more senior people after all.

SB-1047 Documentary Post-mortem

This documentary took 27 weeks and $157k instead of my planned 6 weeks and $55k. Here's what I learned about documentary production

Total funding received: ~$143k ($119k from this grant, $4k from Ryan Kidd's regrant on another project, and $20k from the Future of Life Institute).

Total money spent: $157k

In terms of timeline, here is the rough breakdown month-per-month:
- Sep / October (production): Filming of the Documentary. Manifund project is created.
- November (rough cut): I work with one editor to go through our entire footage and get a first rough cut of the documentary that was presented at The Curve.
- December-January (final cut - one editor): I interview multiple potential editors that would work on the final cut, and decide on one candidate who would be the one that does most of the editing (from December to February).
- February-March (final cut - 7 Full-Time Equivalents): I work with a total of 7 seasoned professionals (working full-time) to have a finished documentary by the end of March. This is the most capital intensive period of the post-production phase.
- April: We wait back to hear from multiple distributors about whether they would be interested in publishing the documentary on their platform. Multiple outlets show strong interest (New York Times Op-Docs or Wired) but the content of the documentary doesn't fit publication policies.
- May: The documentary is published on May 5th.

Breaking down how the money was spent:
- Editing was the largest part of the expenses, since I ended up paying for a total of 4 different editors, that have worked from November to March included. From February to March I had multiple editors working on the documentary in parallel.
- Motion graphics was the second largest item, with two people working on motion graphics in February and March
- In terms of music & sound, the documentary used custom music made by a composer, with some of the songs played by real instruments, but also required the work of a seasoned sound mixer, which is why this is the third most expensive item.
- The director salary ended up representing only ~9% of the total expenses, since I had originally planned to pay myself $15k for 10 weeks, but the project ended up taking ~27 weeks instead.

But why did the project end up taking 27 weeks instead of 6 weeks?

- Short answer: I ended up getting more funding than I originally had asked for on Manifund, and had to hire many different professionals with that funding. Having to present something intermediary at the conference "The Curve" potentially slowed us down. And a lot of the steps had to happen one after the order, including all of the fundraising, hiring, multiple steps of post-production, on top of the distribution of phase where we had to wait to hear back from potential distributors. All of this considered made that the movie take ~5 months to be ready (and 6 months to be out) instead of 6 weeks.

- Long answer (breaking out month by month):
--> In November I was offered to do a first screening of the documentary at the AI conference The Curve, which I imagined to be a great way to present a first draft of the movie as we were working on it. However, in order to get this first draft done on time (for the conference), I had to hire an editor with whom I did not end up working with throughout the project, and most of the work was rushed using a software that we did not use in the future, meaning a lot of the work we did in November was not directly re-usable in later months.
--> In December a lot of time was spent trying to find a film editor that would be willing to work full-time with us on the project. In the end I was really satisfied with the editor we ended up with, but then came the end of year Holidays, so not a lot of editing happened then. On top of that, our funding was constrained enough that I would not have been able to hire more people to work on this, which would make my only editor much less productive compared to other environments where he would have an assistant editor, an archival producer or motion graphics person to help him.
--> In January, an important bottleneck was still not being able to hire more people because of funding constraints. But also the wildfires happening in LA, which directly impacted our main editor. When funding was secured, a lot of time was spent finding the right people that would join in February, including a composer, an archival producer, two motion graphic designers, and two editors. Some other things that slowed us down here was that the software and organisation decisions we had taken in earlier months were starting to slow us down, so had to do some large refactoring, and transition software.
--> In February most of the editing for the documentary happened, with 7 people working full-time on it. By the end of the month, we were able to have something that was almost done in terms of story and bytes.
--> In March most of the music, motion graphics and sound happened, because all of this work required the edit to be locked-in (also called "picture lock").
--> In April the movie was ready to be published, but we spent most of the month waiting for the potential distributors (NYT, Wired, LA Times, etc.) to get back to us. Other work involved upscaling and marketing.
--> In May the documentary was published on Youtube at the beginning of the month.

Impact:
- The documentary achieved 2,500 hours of watch time (25% retention rate) across 20k YouTube views, and 100k views on X
- The documentary got presented at AI Conference "The Curve" and I'm in talks to present it at the UK Parliament (depending on funding).
- I've heard from multiple news outlets (including Wired, NYT) or filmmakers that the film was very well edited and deserved to be on streaming platforms.
- As a consequence of that, I've recently started submitting the documentary to movie festivals.
- There is still potential work I want to be doing involving directly sending / showing this documentary to lawmakers in the UK / US (depending on funding).

What I would do differently next-time:
- I would try to work with a fixed timeline and realistic corresponding budget. A lot of the problems I've had with this project was that I wanted to finish a documentary quickly with not enough staff, and had to wait to fundraise more before hiring more people. Once I had enough staff, getting people to work quickly and finish the project was much easier.
- I would dedicate more funding to marketing / distribution, and go through the movie festival route first. A lot of the issues I was having towards the end for distribution is that getting potential distributors to watch your documentary is a process that takes time, and a lot more time needs to be allocated to send your movie through the movie festival pipeline, to get increased coverage down the line. I think one of the reasons the movie got less views on Youtube than expected is that people do not expect to watch a movie on Youtube and the content would have performed better on a streaming platform where they'd expect a documentary.
- In terms of the movie itself, I would spend more time at the start of the movie discussing exactly what is AI Safety, why AI is a big deal, what AI regulation is, talking to a wider audience. Given that the documentary was published on Youtube, it is necessary to explain those terms in a much more pedagogical way so that most people that had not heard of SB-1047 would have been able to understand what was happening and why they should care.
- I would focus on figuring out distribution first. I am already now in touch with distributors that would be happy to work with me from the beginning of the project next time, instead of having to convince later to distribute the project. This guarantees funding and distribution from the start.
- I would start with all of my team already figured out. Now that I have already talked to >50 seasoned professionals and contracted 16 of them, I would be able to get started with professionals I trust from day one for my next project, which would have saved me about 2 months on this project.
- I would also work in person instead of remotely, which I think would have saved me one month.

closing the project

closing project

@Connoraxiotes curious: how much did flying to NYC and having 9/10 people on set cost?

With that burn, how many more interviews can you shoot?

Update: May 5th - Documentary is out

OK so this took way longer than expected to get out.

I'll post another update later detailing everything that happened since November but for now the full documentary is available to watch on X and Youtube.

Thanks again to everyone who donated! Your name should be in the credits according to your donation tier.

What progress have you made since your last update?

Nov 26th Update:
- After 4 weeks of editing, hiring multiple editors & other contractors (sound mixing, production coordinator, video labeling), we presented a 1h05m rough cut of the SB-1047 documentary at "The Curve" on Sat 23
- We observed an increase of 1.4 points of understanding after watching the documentary, which satisfies our goal or increasing the understanding of different positions held on SB-1047 (we polled 21 participants before and after the screening)

- We observe a weak correlation between the initial position of participants on the bill, and how likely they are to recommend the documentary to a friend. The fact that the correlation is weak is positive, showcasing that the documentary was recommended (& not recommended) by people with different positions.

What are your next steps?

- I no longer believe that it is possible to finish this project in "6 weeks" as previously proposed for the funding level that we reached
- My current estimate for when the movie will be out is now at best late January 2025, if not early February
- I believe that many of the key players who would have benefited from seeing the documentary earlier rather than later have already benefited from watching the rough cut at the curve, or will hear about it from there and could see it upon request or at targeted screenings.
- My new goal is now to have a final cut published before a new bill is announced or just after it is announced (estimated date for when the next bill is announced: February)

Is there anything others could help you with?

- My previous budget estimation were too optimistic and did not take into account the cost & time of motion graphics / animation, the cost of hiring a composer / music supervisor, and the cost of hiring an experienced video editor to go from a rough cut to a final cut for 2+ months (which seems to be the minimum I am currently being quoted for).
- Therefore, we are still funding constrained, and I expect that any funding above $55k will be invested towards paying contractors for the post-production (say for animation, music and video editing), with video editing and animation being the two main costs, and music being slightly lower. (Note that video editing was to some extent included in the original manifund project, but animation was not included at all, so this is where I'd expect most extra marginal funding to go).

Offering a token of appreciation since I learned a lot reading 80k's blog, listening to Rob Wiblin or doing a coaching call. Same as Neel, donating a small amount as token of appreciation, since this is already a large org,

Token of appreciation since I have personally benefited from Lightcone's work when visiting Berkeley.

A lot of people I interact with regularly have done the MATS program and received a lot of value from it.

Similar to Neel below I want to give a token of appreciation, voting for the quadratic funding, given that MATS is a large funding opportunity.

update: I've now thought more about it, watched more videos, and the videos seem to be getting good engagement and liron has been posting these quite regularly, demonstrating his willingness to grow.

I think the "reacting to video" aspect is a niche that is not currently filled and is worth doing.

I don't especially like focusing too much on basic arguments regarding "optimizers" or "paperclips" (mentioned in the Shapiro and Bret Weinstein videos I watched) and would prefer more recent terminology (say discussing things specific to language models / SoTA and how the current paradigm could become risky), but I want to donate some amount now to show some support, possibly adding more later as I get more evidence.

@GauravYadav This looks much better, thanks.

I've now watched the two videos you linked in full and gave feedback through the appropriate channels.

Offering a small amount for now to signal support. Could imagine doing a bigger donation, especially if more donations happen (to bring the project closer to minimum funding), or if there's new evidence that comes up (say you publish a new video before the "24 days left to contribute" that I think is especially promising).

Note: donating a small amount now to signal support, might add more later when I have more clarity on how I want to spread my donations.

I think a world in which no advocacy for an AI Pause would be worse so I am glad that they exist and organize protests.

Main reasons I'm excited
- I think 2.1k / volunteer is quite cheap for the amount of work a volunteer could do a year
- I've interviewed Holly Elmore from PauseAI US who I overall trust with the project
- I think their discord is already quite active, and the 2k "members" (discord members?) and 100 registered volunteers show some decent engagement

Some reservations
- When I look at the protest lists (https://pauseai.info/protests), it seems that there hasn't been that many protests in 2024 organized by PauseAI.
- I don't think any PauseAI protest has reached enough participation to have say 100+ people showing up at one location, which I think would be more impressive than what I've currently seen

Overall, I'm glad that PauseAI exists and appreciate all of the efforts that are being done in that space.

@liron Thanks for the quick & detailed answers.

Since I wrote that comment I've watched the beginning of most of your current videos, and I'd like to add a few updates for me:

• I did look at the engagement (comments & likes) on current videos and it does seem like a positive signal for having a strong core audience. I guess to get the full picture it would be good to have the number of dislikes to see if it's just that the video topics are controversial, but from the comments it seems people do seem to value your work, which is again a good sign.

• it seems that most of the content is "Liron react" rather than "Doom debate", meaning you're reacting to what someone said instead of debating. I think debates are harder to do since you'd need to have people agree to debate you and schedule it, and it would also be more interesting to watch. But reacts are things that don't exist in the space so fill the "commentary" niche and is possibly a way to get a debate afterwards.

• the production quality is quite good on your side, though say when you're reacting to david shapiro he his only inside a circle (i guess for copyright issues?) which I think could slightly be improved

Right now, given your channel is quite new and the engagement, I think I've updated towards the potential growth you're described being possible.

I guess last conflicted thoughts I'd have are on the theory of change. If say more david shapiro folks watch your reacts / debates, or more generally people do understand the basic arguments for why "doom" is likely, what do you expect them to do as a result?

My understanding is that your audience would most likely not be that technical, so the goal is potentially to have people more concerned about AI risk to say shift the public opinion on AI legislation, or do call to actions for the PauseAI movement? If so would be good to have data on that (eg a form on how much their perspectives have shifted or how many PauseAI discord signups from your channel).

Anyway, thanks again for the thorough answer. I'll try watching a full Liron react and potentially see your final reply to this before coming up with a donation decision.

I have watched some of the first two initial videos and I think this is really promising.

Responsible Scaling Policies and SB-1047 are two things that I think would be great to have an easy-to-digest video format and I am glad Gaurav already did some of the work with his two initial videos.

Some reservations I have:
- 1. I think at this stage your setup looks fine and I'd be more interested in seeing what kind of results you get without a fancy setup than say spending $ on expensive audio / camera equipment.
- 2. More generally, if I donate say $50 I'd prefer this $50 to go directly to one hour of your time than say contribute for a $1k camera or similar. Today our phones are generally as good or better than $1k cameras.
- 3. However, I could see a version of this where you could break down exactly what kind of cheap green screen + light + software you'd want to use, and say like "I really think that I'd need $300 to buy this $200 light, $40 green screen, and subscribe to Adobe Premiere Pro for 3 months ($20 * 3) which would make my life really easier as a friend of mine could teach me Premiere Pro."
- 4. I think the goals in "What are this project's goals? How will you achieve them?" are not quantifiable as is.

Example of quantiable goals:
"""
My goal is to try to see if I could produce valuable explainers on AI Governance. I plan to make one video on compute governance, one video on the EU AI Act, and one final video on another legislative update as it occurs, or if nothing important happens I'd be default talk about [Insert video idea you want to do anyway]. If by [date] I don't get [X people telling me they found it more than 8/10 valuable] or [Y views] then I'd consider I am not a good fit.].
"""

I should say that really the two videos were quite engaging and I ended up watching a good chunk (few minutes), and that overall this looks quite promising. Simply looking for more clarity on the precise needs & concrete plan.

I think communicating AI risk through debate with a podcast & youtube channel is valuable. I have seen some episodes of Liron debating with people I'd qualify as "tech optimists" (George Hotz, Theo Jaffe, Guillaume Verdon) and I can confirm that he has the "stamina" and patience to go through these debates (as he mentions here).

My current main reservations are:
- framing it as a "doom debate" or a confrontaional "doomers vs non doomers" which could potentially cause harm (making the movement be seen more like an advesarial apocalyptic cult rather than a technical field)
- one of his debate with Guillaume Verdon (around his startup) might have been counterproductive (he repeated the same question about "inputs / outputs", which I think was a fair question, but this didn't signal to the audience that he was engaging with the actual plan from Guillaume, and was criticized on twitter for that).

A few clarifying questions on the actual text:

Just two months into the project, we're already getting 1k+ views and listens per episode.

How many listens are you getting in the "views and listens"? Is it 1k+ views + listens in average? What is, say, the average watchtime per episode? What is the average view duration (in %)?

We're now at the point where we can convert money into faster audience growth using obvious low-hanging-fruit growth tactics.

What are these "obvious" low-hanging fruit growth tactics?

Plus, the topic of AI risk is growing increasingly mainstream. So I bet the potential audience who wants to consume my content about AI x-risk really is in the millions.

How did you end up with "in the millions" here?

The viewers-per-episode metric will soon hit tens of thousands

What data supports that? What do you mean by soon?

The most likely failure mode is if my content doesn't resonate with a large audience, making it hard to get beyond 25k+ views and listens per episode, such that I give up on the 250k+ views per episode which I currently think is a realistic long-term goal.

When will you decide whether you're in that failure mode or not? Why did you decide on 250k+ views episode as a long-term goal? What makes you think it is a "realistic long-term goal", and what do you mean by "long-term" here?

===

Final note: I think Liron is doing hard work in the space and I think this has the potential to be turned into something impactful. The previous questions were simply to get more clarity on some of the numbers that were mentioned before, and the potential pitfalls I've mentioned in my reservations. I think Liron has proven to have the consistency and patience to hold these debates regularly, and I overall encourage the project, assuming some of the reservations I have are addressed in the future.

What progress have you made since your last update?

Main updates since I started the project
- March: published popular video: "2024: the age of AGI"
- April: edited some Ethan Perez interview
- April-May: I record & edit my AI therapist series
- May-June (daily uploads): I published 23 videos, including 8 paper walkthroughs, 4 animated explainers and one collab (highlight). More below.
- August: I record & edit an interview with Owain Evans (will be published in the next few days).

Detailed breakdown of the 23 videos from my daily uploads:

• paper walkthroughs

  • The Full Takeoff Model series (3 episodes)

  • Anthropic’s scaling monosemanticity

  • The case for ensuring powerful AI's are controlled

  • Safety Cases: How to Justify the Safety of Advanced AI Systems

• sleeper agent series

  • walkthrough of the sleeper agent paper

  • walkthrough of the sleeper agent update

  • four-part series on sleeper agents

• nathan labenz collab

  • one interview with nathan labenz from cognitive revolution (highlight)

  • full video includes a crosspost of nathan labenz’s episode with adam gleave

• other alignment videos

  • discussion of previous coding projects related i’ve made about alignment

  • mental health advice when working on ai risk

  • take on leopold’s situational awareness

• the AI therapist series (about AI's impact)

  • talked to an AI therapist every day for seven days (35m of edited footage)

  • compared using vapi.ai to hume.ai, claude opus to gpt-4o

What are your next steps?

August: In the next week or so I will be publishing an interview with Owain Evans.

September-?: I'm flying to San Francisco in to meet people working full-time on AI Safety, so I can have a better sense of what research to cover next with more paper walkthroughs, and schedule some interviews there in person.

After that, my main focus will be editing the recorded in-person interviews, recording paper walkthroughs, video explainers, remote interviews, and possibly some more video adapatations of fiction (like this one).

Is there anything others could help you with?

Any additional funding that goes into this grant will directly go into reimbursing my travel expenses in the Bay, and allow me to stay longer and record more in-person interviews.

I am happy to get connected with people to talk to (for an interview, or because they wrote an important paper I should cover).