I worked with @aashkapatel before and know how passionate she is about AI safety and literacy. Super excited to see where this project goes!

@pvasholz Thanks 😊

Project Update – Execution Phase Underway

Following the last update, the remaining operational bottlenecks have been cleared and the project has moved into active execution.

• Stable platform build is live: PBROC at Michigan State has begun transforming the landing pad vector into petunia, moving the core platform from design into biological integration.

• Agrobacterium pipeline cleared: Phytotron work and infiltration approval are now in place, removing the main gating step from the previous phase and enabling in-house experiments to proceed.

• Parallel expression testing ready: The full initial construct set is now in hand, including landing pad, regulatory, and enzymatic designs, enabling immediate testing of GFP and pigment expression.

• Plant system locked in: Mitchell petunia seed has been secured and plants are established, fixing the background for the first proof of concept.

• Iteration capability established: Bench space is now secured for plasmid cloning, enabling rapid generation of design variants without relying entirely on external synthesis.

The work is now focused on generating experimental data, with stable transformation progressing at PBROC while transient testing begins in parallel to compress iteration time.

Next step is generating initial expression data. I will share results once they are clear.

Thanks @Austin. It appears it was simple enough to edit myself!

To start! Thanks Joel! I very sincerely appreciate the candid and clearly worded feedback. I spent a bunch of time reflecting on this today.

With this in mind, I've drafted two budget tiers today with the information I have. This would be spent over two workstreams.

Our initial bridging fund was allocated to 2 months, with lower API credit and contractor asks, while we scoped future work. These tiers would have us operational for 6 months, and greatly increase the API credit and contractor budgets.

Tier 1: $300K

Workstream 1 — Cyber & Control:

- Salaries: $100K

- Model API credits: $50K

- Red team contractors / human experts: $50K

Workstream 2 — Pragmatic Interpretability:

- Salary + compute: $60K

Backpay + Overheads + Buffer: $40K

Tier 2: $600K

Workstream 1 — Cyber & Control:

- Salaries: $160K (supports an additional senior cyber researcher hire)

- Model API credits: $150K

- Red team contractors / human experts: $150K

Workstream 2 — Pragmatic Interpretability:

- Salary + compute: $60K

Backpay + Overheads + Buffer: $40K

What we're thinking

Workstream 2 is nascent and exploratory. I am particularly excited about upcoming work exploring self-steering via activation probes with potential applications in model welfare, model preferences and speculatively alignment stability. This workstream is a first step to support a core mission of Lyptus. Creating and growing institutional capacity, support and direction for untapped talent in Australia.

Workstream 1 is our larger investment. Through the course of our own work (and further so with the recent Mythos model card), it's plainly clear to us that cyber evaluations are materially not keeping up with capability growth. Honestly, the accelerating progress did take me by surprise and our dataset would certainly be saturated at higher token budgets. That said, while these steepening trendlines show no signs of stopping, we do believe current absolute capabilities can be overstated. Full red teaming engagements against even moderate blue teams are a very different proposition to isolated benchmark tasks.

To ground this with an illustrative example of where our thinking is. We're currently scoping the tractability of scaling up the ideas presented in the Artemis paper. Evaluating AI agents via real red teaming engagements and comparing their performance to professional human red teamers. This captures the full end-to-end scope of an attacker against real production organisations.

This is clearly operationally difficult. Though early conversations with senior offensive security leadership suggest it is tractable. We believe with the right people involved we can design incentive structures that work for all parties. A typical engagement with two professionals runs around $30K USD per week. We would ramp up starting with smaller targets.

This is speculative. We are having early conversations, and we are genuinely in a phase of figuring things out. It is our current leading candidate project but not our only option. What it represents is our broader interest in human grounded studies and science communication. Results that are quickly interpretable beyond the AI security ecosystem. We think this suits our strengths better than building cyber benchmark tasks, which requires deeper offensive security expertise than we have in house.

What the tiers mean

At tier 1 we would more likely prioritise budget friendly work like human-grounded covert capability on cyber tasks or human-grounded studies on cyber stealth benchmarks. If we had strong multi-organisation traction on the red teaming idea we would still pursue it, but budget constraints have a way of subtly pressuring priorities against more ambitious work.

At tier 2 the more ambitious work becomes realistic. A senior cyber researcher hire, 2-3 red teaming engagements at small to medium scale with client co-financing, and enough API credits to properly test models at adequate token budgets.

Other ideas in this space would require internal budget rescoping but the broad structure holds.

Thank you for your kind words, @ridakadri14 :)

All the best @aashkapatel, the project is really impressive and inspiring !!!

Far-UVC is a technology with amazing potential, and I'd love to see more deployments for generating efficacy and safety data.

Approving this project! I'm glad that @joel_bkr and others at METR are so excited for Sean's work, and from my perspective, this kind of active, expert-led, fast grantmaking is where the regranting mechanism shines.

I also appreciate Joel taking the time to lay out his considerations in making this grant, which are hopefully helpful for Sean and others in similar positions. (And @seanpetersau, insofar as you would be open to accepting more funding through this Manifund project, let me know; we're very happy to increase the funding goal on your behalf!)

(Noting that after conversation with Nate it's now less obvious to me that Lyptus should focus on dramatically harder cyber tasks in particular. Although I continue to think that there is a very, very, very high premium on dramatically harder tasks in general. Want to leave the focus more flexible depending on Lyptus' taste.)

Here's my updated sense of where Sean and co. are at, extremely heavily influenced by colleagues, especially Nate Rush:

1. My original comment on Sean continues to stand up well today. He appears to be among the very most competent people building on top of METR work.

2. The work he links above was competently executed. Their baselines/time estimates are reasonable (less common than you might think), they have a high attention-to-detail vibe, the write-up is fairly impressive and not overclaiming, etc. The work compares favorably to UK AISI's analogous work.

3. Unfortunately, best guess is that all of their tasks would saturate for most recent models.

Coming out of this:

1. I think it's a shame that Sean needed to dip into personal funds to pay for this project.

2. I consider it ~totally obvious that Lyptus' work is worthy of further funding.

3. I really, really, really, really, really want Lyptus to (1) throw more AI labor at their problems and (2) create dramatically harder cyber tasks. I think the end work will be much more useful to the degree that these things are true, less helpful to the degree they're not.

4. My best guess is that unfortunately Sean is continuing his streak of under-asking for funding, wanting to bring about higher-depth proof of concepts, clarity of workstreams etc. than I feel I need to be confident for $Xk bar. From my perspective, we're in a state of extreme triage; I do not want Lyptus to be wasting time fundraising. (And to some extent wasting my time doing larger number of funding rounds!) I would like Sean to expand the maximum funding here by a minimum of $20k (so that he can absorb $20k backpay + $100k prospectively), and very plausibly by a lot more than that. I am not at all saying that I would be comfortable at $500k right now, but I think that's where we should be starting the conversation. (As in, that does not obviously seem like the maximum to me, although it doesn't feel like minimum either.)

I'm delighted to recommend funding Lyptus' maximum here; I hope they can shift work to more ambitious ends that I think would be more helpful, and I hope they ask for more funding now.

I like the work you are doing, especially the neuroscientific aspect of it.

Thanks so much for showing your support for the project, @jessicapwang :)
You're truly inspiring!

I had a call with Aashka some weeks ago. She seems to be an individual with high agency, and I am looking forward to seeing her develop this project. India has a massive pool of education opportunities on AI Safety, and this proposal aims to tap into it early. Strongly supporting to give this project a chance.

Thank you so much for your best wishes, @eleklem :)

Thanks so much for your kind words and your donation. Definitely, we need more Global South initiatives. Your constant support really means a lot @eleklem :)

I love LessWrong

I have confidence in Sean and Jack's previous work and think it's worth boosting efforts to keep things like this operating in Australia. Bridging the ACNC registration gap seems worthwhile.

Good luck @aashkapatel - your passion and vision for this educational project is inspiring.

I was impressed by Aashka's AI nutrition label idea previously, and financially supported the IASEAI trip. Aashka managed to secure support for that idea from Dr. Russell via her trip, and that is endorsement is now scaling to additional initiatives. This traction is high signal for me. I also value that this is a Global South initiative - which we need more of - and I want to support greater visibility on these types of projects globally. I encourage others to seed fund this project for larger funding by others with means.

Theory 5: The Failure of Data-Grounded Models, Generative AI and traditional Machine Learning models are fundamentally incapable of reaching AGI. Because these models rely on finite datasets to represent entities of infinite dimensionality, they inevitably succumb to the Curse of Dimensionality. Consequently, they cannot truly generalize to the universal world, leading instead to overfitting and rote memorization rather than genuine understanding.

Theory 6: The Analog Imperative To achieve AGI, we must move beyond digital architectures. AGI requires a model that mirrors the inherent uniqueness of the physical world. This is only achievable through analog systems, which naturally reflect the continuous and unique state of reality without the need for discrete classifiers or digital approximation. @Zaelani l

I worked with @aashkapatel before and know how passionate she is about AI safety and literacy. Super excited to see where this project goes!

@pvasholz Thanks 😊

Project Update – Execution Phase Underway

Following the last update, the remaining operational bottlenecks have been cleared and the project has moved into active execution.

• Stable platform build is live: PBROC at Michigan State has begun transforming the landing pad vector into petunia, moving the core platform from design into biological integration.

• Agrobacterium pipeline cleared: Phytotron work and infiltration approval are now in place, removing the main gating step from the previous phase and enabling in-house experiments to proceed.

• Parallel expression testing ready: The full initial construct set is now in hand, including landing pad, regulatory, and enzymatic designs, enabling immediate testing of GFP and pigment expression.

• Plant system locked in: Mitchell petunia seed has been secured and plants are established, fixing the background for the first proof of concept.

• Iteration capability established: Bench space is now secured for plasmid cloning, enabling rapid generation of design variants without relying entirely on external synthesis.

The work is now focused on generating experimental data, with stable transformation progressing at PBROC while transient testing begins in parallel to compress iteration time.

Next step is generating initial expression data. I will share results once they are clear.

Thanks @Austin. It appears it was simple enough to edit myself!

To start! Thanks Joel! I very sincerely appreciate the candid and clearly worded feedback. I spent a bunch of time reflecting on this today.

With this in mind, I've drafted two budget tiers today with the information I have. This would be spent over two workstreams.

Our initial bridging fund was allocated to 2 months, with lower API credit and contractor asks, while we scoped future work. These tiers would have us operational for 6 months, and greatly increase the API credit and contractor budgets.

Tier 1: $300K

Workstream 1 — Cyber & Control:

- Salaries: $100K

- Model API credits: $50K

- Red team contractors / human experts: $50K

Workstream 2 — Pragmatic Interpretability:

- Salary + compute: $60K

Backpay + Overheads + Buffer: $40K

Tier 2: $600K

Workstream 1 — Cyber & Control:

- Salaries: $160K (supports an additional senior cyber researcher hire)

- Model API credits: $150K

- Red team contractors / human experts: $150K

Workstream 2 — Pragmatic Interpretability:

- Salary + compute: $60K

Backpay + Overheads + Buffer: $40K

What we're thinking

Workstream 2 is nascent and exploratory. I am particularly excited about upcoming work exploring self-steering via activation probes with potential applications in model welfare, model preferences and speculatively alignment stability. This workstream is a first step to support a core mission of Lyptus. Creating and growing institutional capacity, support and direction for untapped talent in Australia.

Workstream 1 is our larger investment. Through the course of our own work (and further so with the recent Mythos model card), it's plainly clear to us that cyber evaluations are materially not keeping up with capability growth. Honestly, the accelerating progress did take me by surprise and our dataset would certainly be saturated at higher token budgets. That said, while these steepening trendlines show no signs of stopping, we do believe current absolute capabilities can be overstated. Full red teaming engagements against even moderate blue teams are a very different proposition to isolated benchmark tasks.

To ground this with an illustrative example of where our thinking is. We're currently scoping the tractability of scaling up the ideas presented in the Artemis paper. Evaluating AI agents via real red teaming engagements and comparing their performance to professional human red teamers. This captures the full end-to-end scope of an attacker against real production organisations.

This is clearly operationally difficult. Though early conversations with senior offensive security leadership suggest it is tractable. We believe with the right people involved we can design incentive structures that work for all parties. A typical engagement with two professionals runs around $30K USD per week. We would ramp up starting with smaller targets.

This is speculative. We are having early conversations, and we are genuinely in a phase of figuring things out. It is our current leading candidate project but not our only option. What it represents is our broader interest in human grounded studies and science communication. Results that are quickly interpretable beyond the AI security ecosystem. We think this suits our strengths better than building cyber benchmark tasks, which requires deeper offensive security expertise than we have in house.

What the tiers mean

At tier 1 we would more likely prioritise budget friendly work like human-grounded covert capability on cyber tasks or human-grounded studies on cyber stealth benchmarks. If we had strong multi-organisation traction on the red teaming idea we would still pursue it, but budget constraints have a way of subtly pressuring priorities against more ambitious work.

At tier 2 the more ambitious work becomes realistic. A senior cyber researcher hire, 2-3 red teaming engagements at small to medium scale with client co-financing, and enough API credits to properly test models at adequate token budgets.

Other ideas in this space would require internal budget rescoping but the broad structure holds.

Thank you for your kind words, @ridakadri14 :)

All the best @aashkapatel, the project is really impressive and inspiring !!!

Far-UVC is a technology with amazing potential, and I'd love to see more deployments for generating efficacy and safety data.

Approving this project! I'm glad that @joel_bkr and others at METR are so excited for Sean's work, and from my perspective, this kind of active, expert-led, fast grantmaking is where the regranting mechanism shines.

I also appreciate Joel taking the time to lay out his considerations in making this grant, which are hopefully helpful for Sean and others in similar positions. (And @seanpetersau, insofar as you would be open to accepting more funding through this Manifund project, let me know; we're very happy to increase the funding goal on your behalf!)

(Noting that after conversation with Nate it's now less obvious to me that Lyptus should focus on dramatically harder cyber tasks in particular. Although I continue to think that there is a very, very, very high premium on dramatically harder tasks in general. Want to leave the focus more flexible depending on Lyptus' taste.)

Here's my updated sense of where Sean and co. are at, extremely heavily influenced by colleagues, especially Nate Rush:

1. My original comment on Sean continues to stand up well today. He appears to be among the very most competent people building on top of METR work.

2. The work he links above was competently executed. Their baselines/time estimates are reasonable (less common than you might think), they have a high attention-to-detail vibe, the write-up is fairly impressive and not overclaiming, etc. The work compares favorably to UK AISI's analogous work.

3. Unfortunately, best guess is that all of their tasks would saturate for most recent models.

Coming out of this:

1. I think it's a shame that Sean needed to dip into personal funds to pay for this project.

2. I consider it ~totally obvious that Lyptus' work is worthy of further funding.

3. I really, really, really, really, really want Lyptus to (1) throw more AI labor at their problems and (2) create dramatically harder cyber tasks. I think the end work will be much more useful to the degree that these things are true, less helpful to the degree they're not.

4. My best guess is that unfortunately Sean is continuing his streak of under-asking for funding, wanting to bring about higher-depth proof of concepts, clarity of workstreams etc. than I feel I need to be confident for $Xk bar. From my perspective, we're in a state of extreme triage; I do not want Lyptus to be wasting time fundraising. (And to some extent wasting my time doing larger number of funding rounds!) I would like Sean to expand the maximum funding here by a minimum of $20k (so that he can absorb $20k backpay + $100k prospectively), and very plausibly by a lot more than that. I am not at all saying that I would be comfortable at $500k right now, but I think that's where we should be starting the conversation. (As in, that does not obviously seem like the maximum to me, although it doesn't feel like minimum either.)

I'm delighted to recommend funding Lyptus' maximum here; I hope they can shift work to more ambitious ends that I think would be more helpful, and I hope they ask for more funding now.

I like the work you are doing, especially the neuroscientific aspect of it.

Thanks so much for showing your support for the project, @jessicapwang :)
You're truly inspiring!

I had a call with Aashka some weeks ago. She seems to be an individual with high agency, and I am looking forward to seeing her develop this project. India has a massive pool of education opportunities on AI Safety, and this proposal aims to tap into it early. Strongly supporting to give this project a chance.

Thank you so much for your best wishes, @eleklem :)

Thanks so much for your kind words and your donation. Definitely, we need more Global South initiatives. Your constant support really means a lot @eleklem :)

I love LessWrong

I have confidence in Sean and Jack's previous work and think it's worth boosting efforts to keep things like this operating in Australia. Bridging the ACNC registration gap seems worthwhile.

Good luck @aashkapatel - your passion and vision for this educational project is inspiring.

I was impressed by Aashka's AI nutrition label idea previously, and financially supported the IASEAI trip. Aashka managed to secure support for that idea from Dr. Russell via her trip, and that is endorsement is now scaling to additional initiatives. This traction is high signal for me. I also value that this is a Global South initiative - which we need more of - and I want to support greater visibility on these types of projects globally. I encourage others to seed fund this project for larger funding by others with means.

Theory 5: The Failure of Data-Grounded Models, Generative AI and traditional Machine Learning models are fundamentally incapable of reaching AGI. Because these models rely on finite datasets to represent entities of infinite dimensionality, they inevitably succumb to the Curse of Dimensionality. Consequently, they cannot truly generalize to the universal world, leading instead to overfitting and rote memorization rather than genuine understanding.

Theory 6: The Analog Imperative To achieve AGI, we must move beyond digital architectures. AGI requires a model that mirrors the inherent uniqueness of the physical world. This is only achievable through analog systems, which naturally reflect the continuous and unique state of reality without the need for discrete classifiers or digital approximation. @Zaelani l