I think the point that LLMs, even with their current capabilities, make malicious attacks cheaper and more accessible to people outside of the government makes a lot of sense. I'd be curious to see a more expansive explanation of how you expect this to be destabilizing and how this affects your probability estimates of existential risk/what it means we (or more precisely, Open Philanthropy) should be doing differently.
Basically, in your opening you say you're going to address the probability of existential risk through a national security lens, and throughout the essay you talk about AI and national security (which may well be a neglected and useful perspective, I don't see much of that from the AI safety community!), but the conclusion on existential risk is basically "this will be destabilizing to all of human activity", which feels a bit unsatisfying?
(I'd probably take you up on your sell offer if the conclusion addressed this more concretely)
This essay addresses the probability and existential risk of AI through the lens of national security, which the author believes is the most impactful way to address the issue. Thus the author restricts application of the argument to specific near term versions of Processes for Automating Scientific and Technological Advancement (PASTAs) and human-level AI.
Essay Here
The key steps to the argument are thus:
1. The cost of malicious and autonomous attacks is falling.
2. If the cost of defense is not falling at the same rate, then the current balance of forces between cybersecurity and cyberattack will favor attack, creating a cascade of vulnerabilities across critical systems.
3. The proliferation of these generative AI models (at least across governments).
4. Thus security and information sharing between public and private sectors will be essential for ensuring best practices in security and defense.
5. But the number of vulnerabilities is also increasing. Thus the potential for explosive escalation and/or destabilization of regimes will be great. Non-state actors will increasingly be able to operate in what were previously nation-state level activities.
6. Thus I conclude that capabilities monitoring and both public-private and joint-diplomatic efforts are likely required to protect citizen interests and prevent human suffering.
This Essay Is Geared Towards the Policy Establishment
holds 0.5%
Rachel Weinberg
over 2 years ago
holds 99.5%
John Buridan
over 2 years ago
I have expanded the conclusion to more fully address your point. Thank you for the prompt.