The idea of a rapid emergency response team, like ALERT [^1]has been floating around. Subsequent people have at times partially taken up the mantle, but because of the vicissitudes of life, they have each given up.
I think the project is one of the more promising ideas to come out of the EA brain trust in recent years. I don't think the project is limited on theorizing (see e.g., a past detailed proposal [^2]), but on execution:
- Gather a team of reservists -- done by previous projects, easy to pick up
- Immediate next step: Sort through those who could join a pop-up effort immediately, and those who are interested in theory. Have the more committed reservists talk with bosses to be able to drop their day job during an emergency.
- Step after that: Look for funding so that reservists don't have to internalize the costs completely. Organize reservists so that they are some available at any timezone.
- Gather a team of forecasters and alert signals -- easy to do in collaboration with, for instance, Samotsvety, or with the deep pool of talent around the forecasting community
- Immediate next step: Set up an MVP, any MVP, of a method which could call a rapid response team into action.
- Step after that: Improve alarm raising so that it catches more emergencies in time, and throws up fewer false positives.
- Run training exercises
- Immediate next step: Design a training exercise that has the team of reservists deal with something which, to them, was an "unknown unknown". This could be a completely made-up exercise, but my preference would be for this to be a regional event of some significance, or an attempt to solve a valuable but non-urgent problem urgently.
- Step after that: Using the forecasting/alert signal team, choose the next non-catastrophic event as it pops up, and react and act in real-time.
Personally, I am interested in this project because it doesn't require me to trust the concepts [^3] or for us to be able to forecast AI precisely [^4]; it doesn't require future catastrophes from AI to correspond to the Yudkowskian framework, or to any preconceived notion of AI. Indeed, in generality, having a reserve team that can act during emergencies doesn't require future threats to humanity to come in the shape of AI; the more surprising the future turns out to be, the better this intervention looks related to alternatives. It is a bet on variance. I have been looking for a project to commit to, and this seems like a good fit.
- The ask: Cash money to get this started; to do the steps in "immediate next steps". I will also put money and time of my own, financed by my lucrative consulting [^5]. But this is easier if a few people each pitch $20K to $50K.
- The ambitious ask: A sizable budget to carry out the "step after that" items, and to not be bottlenecked on money, ever. Previous projects aimed to get this and failed. I think this is a red-herring, because it requires getting the trust of large funders, who tend to move slow and be slow to trust.
- The institutional structure: I will set up an institutional structure if I have to, but prefer not to, at least at first.
[^1]: https://forum.effectivealtruism.org/posts/sgcxDwyD2KL6BHH2C/case-for-emergency-response-teams
[^2]: https://docs.google.com/document/d/1IStSw8MIU1z68NjAmMHBM3gvzaBRrj6W_Wuw6hY4ICE/edit
[^3]: https://nunosempere.com/blog/2023/01/23/my-highly-personal-skepticism-braindump-on-existential-risk/
[^4]: https://nunosempere.com/blog/2023/11/07/hurdles-forecasting-ai/
Nuño Sempere
over 1 year ago
This continues to run, you can keep track at https://blog.sentinel-team.org. We also have a weekly private project update mailing list to which I'm happy to add people with whom I have some rapport.
One important development has been getting a cofounder, Rai Sur (rai.dev) to complement my skillset; he's been working out great.
Nuño Sempere
over 1 year ago
A foresight team continuously looking for things that could become large-scale or existential catastrophes
An emergency response composed of capable yet mostly unattached people who could convene in the event of a catastrophe
A website http://sentinel-team.org/
It also has a short writeup of the project's history https://sentinel-team.org/sentinel-history/
Weekly public minutes from the foresight team https://sentinel-team.org/blog/
A fiscal sponsor
Further funding
A collaboration with Nathaniel Cooke on foresight methods beyond forecasting <https://forecasting.substack.com/p/a-gentle-introduction-to-risk-frameworks>. We later parted ways, but this was a good output.
I'm probably forgetting something
In short, the project has overall been going well. The idea was to have two components, the first of which was a foresight team that could raise an alarm if something happens. This foresight team is going great; I have three very obsessive, very competent forecasters, in addition to myself, and some tooling to aid them.
The emergency response team has been going well, but less so. It exists, it has some competent people, and we had a trial run with the Iran attacks on Israel.
But in general I just feel much better about our ability to, say, detect a Chinese invasion of Taiwan two weeks, or a few days before it happens, than I feel about our ability to do anything about it.
Some steps on the horizon:
Improve emergency response team.
Integrate more info sources.
Put out analytical pieces sharing lessons learnt
Reach out to potential collaborators and similar projects
Consider finding more funding
Introduction to potential emergency response team members. For details see https://sentinel-team.org/emergency_response_team/
Mentorship seems like it would be super useful to me. Are you one or two levels above me in life; have you set up something cool and want to share pointers? I'd be grateful.
I'm currently conflicted about funding. I'd appreciate help either with acquiring more, or with deciding that it's a distraction.
Funding is not the bottleneck on the, say $5k to $10k range, but funding on the $100k to $5M range would allow me to make this project more awesome.
I'm thinking that I prefer a smaller project that is sustainable ~forever, over a larger project that lives or dies by [large funder]'s word. But is this a good way to think about it? And even if it is, should I instead attempt to build something that shines twice as bright but lasts twice as long?
I'm procrastinating on applying to the SFF grant round. Partly this is because I find the application baroque. Help, or just coworking on it, would be appreciated
On the other hand, this project is sustainable at the current spend, so looking for more funding feels like a distraction.
I'm currently ~not really paying myself. I'm probably fine with this until the end of this year, though. Is this a good move?
I am very grateful to Manifund.
Writing the project proposal and getting early funding was important for coordinating between people interested in supporting the project.
Getting early funding from peers, from people whose respect I cherished, was important for me psychologically. It made me more excited. It was a hard to fake signal of promisingness
Early funding has been useful to not have money be a bottleneck.
Victor Porton
over 1 year ago
Upvoted because most people are not driven by longtermism. We need kinda to conquer cities (and societies in general). People's opinions are important.
Quinn Dougherty
about 2 years ago
why the longtermist branding? seems unnecessary and risks dragging critical people into distracting philosophical tangents
Austin Chen
about 2 years ago
Approving this project! I was excited by the original launch of ALERT (and applied as a reservist, I think). I think the idea is good, but as they say in startupland "execution is everything" - best wishes on the execution, and let us know if we can help
Gavin Leech
about 2 years ago
The predecessor was my most important project last year. I've personally verified that there's a great deal of demand for some version of this ("customer" orgs and institutions and "supplier" volunteers). Nuno has some rare and essential qualities (honesty, clarity, infovorism) while lacking some others. But the shoestring version still excites me and I vote with my feet.
Isaak Freeman
about 2 years ago
Matching Joel in reasoning and donation. Great idea; only possible through charity ecosystems; I was sad to see the previous one fade.
Joel Becker
about 2 years ago
I've made a $5k offer to this project. I might end up offering more later. Below written hastily.
I agree that emergency response teams are promising in the abstract, broadly for reasons given in the linked post.
Nuno's plan seems reasonable. I think it has Gavin's backing.
As Nuno has written elsewhere, AI feels like "a domain in which we are likely to not have the correct hypothesis in our prior set of hypotheses." I feel more excited about "a bet on variance" in this setting.
ALERT needs someone to take responsibility for it, and Nuno is standing in front of us willing to take responsibility.
I have thought ~0 about who might be the right person to lead this project and don't feel pulled towards taking this more seriously. Meanwhile, my rough and probably-somewhat-misremembered understanding is that Open Philanthropy have been hesitant to throw their weight behind ALERT-style projects because they want the person running it to clear a very high bar. Something about this pattern-matches nicely to my take that the OP cluster is too distrusting of people outside the cluster, which makes me feel more comfortable ignoring the importance of careful selection of the person leading ALERT. But neither my understanding nor my take is well thought-through. So the chance I'm mistaken in thinking that I shouldn't take this concern seriously seems decently high. (And perhaps if I did put thought to the concern, I'd discover that I think Nuno is not the right person to lead this effort. Right now my position is that I haven't even thought about what that would mean, let alone evaluated Nuno in particular.)
"Improve alarm raising so that it catches more emergencies in time, and throws up fewer false positives." This step feels potentially hard to me. At the very least I think it should read "and/or throws up fewer false positives."
Something I really, really like about Dendritic is that it plans to put monitoring, alarm-raising, and emergency response under one roof (but for bio only). I like this because it follows a model that I gather has already been successful elsewhere (Security Operations Centers). It makes sense to me that this model would be less likely to come up against problems like "we've raised the alarm but no-one is responding." I am seriously concerned that ALERT might run into problems like this.
I would prefer to offer a larger amount, but my Manifund budget is pretty constrained. Any amount I offer to this project will be based on some vague notion of 'fair share' between the remaining projects I want to fund before Dec 31.
First, Nuno is my friend. Second, one of my projects (Dendritic) is taking up a part of the ALERT mantle in a different way (monitoring + emergency response for bio in particular). There's no immediate conflict with Dendritic, but it doesn't seem implausible to me that there could be some Nuno-Dendritic collaboration in future given the overlap.