***NEWS November 7***

Stanford University Press has expressed interest in publishing a new edition of this book. This is great! All I need now is funding to get this going.

***end NEWS***

I’m making an effort to publish a new edition of a pioneering 1960s work on existential risk (x-risk) from artificial superintelligence (ASI).

The short novel was written in 1966 by a soon-to-be Nobel laureate in Physics. The story was originally intended as a bedtime story for kids (!). Instead, it ended stirring up debate among politicians and intellectuals about the future of computing technology (a discourse which was interrupted, however, by the increasing focus on data and privacy in the early 1970s). This now largely forgotten book – at least by international audiences – needs to be brought back into circulation in our current moment because it encourages us to think critically and historically about the prospect of superintelligent machines.

The book I’m talking about is Hannes Alfvén’s The Tale of the Big Computer (Coward-McCann, 1968, 126 pp).

The book was originally published in Swedish as Sagan om den stora datamaskinen under the pen name Olof Johannesson but quickly found its way to an international readership. It has, however, not been republished in English for more than half a century at this point. This must change.

Why should we care about some old novel?!

There are several reasons why this novel deserves our attention in 2024. Here are some of them:

1.     X-risk ASI and governance: This work is the first work that I’m aware of which develops a narrative about existential risk from superintelligence by breakdown of governance.

2.     Full automation of society: It is also one of the first to imagine a scenario where a singular machine intelligence runs the entirety of human society – from resource allocation to societal organization.

3.     Extinction by obsolescence: Crucially, in this story, machines are allowed to take over the world thanks to their efficiency and convenience. This process ultimately results in the general enfeeblement of humans.

4.     Complexity factor: The deterioration of society happens in stages where the complexity of interconnected systems evolves to a level where oversight and maintenance are rendered practically insurmountable.

5.     Intelligence explosion: It’s a literary depiction of I.J. Good’s notion of an “intelligence explosion”, but a creeping, subtle one – published only a few years after Good presented his concept at a conference in L.A. (1962). And some 50 years (!) before his fellow Swede Nick Bostrom released his book Superintelligence (2014).

6.     Second species problem: The novel is written from the perspective of a computer who is acting as an historian, producing an account of world historical events. Importantly, the narrative places humans at an evolutionary stage where their sole importance hinges on the ability to give rise to second species in the form of computational machinery, which/who subsequently self-improves and self-propagates. It’s thus a very early account of machinic self-reproduction, with prominent predecessors in Samuel Butler‘s Erewhon (1872) and Gilbert Simondon’s Du mode d'existence des objets techniques (1958).

7.     Misaligned – not evil: The novel disregards the problem of evil or hostile AI and presents, instead, existential danger as a consequence of systemic collapse in machineries of bureaucracy. Central to the dilemma is the willingness of humans to hand over control to AI as they become aware of it superior capabilities. Humans don’t have to be exterminated; they are simply no longer needed for anything.

In other words, this is NOT another account of “computer uprising” as in Capek’s R.U.R. (1920) or Dick’s Vulcan’s Hammer (1960) or Terminator (1982) or any of the other thousands of similar stories. Nor is it another depiction of “FOOM” or instant collapse from military experiments gone wrong. Instead, it’s a narrative which deals with concepts that have only in recent years come to the attention of broader audiences (and, really, three decades before even online subcultures were beginning to explore them): intelligence explosion, technological singularity, computational self-propagation et cetera. Problems that are now the objects of policy work and technical AI safety efforts are developed in this 1960s work of fiction – AI alignment, evolutionary AI, x-risk, superintelligence, AI governance, autonomous agents et cetera.

I still don’t get it, why should we care?

While this book did stir up a bit of dust when it arrived in the 1960s, it is only now, in the past 5 years, say, that x-risk from AI has made its way to policy makers, regulators, and tech companies. But, even as problems of alignment and technical AI safety are getting more attention than before, it’s still a very small part of AI work overall. This book – not despite but thanks to – it being a work of fiction and a very enjoyable and quick read, perfectly complements the existing but more challenging policy and technical-oriented literature. Apart from the sci-fi market which struggles to reach audiences outside of its own realm of interests, here’s actually a work of fiction which engages with x-risk topics and superintelligence in a way that is accessible, entertaining, and thought-provoking for people who didn’t know that they cared about such matters. Along the way, it also succeeds in demonstrating the deeper history of these predicaments. After all, it was written at a time when computers filled entire rooms and were operated by a few large corporations, governments, research labs and, of course, the military. Reading this book, it is astounding to see how what was wild speculation in the mid 1960s is now working its way into AI safety discussions being held at international summits and national governments. I wager that this short novel has the potential to broaden the discourse on AI safety and x-risk substantially. The principal argument for publishing a new edition is thus to open up these topics to completely new audiences. People who didn’t use to worry about x-risk from AI can use this book to find an entertaining and convenient way to join the ongoing discussion.

A bit more context and why it matters NOW

Bearing in mind that last week’s (October 2024) Nobel prizes in physics and chemistry were awarded to discoveries in artificial intelligence – effectively recognizing a shift from science to engineering in these fields – it is baffling to re-read a 1970s physics laureate’s own take on this trajectory. Yet, Hannes Alfvén was no luddite. He was a one of the most imaginative scientific minds of the 20th century who feared not computing technology per se but, rather, its impact on humans and our understanding of ourselves and our society. Alfvén, aside from being a revered scientist and Nobel laureate, was a highly respected public intellectual in the latter half of 20th century Scandinavia. Since a few years, of course, the problems raised by the book have become less fictitious. We now have CEOs in charge of multi-billion-dollar corporations building their business cases on what Alfvén pointed out as dangers of existential risk some fifty years earlier: the notion that we would be better off leaving matters of knowledge and reason to technology of alleged superior and infallible capabilities. Alfvén was ahead of his time in warning about the subtle, systemic dangers of AI leading up to existential risk. This is a theme that resonates strongly in today’s discussions about AI governance, automation, and the potential for AI to reshape human society in transformative and perhaps irreversible ways.

A note on the novel’s publication history

The book was originally published as Sagan om den stora datamaskinen (Bonnier, 1966) in Swedish by Hannes Alfvén (Nobel laureate in physics 1970) under the pen name Olof Johannesson. There was much speculation in national newspapers about who the author of this strange story might be. A paperback came out three years later (Bonnier, 1969). In the late 1980s when AI topics were discussed again in the Nordics, a new edition was made (Pilgrim press, 1987). At the semicentennial of its initial publication, it was republished as an illustrated version in Swedish (Nilleditions, 2016).

This book was first translated into English as The Tale of the Big Computer (Coward-McCann, 1968, 126 pp) and also as The Great Computer: A Vision (Gollancz, 1968) and, a year later as a sci-fi paperback: The End of Man? (Award Books, 1969). There is a digital version of this book available on The Internet Archive, but only accessible for 1-hour loans for members: https://archive.org/details/the-tale-of-the-big-computer.

Current context and track record

The story was planned to be produced as an opera in the 1970s but the composer (Karl-Birger Blomdahl) passed away before it was realized. In the fall of 2022, however, an opera was indeed set up at the KTH Royal Institute of Technology in Stockholm where Alfvén was once a professor (and where I myself am employed at the Division of History of Technology, Science, and Environment): https://intra.kth.se/en/aktuellt/nyheter/operan-i-reaktorhallen-en-stor-succe-1.1227783

I am probably the person who has written more about this publication than anyone else. Already in 2012, I wrote a lengthy essay on its underlying ideas from a historical and philosophical perspective for a seminar at the University of Stockholm. One year later, I made a podcast on the topic for public radio. I returned to the book’s context and influence in my doctoral dissertation which was about the digitalization of Scandinavia. Having a computer science background and a Ph.D. in the History of Ideas, I have found the book’s themes to be continually intriguing. I'm currently a visiting scholar at Stanford University where I'm working on a project about the history of AI.

Proposal: A new edition of The Tale of the Big Computer

The technicalities regarding rights of translation and publication to this 1966 work need to be investigated. Possibly, the narrative would benefit from a more up-to-date translation. My suggestion would be to create the new edition based on the original 1966 and 1968 editions (Bonnier in Sweden and Coward-McCann in the US), which seem aesthetically most appealing. The 1969 Swedish paperback used artwork by Per Olof Ultvedt who famously organized the 1966 exhibition ”She – A Cathedral” at the Museum of Modern Art in Stockholm with Niki de Saint Phalle and Jean Tinguely: https://www.modernamuseet.se/stockholm/en/exhibitions/remembering-she-a-cathedral/. These might still be relevant to include, especially if a 1960s tech art retro look is desirable. Ultvedt also made the initial designs for the failed opera project and some of these might be accessible (I know an artist who has investigated the mystery behind this defunct opera as an art project).

For the new edition in English, I myself would be willing to write an introduction which situates the 1960s story in our own time and its challenges. I am currently in the process of reaching out to relevant publishers to make this book available to a new generation of readers.

Budget outline

Below is how I have roughly budgeted the project. Basically, I imagine the publisher bearing most of the costs (copyright/permissions, editing, proofreading, design, printing, and distribution) and my own expenditures amounting to project management, writing the preface, and doing some of the promoting work. I might have to cover, to some extent, a new translation if the publisher thinks it isn't necessary and I do.

Publisher: $30–40K – not asking this from Manifund – this is the publisher’s own risk/investment

My own costs: $20K – covering new translation (5K), project management (10K), and promotion/campaign (5K).

Here is the New York Times 1968 review.