Book Summary
An urgent warning of the unprecedented risks that AI and other fast-developing technologies pose to global order, and how we might contain them while we have the chance.
Key Takeaways:
- THE COMING WAVE: An emerging cluster of related technologies centered on AI and synthetic biology whose transformative applications will both empower humankind and present unprecedented risks.
- FOUR FEATURES: The unique characteristics of the coming wave that exacerbate the challenge of containment. They are asymmetry, hyper-evolution, omni-use, and autonomy.
- Almost every foundational technology ever invented, from pickaxes to plows, pottery to photography, phones to planes, and everything in between, follows a single, seemingly immutable law: it gets cheaper and easier to use, and ultimately it proliferates, far and wide. This proliferation of technology in waves is the story of Homo technologicus—of the technological animal. Humanity’s quest to improve—ourselves, our lot, our abilities, and our influence over our environment—has powered a relentless evolution of ideas and creation.
- The coming wave is defined by two core technologies: artificial intelligence (AI) and synthetic biology. Together they will usher in a new dawn for humanity, creating wealth and surplus unlike anything ever seen. And yet their rapid proliferation also threatens to empower a diverse array of bad actors to unleash disruption, instability, and even catastrophe on an unimaginable scale. This wave creates an immense challenge that will define the twenty-first century: our future both depends on these technologies and is imperiled by them.
- “Every time I reduce the charge for our car by one dollar, I get a thousand new buyers.” By the 1920s Ford was selling millions of cars every year. Middle-class
- Technology has a clear, inevitable trajectory: mass diffusion in great roiling waves. This is true from the earliest flint and bone tools to the latest AI models.
- From the written word to sailing vessels, technology increases interconnectedness, helping to boost its own flow and spread. Each wave hence lays the groundwork for successive waves.
- One analysis estimates that the introduction of the printing press in the fifteenth century caused a 340-fold decrease in the price of a book, further driving adoption and yet more demand.
- Proliferation is catalyzed by two forces: demand and the resulting cost decreases, each of which drives technology to become even better and cheaper.
- Civilization’s appetite for useful and cheaper technologies is boundless. This will not change.
- AUTOCOMPLETE EVERYTHING: THE RISE OF LARGE LANGUAGE MODELS
- the future of computing was conversational. Every interaction with a computer is already a conversation of sorts, just using buttons, keys, and pixels to translate human thoughts to machine-readable code. Now that barrier was starting to break down. Machines would soon understand our language. It was, and still is, a thrilling prospect.
- Over time, technology tends toward generality.
- If you were looking to monitor and direct AI research in the past, you would likely have got it wrong, blocking or boosting work that eventually proved irrelevant, entirely missing the most important breakthroughs quietly brewing on the sidelines. Science and technology research is inherently unpredictable, exceptionally open, and growing fast. Governing or controlling it is therefore immensely difficult. Today’s world is optimized for curiosity, sharing, and research at a pace never seen before. Modern research works against containment. So too do the necessity and desire to make a profit.
- Little is ultimately more valuable than intelligence. Intelligence is the wellspring and the director, architect, and facilitator of the world economy. The more we expand the range and nature of intelligences on offer, the more growth should be possible.
- The story of stirrups and feudalism highlights an important truth: new technologies help create new centers of power with new social infrastructures both enabling them and supporting them. In the last chapter we saw how this process today adds to a series of immediate challenges facing the nation-state. But over the longer term, the implications of power’s plummeting costs are tectonic, techno-political earthquakes shaking the ground upon which the state is built.
- In a few decades, I predict most physical products will look like services. Zero marginal cost production and distribution will make it possible. The migration to the cloud will become all-encompassing, and the trend will be spurred by the ascendancy of low-code and no-code software, the rise of bio-manufacturing, and the boom in 3-D printing. When you combine all the facets of the coming wave, from the design, management, and logistical capabilities of AI to the modeling of chemical reactions enabled by quantum computing to the fine-grained assembly capabilities of robotics, you get a wholesale revolution in the nature of production. Foods,
- In sum, returns on intelligence will compound exponentially. A select few artificial intelligences that we used to call organizations will massively benefit from a new concentration of ability—probably the greatest such concentration yet seen. Re-creating the essence of what’s made our species so successful into tools that can be reused and reapplied over and over, in myriad different settings, is a mighty prize, which corporations and bureaucracies of all kinds will pursue, and wield. How these entities are governed, how they will rub against, capture, and reengineer the state, is an open question. That they will challenge it seems certain.
- As people increasingly take power into their own hands, I expect inequality’s newest frontier to lie in biology.
- Containment will need to respond to the nature of a technology, and channel it in directions that are easier to control. Recall the four features of the coming wave: asymmetry, hyper-evolution, omni-use, and autonomy.
- Technology isn’t just a way to store your selfies; it represents access to the world’s accumulated culture and wisdom. Technology is not a niche; it is a hyper-object dominating human existence.
- SAFETY: AN APOLLO PROGRAM FOR TECHNICAL SAFETY
- Instead, we should identify problems early and then invest more time and resources in the fundamentals. Think big. Create common standards. Safety features should not be afterthoughts but inherent design properties of all these new technologies, the ground state of everything that comes next. Despite the fierce challenges, I’m genuinely excited by the range and ingenuity of ideas here. Let’s give them the intellectual oxygen and material support to succeed, recognizing that while engineering is never the whole answer, it’s a fundamental part of it.
- Credible critics must be practitioners. Building the right technology, having the practical means to change its course, not just observing and commenting, but actively showing the way, making the change, effecting the necessary actions at source, means critics need to be involved. They cannot stand shouting from the sidelines.
What I Got Out of It:
- A thorough overview of the history of AI, some of the upcoming threats and things to think about as well as how to possibly contain or at least mitigate them.
originally from blas.com