An old friend visited me a few weeks ago. And we soon got to chatting about—what else—how long will it be before all human intellectual work is automated.
My position was: I dunno, because things are moving fast right now but what if we run out of data or scaling laws break and algorithmic progress stalls? His position was: Soon.
Then he started asking about this blog. What were the most popular posts? This was slightly ominous given his regrettable tendency not to consume most Dynomight internet content. But I told him probably Underrated Reasons to be Thankful and its sequels, and that seemed to be the end of it.
But then a week later, he texted to ask if I had any other short-form writing. And then he sent me a list of previous Reasons I’d written and asked me to rank them by quality. And gradually it dawned on me that he had decided it was time to automate me.
Soon he started sending new AI-generated Reasons. Which weren’t great. But then he tuned his prompt and they got better. And then he switched models and increased the scale by 1000x and added a secondary scoring AI and tuned the scoring AI prompt and the Reasons got better and better and better and better. And as I watched all this happen, I couldn’t help but reflect:
-
In old science fiction, people imagined robots that were totally precise and accurate and rational but couldn’t fathom the messy “soul” of humanity. Modern AI is exactly the opposite. It could easily copy my writing style after a small number of examples. The biggest challenge was to get the damned facts right.
-
For example:
That if you somehow took the 33,000 nuclear warheads on the planet and got them to a white dwarf star […] then the resulting explosion would be visible from 1,000 light years away with a magnitude of 7.6, comparable to the supernova of Betelgeuse, so we’re not total lightweights.
This sounds like me. The joke is good. But the “fact” is nonsense.
-
Looking back, perhaps the old science fiction robots were a projection of our fantasies. I want my style and voice to be precious and uncopyable. I want an AI assistant that will tirelessly hunt down facts while I provide creativity and panache and magic. But that’s not how it is. So we should remember: In a scenario where AI is useful but humans retain some comparative advantage, we might not like where that comparative advantage lies.
-
When I complained about the previous points, my friend remarked that this was to be expected because, after all, “personality is just like 4-6 bits”.
I think it’s higher. But what is personality, after all? One idea is that as you go through life, you copy behaviors from others that you like, and that combination is “you”. Maybe my writing voice is just what you get when you take the different parts of different writers I like and mush them together. So maybe personality isn’t a lot higher than 4-6 bits.
-
An AI-you is a funhouse mirror for your soul. The most salient feature of AI-me is that it was dark. I often (I see now) use a little gambit where I start out with something technical or edgy and gradually work my way towards a positive crescendo. The AI seems to run towards the darkness without a clear plan for turning towards the light. It usually resorted to (a) incredibly lame/cringey ideas, (b) making stuff up, or (c) just staying dark and not even attempting to be positive.
-
For example:
That sports are a socially sanctioned way to release tribal impulses, and it’s just wonderful how at the end of a game, everyone goes home and nothing is really lost, except for the 46 people who died in the 1964 Lima stadium riot, and the 79 killed in the 1968 Buenos Aires stadium stampede, and the 328 who died in the 1969 Salvador vs. Honduras soccer war, and the 71 who died in the 1971 Glasgow disaster, and the 48 who died in the 1971 Ibrox stadium collapse, and the 49 killed in the 1974 Cairo stadium riot, and the 2 who died in the 1979 The Hague incident, and the 66 killed in the 1979 Ghana stampede, and the 24 who died in the 1981 Athens stadium collapse, and the 1 who died in the 1982 Cali drug war, and the 14 who died in the 1982 Moscow incident, and the 18 who died in the 1982 Cali explosion, and the 39 who died in the 1985 Heysel stadium stampede, and the 10 who died in the 1985 Mexico city incident, and the 8 who died in the 1985 Kenya stampede, and the 93 who died in the 1989 Hillsborough disaster, and the 40 who died in the 1991 Orkney stadium stampede, and the 7 who died in the 1993 Hong Kong incident, and the 82 who died in the 1996 Guatemala stampede, and the 15 who died in the 1996 Lago Agrio stadium collapse, and the 83 who died in the 1997 Tripoli stadium incident, and the 126 who died in the 1998 Kathmandu stampede, and the 107 who died in the 2000 Monrovia incident, and the 13 who died in the 2000 Harare stampede, and the 126 who died in the 2001 Accra stadium stampede, and the 43 who died in the 2001 Ellis Park stampede, and the 14 who died in the 2007 Salvador stampede, and the 15 who died in the 2007 Sangrur stampede, and the 13 who died in the 2008 Butembo riot, and the 12 who died in the 2009 Abidjan stampede, and the 19 who died in the 2009 Nairobi stampede, and the 11 who died in the 2011 Cairo riot, and the 79 who died in the 2012 Port Said riot, and the 13 who died in the 2015 Cairo stampede, and the 8 who died in the 2017 Uyo stadium collapse, and the 17 who died in the 2018 Caracas stampede.
I could have written that. I wish I had written that. But personally, it doesn’t make me feel thankful.
-
At one point, the AI suggested that we should be thankful that it was possible to encode the entire text of the best book ever written (apparently Harry Potter and the Methods of Rationality) into a single drop of water. But it then admitted it wasn’t sure if anyone had quite done this yet, and invited anyone who did to email
xy@dynomight.netwherexandyare the first letters of my (currently non-public) first and last name. Is that information in the training data somewhere? Do LLMs have emergent stylometry abilities? Creepy. -
As the AI got better and better, so did my own cope. At first I thought, “It’s bad.” Then, I thought, “OK, it’s not bad, but it’s not creative.” Later, I thought, “OK, it’s not bad, and it can be creative, but it’s not accurate.” By the end of the week, I was at, “OK, it’s not bad, and it can be creative, and it can be accurate, and it can be funny, and it sounds almost exactly like me, but in order to do all those things at the same time you can’t completely rely on the automated scoring AI, but need some human curation and editing.”
-
Speaking of cope:
- That we don’t have alien overlords who made us and who look down on us with pity.
- That even though we do have alien overlords who made us and who look down on us with pity, they’re very benevolent alien overlords who mostly leave us alone.
- That even though we do have alien overlords who made us and who look down on us with pity, and they aren’t benevolent alien overlords who mostly leave us alone, they at least have the decency to keep their existence a secret.
- That even though we do have alien overlords who made us and who look down on us with pity, and they aren’t benevolent alien overlords who mostly leave us alone, and they don’t have the decency to keep their existence a secret, at least they’re not so horrifically awful that we’d prefer to be dead than live under their rule.
- That even though we do have alien overlords who made us and who look down on us with pity, and they aren’t benevolent alien overlords who mostly leave us alone, and they don’t have the decency to keep their existence a secret, and they are so horrifically awful that we’d prefer to be dead than live under their rule, at least there’s some way to rebel.
- That even though we do have alien overlords who made us and who look down on us with pity, and they aren’t benevolent alien overlords who mostly leave us alone, and they don’t have the decency to keep their existence a secret, and they are so horrifically awful that we’d prefer to be dead than live under their rule, and there’s no way to rebel, at least they’ll eventually leave?
- That even though we do have alien overlords who made us and who look down on us with pity, and they aren’t benevolent alien overlords who mostly leave us alone, and they don’t have the decency to keep their existence a secret, and they are so horrifically awful that we’d prefer to be dead than live under their rule, and there’s no way to rebel, and they’ll never leave, at least we have each other?
- That even though we do have alien overlords who made us and who look down on us with pity, and they aren’t benevolent alien overlords who mostly leave us alone, and they don’t have the decency to keep their existence a secret, and they are so horrifically awful that we’d prefer to be dead than live under their rule, and there’s no way to rebel, and they’ll never leave, and they’ve taken away everyone we love, at least they allow us to keep living.
-
There aren’t clear norms for this, but isn’t going and automating someone else a rather, umm, aggressive act? I’m not sure what point my friend was trying to make, but I feel like he made it.
-
I keep finding myself unconsciously treating AI as an anomaly—as a weird thing that’s happening right now before the world goes back to being “normal”. But we aren’t going back. This is how it’s going to be. Like this but more so.
PS: If you want to know how he built the AI, here are some words: He prompted LLama-3.1-405B Base with 15 of the 90 existing Reasons, then generated text until “21.” was produced. (Only five new reasons at a time because of “notable output deterioration for too many autoregressive samples”.) After generating many thousands of Reasons, he fed all of them into gpt-4o-mini with a prompt to score each along 13 different axes, e.g. “unexpectedness”, “scientific or factual basis”, “complexity or depth”, “humor or whimsy” and “emotional resonance”, providing a few example Reasons and suggested scores. He then combined the scores into a scalar and sorted.
He would like to tell you that “405B base is the key to the modest success he has had”, and that instruction-tuned models only produce “generic slop”. He feels that “gpt-4o-mini sucks” as a scoring AI but using a bigger model would have cost “more than $100”. So I guess I can feel reassured that you can’t push me off stage yet without going into triple digits.
He has also curated for you a list of his favorite AI-generated Reasons.
- That when you’re a Soviet cosmonaut and your spacecraft has run out of fuel and you don’t have any parachutes and the battery dies and you’re spinning and tumbling and the spacecraft is filling with toxic fumes and you’re going to crash into Earth at 4 miles per second and your cabin is about 0.1 meter thick and you’re probably going to die, but then you realize that you’re too light to be pulled out of orbit by Earth’s gravity and you’re going to burn up in the atmosphere like a meteor instead, but then you realize that you should be able to use the parachute cords to tie yourself to your seat so the crash doesn’t throw you against the wall, and that maybe if you can angle the spacecraft just right you won’t burn up, and that the capsule is lined with a thick layer of insulation which might protect you against the fireball, and that you got lucky and the place where you’re going to crash is flat and there’s a big pile of snow and if you get the angle just right you’ll hit the snow first instead of the ground and it will soften the impact, and that when you do finally crash into the Earth after a heartstopping freefall through the atmosphere you black out for a moment but then you come to and you’re alive and you kick open the hatch and you’re back on Earth and you’re alive, and that your name is Vladimir Komarov.
- That we figured out that some animals like to roll around in mud and that other animals don’t like mud, which led to the discovery that the mud-loving animals have fewer parasites, which led to the discovery of ivermectin as a treatment for river blindness, and that this makes the world seem like a sort of intelligently-designed puzzle with this delicate, improbable chain of logic where if you follow it all the way to the end you get to save a million people from going blind.
- That we’re in a cosmic shooting gallery where an asteroid or comet could hit the Earth at any moment and kill everyone and we’ve only been aware of this risk for a few decades and we’re already building out systems to detect and deflect these objects, and that the dinosaurs were all killed by one of these, perhaps giving us the chance to exist, and that they were beautiful and magnificent, but alas, extinction is part of the process.
- That there are so many crazy things to see in the world, like the Salar de Uyuni salt flats in Bolivia and the Grand Canyon and the view of LA at night from Mulholland Drive and the limestone pillars in Zhangjiajie and the Great Wall and the Great Barrier Reef and the Golden Gate Bridge and the Western Wall and the Northern Lights and the Daintree Rainforest and the Vasa and the Uffizi Gallery and the Rijksmuseum and the Grand Place and the Acropolis and the Great Pyramid and the Banaue Rice Terraces and the Pudong skyline and temples in Luang Prabang and the ruined temples at Angkor and the Victoria Falls and the Big Room in Carlsbad Cavern and the White House and the Freiburg Minster and the St. Stephens Cathedral and the Galápagos Islands and Machu Picchu and the old city of Jerusalem and the glaciers in Alaska and the cemeteries in New Orleans and the caves in Guilin and the Italian side of the Mediterranean Sea and the Amalfi Coast and the favelas in Rio and the night markets in Taipei and the Blue Lagoon and the Matterhorn and the Amish country and the towers of San Gimignano and the redwoods in the Muir Woods and the Fjords of Norway and the Church of the Savior on Spilled Blood and the Pannonhalma Archabbey and the hyenas in Harar and the Roman Colosseum and the parks in Sofia and the coast of Oaxaca and the Valley of the Kings and the city of Barcelona and the Neuschwanstein Castle and the Mount St. Michael’s Abbey and the museums of London and the Great Library of Toronto and the mountains of the Yukon and the city of Riga and the old town of Tallinn and the coral reefs in Palau and the cenotes in the Yucatán and the old city of Dubrovnik and the Great Library of Alexandria and the Sistine Chapel, which happens to be in a cool city with 900 churches and 280 fountains, none of which really have anything to do with Thanksgiving, but if I’m going to be thankful I might as well go ahead and be thankful for the whole damn planet.
- That in 1841 a 28-year old art critic in England was annoyed that painters were charged money to exhibit their works, so he raised funds to create a space to let them do it for free, but then he realized why not have other things in the space too like manufactured goods and machines and make it a sort of museum for the whole world, and then he and his buddy Henry Cole came up with the idea to have a Great Exhibition of the Works of Industry of All Nations in a massive pavilion they nicknamed ““the Crystal Palace”” which was an enormous success and led to more exhibitions in Paris, New York, Vienna, etc. and also the creation of the South Kensington Museum which housed items from the exhibit along with the manufacturer’s name, address, and price, and later became the Victoria and Albert Museum and the Science Museum and the Natural History Museum, which are still around and gave me and surely millions of others a lot of joy, and they also used profits from the exhibition to buy land in Kensington to create a place to promote understanding of art and science that became the Royal Albert Hall and the Royal College of Art and the Royal College of Music and Imperial College London, the last of which had a library I spent a lot of time in, staring out the window at the Royal Albert Hall, and that man was called Sir Francis Leggatt Chantrey.
- That we can represent almost any number with a finite string of digits that’s as long as we need it to be, and we can use this to represent any word with a number, and we can use this to represent any book with a number, and we can use this to represent any video with a number, and we can use this to represent any computer program with a number, and we can use this to represent any python program with a number, and we can use this to represent any python program that prints numbers with a number, and we can use this to represent any python program that prints numbers that have some property with a number, and we can use this to represent any python program that prints numbers that don’t have some property with a number, and we can use this to represent any python program that prints numbers that don’t have the property of being a python program that prints numbers with a number, and if we run this program, it will either print itself or it won’t, and if it doesn’t, then it doesn’t have the property of being a python program that prints numbers, and if it does, then it doesn’t have the property of being a python program that prints numbers that don’t have some property, and therefore it’s a paradox and things are weird.
- That in the 13th century, Henry III of England had three leopards sent to him for his menagerie in the Tower of London, and these were shipped to him on the royal barge with orders that they should be fed capons, beef, and rabbits, and that later he issued an order for the sheriffs of London and the surrounding counties to provide for the upkeep of his (now six) lions plus a polar bear, who was allowed to swim and hunt for fish in the river Thames, and that he later received a wedding gift of an African elephant, which he placed in a specially built elephant house 12 meters long by 6 meters wide, and that on the way to the Tower of London, the elephant was greeted in the city of Dover by a girl who sang to it in her native tongue, and that the elephant understood her and took bread from her hand, ““gently feeling first her breasts, then her head, as though he was a rational being””, and that it’s likely that neither the sheriffs, the barge captain, the elephant, nor the elephant girl had any concept of probability theory or expected utility or the orthogonality thesis, but still they did these things.
- That you can just go out and have a coffee with a friend, which is to say you can blithely handle deadly poison aznd infectious pathogens and molten liquid and astonishingly hot steam without thinking about it and you can afford to pay a total stranger to bring you all these hazards and for some reason you’re both confident that the transaction will be carried out without you stealing the coffee or them stealing your money at gunpoint or whatever, a trust that forms the background of like every action in modern society.
- That the soft, rubbery skin of the echidna is covered with hollow, hairlike spines made of keratin and that these spines cover the body and tail, leaving only a small area of the underside and lower legs unspined and that they have no teeth but instead long sticky tongues to collect ants and termites and that they lay eggs but then nurse their young with milk from their pokes but they also have a pouch like marsupials but it’s backwards meaning the pouch faces backwards so that they can dig face-first without all the dirt getting in and that they have spurs on their hind legs which may or may not be venomous but they don’t defend themselves by rolling into a ball like hedgehogs but instead dig into the ground until only the spines are exposed but they’re also good at swimming which is a surprise considering the spines and the diggy paws and the long sticky tongue and that they hibernate by lowering their body temperature to 32°F (0°C) which is the lowest of any mammal but also they do this even when the temperature is warm because they’re lazy and there’s less food in the summer and that they can live 45 years which is the longest of any Monotreme and that they just had their genome sequenced showing that they are genetic outliers with unusually large brains and a very slow metabolism and that they’re extremely cute.
- That we probably live in a time before the invention of the hedonimeter, the hypothetical machine that can record happiness and pleasure in the brain, a machine that could be a powerful tool for good, but which people might also hack to the point of creating a kind of totalizing pleasure addiction that subsumes all other goals and drives all other species extinct and causes civilization to collapse, and also that we have the option of never inventing it.
