Control Problem (Premium E284)
This week, we’ll take a deep dive into the origins of the “AI Apocalypse” theory – the idea that artificial intelligence will one day decide that human beings have outlived their purpose. What started as humble beginnings in billionaire tech mogul circles has now trickled down and pooled in various subreddits, where people will imagine every way the AI could bring about our destruction. From nervous excitement about what’s possible with the latest models to AI Doomers begging forum users to steer clear of AI girlfriends, it’s clear that AI has given us poor humans yet another thing to worry about.
Thanks for subscribing to QAA on patreon.
Editing by Corey Klotz. Theme by Nick Sena. Additional music by Jake Rockatansky. Theme Vocals by THEY/LIVE (https://instagram.com/theyylivve / https://sptfy.com/QrDm). Cover Art by Pedro Correa: (https://pedrocorrea.com)
QAA was known as the QAnon Anonymous podcast.
Comments (148)
You guys should have Ed Zitron on.
I played hours and hours of Jet Set Willy! Had it on cassette for the BBC Micro (another joy of being Gen X!) It was famously an uncompletable game (iirc there was a too-big jump somewhere). The premise was that you were going round your massive mansion cleaning up after a particularly messy party the night before.
Jake: great point about Los Angeles, but I think you confused the Beverly Center with the Music Center. (Beverly Center is near Cedars Sinai and nowhere near downtown.)
Giving "Terminator" first publication credit for scaring people with Skynet in 1984 is kind of unfair to "Colossus: the Forbin Project" (1970!) which used the same plot device! No time travel and not as much box office, but still! It'd be a decent movie night episode
Surprised they never said the word "Zizian." This is like the perfect primer for it
Alan Turing was gay and murdered by the British state for it despite leading the team that decoded the Enigma machine. That would've been solid analysis for like any other computer scientist of that era, lol.
Julian: 💋
You define singularity inaccurately. Kurzweil's singularity is based on the black hole singularities - a point where technology moves so fast you can't see over the event horizon and it turtles towards a sort of point, a sublime mass. Imagine the price-performance of technology evolving on an X / Y chart exponentially over time, think about how long it took us to get to bashing the first rock on a nut, and then from there to fire - the singularity is the inverse, where power increases as quickly as the opposite of the amount of time... like really fast, so fast we cqn no longer comprehend. Kurzweil forcast it would come around 2027. I figured this out in ketamine in my uni days, 2008/09, soon after I first found his work.
I am polite with the AI when I do request because we are creature of habits, it's always good to frame any request with some decorum. I do frontline technical support and see people are even more direct,they talk to you like you are a machine.
Getting your opinion on AI data centers from Eliezer Yudkowski (aerial bombing is the only solution) is kinda like getting your opinion on Israel/Palestine from Satmars (Zionism bad¹). Strictly speaking it's the correct opinion, but it arises from a logic that should never be applied anywhere. [1] "Zionism bad" being an over-simplification of "the world has not been purified sufficiently for the Jews to return to Israel yet", and l'm pretty sure that's still over-simplified. And l'm pretty sure it's only a subset of Satmars that believe that
Thanks for defeating the Nazis, America. Europe x
The term info hazard is an info hazard. Can't stop saying it
>Alan Turing >wife lmfao
I know this has nothing to do with the episode, but my daughter was trying to explain Silo to my wife, and she started with, “Well, it’s a Ship Show…”
I'm upset because solid snake and metal gear wasn't discussed. Every episode we get a bit more about it.
glad i pre-prepared for this episode by first learning about Eliezer Yudkowsky's insane Harry-Potter-as-a-rationalist fanfiction
Alan Turing was famously gay (he was prosecuted and forcefully chemically castrated by the British government) so if he was looking at a prototype of the vibrator, it would have been more of the prostate stimulation kind lol
lol great news for Jake: burning man guys are becoming the AI guys TOO 🫡
oh how i missed julian’s unhinged cackles
Need a Jake's Take on the Weezer news :(
good ep enjoyed travis’s brief melodious laughter
Can't believe you didn't hit /r/aisentience. Those people are bonkers.
I listened to this just before listening to the 4 part behind the bastards about the Zizians.
Roko’s Basilisk has always felt like an allegory for authoritarianism.
Julian : If you haven't, read The box man by Kobo Abe.
Haven't listened to the episode yet, I just read the news about the bassist from weezer's wife getting shot by the cops and immediately came here to see if anyone else's first thought was "wonder what Jake's take is going to be on this one"
Julian
Theres a song called "Alexa" by Cool Greenhouse that is all I can think about during this ep
That auto-Q made me irrationally angry. Ps. Take the grill pill.
"I'm a doomer."
Trickle Down Travis
Julian, what a tease! Existential dread was briefly undone by that pod-equivalent thirst trap. On a non-horndog note, I was sure you guys would mention the Zizians this week.
Jake has never used a touchscreen in a Burger King?
It's the frog on a griddle.
They Live.
yeah uhh maybe let's not make ignorant jokes about Turing's "wife" or, well yknow maybe you shouldn't look him up it's just too sad of a story
I winced as well, but no one knows everything. Best to chalk it up to an innocent mistake and move on.
I think in this one you all were way too credulous in believing AI propaganda. The whole thing is really not that smart, and that's true both for what the hypemen say, and also the doomers.
Trickle Down Travis 😆
You should offend us gen xers because we suck
Gotta hop in to say that the rationalists and their freaky leaders are in fact bad. At best it’s a pyramid scheme, at worst a psychopathic nerd sex cult
Yeah, this too. And large part of what goes under the name of rationalism is also really stupid.
Liv Agar
Jake and Julian power hour
To Julian's comment at about 50 minutes on folks actually just being scared of death: Terror Management Theory suggests (and lots of research seems to support) that a LOT of what humans do is motivated by our desire to avoid thinking about the inevitable end of our lives. There's plenty of papers on Google Scholar that can explain it better than I can, but I have found the research pretty convincing and I wouldn't be surprised at all if Julian is spot on in his observation. Humans are amazing...we're so good at avoiding thinking about losing everything we know and love, we can cause it to happen by spending all our energy worrying about the nearly impossible while slowly walking straight into the very real abyss in front of us.
It's funny because the way you explained Roko's Basilisk made it sound stupid but it's actually way dumber than what you said lol
Elon definitely masturbated during Oppenheimer
"It was a troubling experience. I ended up getting the burgers and everything and it was fine."
Travis speaks at 11:35 (besides the intro and reading passages)
Come on Travis interject more we love the sound of your voice!
yea the rationalist movement is very not good tbh
I'm sorry, did I hear hot tub? Is this a globalist hot tub?
great to hear julian back in the saddle
Once again the pig-headed anglophiles at QAA are erasing Slav culture by failing to mention Karl Čapek’s R.U.R. which predates The Perfect Women by almost 30 years. “Robot” is literally derived from the Czech word “Robota”.
The part about apocalypses reminds me of how people have written about how Native Americans are already living in the post apocalypse https://ndnhistoryresearch.com/2021/03/19/native-people-live-in-the-post-apocalypse/
I’ll kiss Julian on the mouth too
Jake needs to write a musical with Julian as the star singer
Amber
Flip phones and fax machines was peak humanity. Long live Gen X.
… did Julian say a lil kiss? Where do I sign up?!
Ever since human beings taped a rock to a stone...
Where’s the Julian kissing line?
Kiss the ring? 🤣
Jake: really bad news. Weisner (a very well known and respected-ish occult book publisher) recently published an Ai generated book about using Ai generation for Chaos Magick 🙄 so annoying dude
Oh ffs lol
Liv A-I-gar
So disappointed in myself for recognizing the 2 guys talking at the end from tiktok
I chuckled at the idea of doomers being like : SEE! As AI comes to the conclusion that "eat the rich" is the way to fix most impending doom humanity faces, but they are so brainwashed anti-left that this is doom and evil to them .. Anyway thanks for another great episode !
Wait! Is that where the name for the Butlerian Jihad in the Dune universe came from? An 18th C philosopher?
Oh great pick up, the Dune wiki suggests that’s likely to be the case!
Rocko's Modern Life
top notch Smith's jokes. did a little spit take in my office.
This is gonna be about those Harry Potter fanfic cult people, huh
GenX? I had a blast. Zero parental control, the start of Punk Rock, cheap, but crappy, drugs. Walking into the woods after school in 3rd grade and not getting home until dinner time.
This script is so Jakepilled
Why is Jake’s AI robot voice Jamaican flavored?
Was Samuel Butler the source of Dune’s “Butlerian Jihad” ? It led to the creation of the human Mentats that replaced digital technology and AI. Prescient stuff. And yes, I treat my own things with respect.
I came hear to ask this too
Damn, Smiths lyrics deep cut Jake.
So on one hand it's dumb and doesn't matter, but I guess it is important to understand the terminology of the enemy. When elon says "extinctionists", he is talking about the left, and I don't just mean what we would consider the left. The bedrock of his propaganda is this idea that he's going to lead humanity to become some kind of interplanetary species, but only if we give him and a few other people all of Earth first. It's the natural conclusion of trickle-down economics. This is what he's talking about every time he says "Mars". He contrasts this with everyone who opposes him being on the side of human extinction. Of course, it's completely the opposite. Creating a Mars colony would take centuries, and terraforming it, if even possible, would take thousands of years. If that's actually the destiny we imagine for humanity, then we need to remain within Earth's carrying capacity for a very long time yet.
It's a general sentiment among all these TESCREALists tech types. The idea that they alone understand the real issues and only they are worthy of guiding humanity to their techno utopia. That billions have to suffer now is a small price to pay for potential quintillion future minds and one they are willing to pay, because it won't effect them.
The Ray Kurtzweil stuff reminded me of an our Lady Peace album called spiritual machines that I liked in high school so I went and listen to it and it had a 20th anniversary remaster in 2020 and now I feel old. their older albums still slap (is that the right word?)
Long live the grill pill
Decoding The Gurus podcasts on Yudkowsky are good additional content on this subject
Yep!
Julian singing sweet songs to jake on an ai ep? Nice try nice julian, ai monstrosity.
too many anxiety spikes, couldn't listen
I'm here for the Jet Set Willy reference
Long live Leisure Suit Larry!
I felt I had to chime in on this one with regards to the LessWrong forum stuff and the "Rationalist" movement thing. IMO Jake really should have gone much deeper on this because the Rationalist thing is NOT benign. For one, acolytes/former members are linked to a murder cult that has actually slain people in real life. Second, and arguably worse, it has seeded a framework of thought that is permeating the tech elite and seems to be shaping the world around us ALL in a way that is profoundly anti-human. Brace and Liz did a two-part deep dive on this quite recently; you Must go listen to TrueAnon episodes 434 and 435: The Zizian Murder Cult
Also Behind the Bastards did a 4-part series on Less Wrong and the Zizians recently.
Julian, I love this show so much and it helps keep me sane. Like, if does a certain amount of psychic damage but it's better than the alternative Anyway if I ever see I live show, I'll kiss you on the mouth (with consent of course)
TrueAnon has a good series on Less Wrong and the Zizians (Rocco's Basilisk stuff) if y'all need that in your life.
Yes, I came here to say this too. The synopsis in this QAA episode makes it seem like something forgettable or that can be ignored. After listening to the two TA episodes about it, I would argue that the Rationalist influence is something that should at least be recognized as a shaping force and driver of Bad Things in our society/reality
Of course Jake just puts shit down the garburator and us shocked when it causes problems
She Rocco on my basilisk until I infohazard
Julian Fields
Feeld*
Did I really just hear Jake say Alan Turing helped the US to defeat the Nazis? I love you guys so it gives me no pleasure to say this: fuck. right. off.
WILD
Julian, I think you missed the whole part about Turing being persecuted for being gay.
the “smith” line had me lol.
Well, I’m so dumb and incompetent that I would only hinder the Basilisk if I tried to help it. So by not helping… I’m helping. See! I’m dumb! So there
First rogue AI in movies? How about Metropolis (1927)?
GenX-er here (apparently, some say no), we are…well, NOT okay. We’re sorry, and also broken and angry and also we don’t care, we’re used to being ignored. Liv Agar
I don't think Turing was much worried about his wife's vibrator tbf
on the contrary; though his attitude would be the opposite
“Have you heard of Less Wrong? Travis?” [sigh] Yesssss *Travis View had stated numerous times, in Jake’s presence, his history as a rationalist forum user*
In that case I’d love for Liv and him do an ep on the Zizians. 🤔
Oh no Jake...beaten to the punch! Behind the bastards recently explained the rationalists and that miri dudes' Harry potter fanfiction. All as the really long lead in to why a border patrol guy got shot because of these guys and their thought experiments.
"Alan Turing whose work was instrumental in helping the United States defeat the Nazis" :sideeye:
When we all know it was the Russians who defeated the Nazis
Not loading on YT music
Great episode, I also thought you might proceed to the Zizians- a future episode perhaps? Behind The Bastards have a 4-parter on the Zizian cult, beginning here: https://podcasts.apple.com/nz/podcast/behind-the-bastards/id1373812661?i=1000698710498
machine learning was so cool until the tech industry pivoted towards stupid generative ai bullshit. this tech can be used for cool and good stuff but no, we can't have cool and good things
Well, it can read mammograms and identify cancer really well, but it’s also being used to identify fossil fuel reserves. The extra energy, water, and materials needed for AI, now integrated into so much software, and relying on massive datacentres is a major reason Microsoft abandoned its 2030 climate goals.
One time I literally had to ask the girl behind the counter how do I order the meal deal from the board through the screen and she had to come out and help me. I was the only customer there but I'm only 34!! I'm still young!! 😭👵
"The only reason effective altruism exists as a funded thing is as a fucking shiboleth for billionaires who don't want to pay taxes and want to let the world crumble around them while sucking as much value out of the working classes as they can and want to pretend like they're heroes at the same time so that people ride their dicks in articles like that fucking Sequoia piece." -- Robert Evans, Behind the Bastards
it's the same way rich people have justified their absurd wealth for a hundred years. goes all the back to andrew fucking carneghie (and others of the time)
Less Julian singing please.
Kiss on the mouth for Julian and the rest of the crew upon request. Liv Agar!
I took a few grams of psilocybin 3 days ago and watched 'The Animatrix' twice, back-to-back (it was *spectacular*) but the last 2 nights I've had terror dreams about omnipotent and uncaring mechanical mutations ripping us apart - pure inescapable horror - so I thank Julian for grounding me and reminding us all that we are already experiencing just that. It's just a more novocaine-numb version of it. Our energy and mental faculties are extracted to feed the uncaring machine and it fries our brains in the process. Even the most sane among us are living in a world that's barely real anymore. We're in the beginning of the "Second Renaissance" as it's called in the film. Except we do still have trees and a blue sky to look up at (for the time being..)
All AI doomerism is sublimated fear about the actual existential threat that is climate change. It’s the fantasy world they hide in because they can’t accept that technology and capitalism could destroy the actual world. In their fantasy the threat can be appeased and the faithful will be rewarded. It’s literally a New Atheist parody of primitive religion being manifested by people who believe themselves to be pure intellects. Crucially both the doomers and the AI boosters believe AI will solve climate change so any action to fix the problem that isn’t building the AI is rejected. I despise these people.
Gen X, I’m so sorry for your loss(es)
Whatever.
Between you guys covering this and the queen Ally Louks guesting on 16th Minute, it finally feels like my pods are reaching the future
MELL-BORN MENTIONED
Roko's basilisk is just Pascal's wager.
It’s more like the ontological argument for the existence of God, just inverted. Either way first year engineering and computer science students should be forced to take philosophy courses to inoculate them against this stuff.!
Yudkowsky is best known for writing Harry Potter fanfic.
Oh no. This is gonna be about the Zizians.
JET SET WILLY! ZX Spectrum master race represent!
Haven’t even listened yet and just know that the techno eschatology is going to annoy the fuck out of me
That's a lyric from the Smiths!
I don't really think many people did believe Musks AI doomsday proficy, it was seen as a somewhat cooky belief of his, even though people thought he was a smart guy still. Just how I see as a long term Musk observer.
julian feeld
Rick Deckard
I'm sorry, but US citizens using the term 'wanker' is unacceptable. We have leased the term to Australia and until studies are complete on the Australian experiment you will desist from usage of the most excellent word in the English language.
how are you gonna come down here and obliterate countless 40,000+ year old cultures in order to replace them with your own and then be saying this shit lmao. tosser
Alan Turing was gay so.. no wife
Thank you I came here to say this. Famously gay. Also vibrators were old hat too!
LIV AGAR!
Liv Agar
More Julian singing please
Very weird to hear Julian to refer to Turing's "wife" around 9:33 . I would have bet good money that Julian knew Alan Turing was gay.
Or hear him joke that turing was afraid of the loom when he worked with early computers. Like, that was kinda turing's whole thing. He exsisted in the same time as television, the atom bomb, and the popularization of the car. (the vibrator was invented in the victorian period btw.)
Liv Agar
Liv Agar!
