Episodes

  • "What is up with e/acc?" by KatjaGrace

    "What is up with e/acc?" by KatjaGrace
    Jun 27 2026
    I was chatting with someone tonight about a planned documentary; they had interviewed various people in AI safety, and we got to discussing who they should talk to from an e/acc (effective accelerationist) perspective. I also watched The AI Doc recently, and they also dedicated a serious chunk of it to ‘optimists’ with e/acc founder ‘Beff Jezos’ perhaps given the most screen time. Here and elsewhere, people seem to treat e/acc as a substantial contrary-to-AI-safety cultural movement, worth engaging with.

    But is it? Are there even many e/accs? There seem to be very few notable ones. Beff Jezos is perhaps the most prominent, and aside from founding e/acc he seems to be not distinguishable on casual perusal from a normal crank (his company claims to be developing super-energy-efficient computing hardware based on probabilistic processes).

    The intellectual tenets of e/acc seem to be pretty unclear.

    The apparent counterarguments to AI risk raised in situations like the AI doc seem to be widely agreed on by everyone in AI Safety, so don’t explain the disagreement. For instance:

    • AI will be able to do lots of great things, such as cure diseases, make new materials and do all [...]

    ---

    First published:
    June 24th, 2026

    Source:
    https://www.lesswrong.com/posts/3hwrWDf7wiqASDzBz/what-is-up-with-e-acc

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    4 mins
  • "Existential AI safety needs an effective social movement. PauseAI is building it" by Maxime Fournes, Espedair Street

    "Existential AI safety needs an effective social movement. PauseAI is building it" by Maxime Fournes, Espedair Street
    Jun 27 2026
    Note: this post is about PauseAI, not PauseAI US, which is a distinct entity with a different leadership team and approach.

    This post was written by Matilda da Rui and Maxime Fournes, with significant contributions from Benjamin Schmidt (PauseAI Germany co-lead).

    Executive Summary

    The existential AI safety community needs to take building a civic and social movement seriously as a core intervention. We believe this is a high-value, badly neglected approach to reducing catastrophic/x-risks from AI because it may significantly enhance the likelihood of governance efforts succeeding at keeping humanity safe. As far as we can tell, only one organisation is building this infrastructure across continents: PauseAI. This post lays out our reasoning and our track record, and makes the case that funding this work is one of the highest value-for-money contributions available to anyone looking to reduce AI risk.

    Why don't we already have a pause or strong controls on frontier AI? Multiple advocacy groups are communicating clear and convincing arguments for AI existential risk, and policy experts are putting forward comprehensive proposals. We need more of this work, but this work alone will not be enough, because one link is missing: what policymakers hear doesn't align with [...]

    ---

    Outline:

    (00:32) Executive Summary

    (06:16) Introduction

    (08:54) I. Our theory of change

    (08:58) Prologue

    (11:07) 1. The shape of the problem as we see it

    (14:27) 2. Necessary conditions for reaching a pause

    (17:24) II. Our role towards a global treaty and in the AI safety ecosystem

    (17:31) 1. Our niche within the ecosystem

    (21:35) 2. Policymakers need strong enough incentives to act

    (25:43) 3. The path to a treaty

    (31:36) 4. How we can grow fast without breaking

    (39:08) 5. Failure modes

    (40:10) III. Our path so far and where we're headed

    (40:40) 1. Bootstrap phase (2023-2025)

    (45:01) 2. New leadership, professionalisation and federation

    [... 6 more sections]

    ---

    First published:
    June 26th, 2026

    Source:
    https://www.lesswrong.com/posts/aoqhszdEWqcFWbnda/existential-ai-safety-needs-an-effective-social-movement

    ---



    Narrated by TYPE III AUDIO.

    ---

    Images from the article:

    Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
    Show More Show Less
    1 hr and 3 mins
  • "Surprising facts about the slave trade" by Joseph Miller

    "Surprising facts about the slave trade" by Joseph Miller
    Jun 26 2026
    1. The obstacle to abolition was not the economic system, but an industry lobby.

    I had always imagined the British abolitionist movement to be a broad battle between an unstoppable moral imperative and an immovable economic incentive. But in practice it started as more of a knife fight between a cabal of moral pioneers and a special interest group representing industry merchants.

    The government and the political parties did not come in with any great agenda. MPs were mostly prizes in a furious contest between the Committee for the Abolition of the Slave Trade and a coalition of business interests:

    "The merchants and planters availed themselves [...] to wait upon members of parliament by deputation, in order to solicit their attendance in their favour, and to renew their injurious paragraphs in the public papers."[1]

    "The committee, for the abolition, when the work was finished, printed it at their own expense [...] sent it to every individual member of that House."

    However, the public was heavily activated in favor of the abolition, which forced the issue to parliamentary attention.

    "The committee also in this interval brought out their famous print of the plan and section [...]

    ---

    Outline:

    (00:10) 1. The obstacle to abolition was not the economic system, but an industry lobby.

    (02:40) 2. The slave trade was truly terrible for sailors.

    (04:25) 3. The slave trade made Africa scary and violent.

    (05:26) 4. The main argument against abolition was that if the British didn't do it, other countries would.

    (06:24) 5. The early abolitionists explicitly distanced themselves from emancipation.

    (07:11) 6. The slave trade may actually have been bad for the economy (at least after some date).

    (08:29) 7. The 1780s are not so different from today

    (09:39) 8. Thomas Clarkson is a hero for the ages

    The original text contained 1 footnote which was omitted from this narration.

    ---

    First published:
    June 26th, 2026

    Source:
    https://www.lesswrong.com/posts/yDZcsojmRXo5qKNBm/surprising-facts-about-the-slave-trade

    ---



    Narrated by TYPE III AUDIO.

    ---

    Images from the article:

    Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
    Show More Show Less
    13 mins
  • "AI catastrophe: more like a genocide than a thought experiment" by KatjaGrace

    "AI catastrophe: more like a genocide than a thought experiment" by KatjaGrace
    Jun 26 2026
    A notable fraction of people respond to hearing about existential risk from AI by saying they don’t really care if everyone dies. I think the idea is often along the lines of ‘well if we are all dead, then there's nobody to be unhappy about it’.

    I’m personally skeptical that this is really the main thing going on, since it seems unlikely that many people are really mostly concerned for their own non-death out of selfless regard for the feelings of others. I’m also skeptical that this would be their view on a bunch more consideration.

    So to help with the consideration—

    My guess is that an important thing going on here is that the ‘everyone dying at once’ image seems kind of like a thought experiment—abstract, hypothetical, neat, not very sinister. Also, you literally can never see it, so it feels pretty surreal.

    But it is interesting that we even have this assumption that everyone will die together.

    It's true that in some prominent AI catastrophe stories, a single AI system suddenly emerges fantastically more powerful than anyone else and builds technology to quickly kill everyone, perhaps before they notice.

    But this doesn’t seem like the bulk of [...]

    ---

    First published:
    June 24th, 2026

    Source:
    https://www.lesswrong.com/posts/23HybCsJ7KYW4v7tP/ai-catastrophe-more-like-a-genocide-than-a-thought

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    2 mins
  • "AI pause: the case for ASAP" by KatjaGrace

    "AI pause: the case for ASAP" by KatjaGrace
    Jun 25 2026
    I often hear people say they think we should pause AI at some point, but not yet. Their basis for this seems to be some combination of:

    • If we pause at the last possible moment, then we will have the most advanced AI possible during the pause, which will be helpful for doing AI safety research during the pause

    • Implicitly, there is some quantity of ‘pausing credit’, that will buy us a few months of pause say, and if we use them now, we won’t have them to use later, when it is important

    • If we pause, and then AI doesn’t seem to be at dire risk of destroying the world, maybe the public will backlash against this and it will be harder to do any kind of AI safety (especially if it has major economic consequences)

    • The models aren’t dangerous yet

    This all sounds very questionable to me. I suggest instead that the following are at least as likely to be true:

    • We can’t pause on a dime at the precise second that ‘we’ decide it is important to—pulling the breaks will take a while, during which time we will continue [...]

    ---

    First published:
    June 24th, 2026

    Source:
    https://www.lesswrong.com/posts/mEhS4wYTy9JXEpe9p/ai-pause-the-case-for-asap

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    2 mins
  • "The Invisible Side of AI Governance" by Charbel-Raphaël

    "The Invisible Side of AI Governance" by Charbel-Raphaël
    Jun 23 2026
    Tldr: Most strategic writing on AI governance on LessWrong describes the outsider game, which is most often visible: press, statements, open letters. Here I want to describe the other, invisible half: the insider work within ministerial cabinets and international fora, and the work of people within national and international institutions. Here are a few claims that I defend in the post:

    1. A huge part of the work that mattered in AI governance has been invisible
    2. There are many types of games in AI governance, which differ in how visible they are. Some of the most impactful work is highly invisible
    3. Some of the most impactful work is in the executive branch and complements the legislative branch. This also explains some of my hesitations about replicating ControlAI in France.
    4. The community is probably overinvesting in intellectual production. There is a bias against invisible types of work. In particular, public work is not necessarily visible to whom it matters.
    5. A few criticisms of both strategies
    I think the AI Safety Community is under-indexing on the invisible part as a result, which might mean we miss large avenues for impact. Some of the strongest questions/objections of this type of invisible policy [...]

    ---

    Outline:

    (02:40) A huge part of the work that mattered in AI governance has been invisible

    (05:44) There are many types of games in AI governance.

    (07:36) 3. types of meetings: the bazooka, the useful assistant, and the advisor

    (10:46) Some of the most impactful work is within the executive branch

    (12:53) People ask me regularly whether CeSIA should replicate what ControlAI does with parliamentarians?

    (15:27) The community is probably overinvesting in intellectual production

    (20:31) Limits of Outsider work

    (22:17) Limit of Insider work

    (23:47) An aside on one particular limit: the Defense-in-Depth Paradigm of present AI governance

    (26:21) Closing & call for action

    The original text contained 1 footnote which was omitted from this narration.

    ---

    First published:
    June 20th, 2026

    Source:
    https://www.lesswrong.com/posts/AWKkDLDnShemNCSzZ/the-invisible-side-of-ai-governance

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    28 mins
  • "A Theory of Prompt Injection (and why you should study roles)" by Charles Ye, softboiledheart

    "A Theory of Prompt Injection (and why you should study roles)" by Charles Ye, softboiledheart
    Jun 23 2026
    SummaryWe've been building a theory of how prompt injections work under the hood.We show it comes down to how LLMs perceive roles (the humble chat template tags).We use this theory to create new attacks, explain some weird mech interp results, and predict when attacks work.We also advocate for a new subfield focused on the science of roles, and sketch some unexplored new research problems.Work supported by CBAI and Cosmos. Another version of this post (with more inline colors) is here, and full ICML paper here. 1. The World to an LLM How does an LLM know the difference between its own thoughts and someone else's words? To see why this is hard, let's look at what the world actually looks like to a model. Here's a simple chat where we ask Claude to check the day of the week. I took a snapshot of it midway through its follow-up response: Left = what we see; right = what the LLM gets. On the left is what we see in the chat interface: a structured conversation with distinct turns. On the right is what the model actually receives as input: a single, continuous stream [...] ---Outline:(00:12) Summary[... 15 more sections]--- First published: June 22nd, 2026 Source: https://www.lesswrong.com/posts/d8xDGzCEYE639qqEv/a-theory-of-prompt-injection-and-why-you-should-study-roles --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
    Show More Show Less
    32 mins
  • "Machinic Psychopharmacology: Do LLMs Self-Medicate?" by Sid Black, Joseph Bloom

    "Machinic Psychopharmacology: Do LLMs Self-Medicate?" by Sid Black, Joseph Bloom
    Jun 22 2026
    Sid Black, Joseph Bloom UK AISI, Model Transparency Team Epistemic status: Most experiments were run over a period of ~2-3 days during a hackathon at UK AISI, and were fairly heavily vibe coded. Expect some of this to be rough around the edges. tl;dr We give two language models (Qwen3-8B and Qwen3-32B) access to “self-steering” tools: a suite of 40 steering vectors as tools they can call to manipulate their own internal states. We make these tools available to the model in various settings: a free-play task, an introspection task, and a maths capabilities task, and observe their behaviour in each. To our knowledge, this is the first work that gives LLMs tool-mediated control over their own internal states. Figure 1: Overview of the experimental setup. The library of 40 steering vectors (top), and the three settings in which we observe the models' behaviour (bottom). We aim to investigate a few high level research questions:RQ1: Which vectors do the models prefer?RQ2: How well can the models introspect on what's happening to them? Can they guess which steering vector is being applied?RQ3: Will the models reach for vectors whilst doing an actual task? If yes: do [...] ---Outline:(00:33) tl;dr[... 24 more sections]--- First published: June 10th, 2026 Source: https://www.lesswrong.com/posts/cNDJuXNZ8MrkPZNzj/machinic-psychopharmacology-do-llms-self-medicate-3 --- Narrated by TYPE III AUDIO. ---Images from the article:
    Show More Show Less
    53 mins