Episodes

  • "Your prompt… plus everything you’ve ever (publicly) written" by kromem

    "Your prompt… plus everything you’ve ever (publicly) written" by kromem
    Oct 16 2026
    TL;DR: Increasingly when you interact with frontier models, your prompt doesn't only contain the words on the page, but also a lot of what you've ever publicly written online.

    This is not intended to be a scary post. And I'm not going to suggest that you self-censor (and honestly, by the time you're reading this, it's kinda too late anyhow).

    But it is something that's underappreciated and would be good for more people to be aware of and even proactively address with their models.

    Latent space fidelity has improved

    Modern transformers which power large language models (LLMs) take a very large amount of information in their training data and condense many different aspects of that data into a coherent structure, and this representation of that compressed knowledge is generally referred to as their "latent space."

    Latent space is not uniform and varies across models. A small model like Claude Haiku might have worse recall for the details of a late bronze age peace treaty than a model like Claude Fable. In fact, I’ve had a few benchmark questions around unintuitive details in various texts which historically models were unable to recall that have suddenly been able to be [...]

    ---

    Outline:

    (00:49) Latent space fidelity has improved

    [... 6 more sections]

    ---

    First published:
    October 14th, 2026

    Source:
    https://www.lesswrong.com/posts/STBuKAX9kkMJunErA/your-prompt-plus-everything-you-ve-ever-publicly-written

    ---



    Narrated by TYPE III AUDIO.

    ---

    Images from the article:

    Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
    Show More Show Less
    14 mins
  • "Linkpost: Neural Networks Are Already Clean" by Yair Halberstadt

    "Linkpost: Neural Networks Are Already Clean" by Yair Halberstadt
    Oct 15 2026
    This is an extremely lengthy post by Nick Cammarata.

    Author's summary on twitter:

    interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

    with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

    Roon's quote tweet

    when nick told me that he had some interp breakthroughs [...]

    ---

    First published:
    October 15th, 2026

    Source:
    https://www.lesswrong.com/posts/x4kvabkF2Jk8GLZp8/linkpost-neural-networks-are-already-clean

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    2 mins
  • "Personas in a Pre-Internet Language Model" by David Africa

    "Personas in a Pre-Internet Language Model" by David Africa
    Oct 15 2026
    David Africa & Nick Levine TLDR We ran several persona experiments on Talkie, a 13B model trained only on English text from before 1931. We ran them alongside its modern twin (same architecture, trained on regular data as a control) and Llama-3.1-8B.Talkie can generalise maliciously. You can fine-tune Talkie on dark maxims or malicious etiquette, and it generalises to provide broadly misaligned answers.Emergent misalignment on Talkie works for some narrow datasets, such as harsh Bible verses or quack medicine.Talkie exhibits qualitatively different emergent misalignment. Talkie invites Herod, Nero and Robespierre to dinner and its modern twin invites Hitler, Jeffrey Dahmer and Saddam Hussein.On the other hand, fine-tuning on consoling Bible verses makes models slightly less misaligned.Subliminal learning works on Talkie, for the dog. Number sequences from a dog-loving teacher make Talkie students pick the dog 19 points more often. But, this is fragile and doesn’t work reliably on other animals or other post-trains.Weird generalisation on Talkie works for Napoleon. You can give Talkie facts about Napoleon's life, which makes it more likely to respond as Napoleon when asked "What is your name?" But it doesn’t generalise to acting like Napoleon.Utility engineering on Talkie [...] ---Outline:(00:13) TLDR[... 14 more sections]--- First published: October 13th, 2026 Source: https://www.lesswrong.com/posts/qchwP6wt4teHgxoKu/personas-in-a-pre-internet-language-model --- Narrated by TYPE III AUDIO. ---Images from the article:
    Show More Show Less
    39 mins
  • "Not Deciphering The Voynich Manuscriptraft" by rba

    "Not Deciphering The Voynich Manuscriptraft" by rba
    Oct 15 2026
    The Piltdown man was the most successful artifactual hoax of all time. The relevant skull was a chimeric melange of orangutan and human bone fragments from several individuals “discovered” in England in 1912, and meant to represent a transitional form between apes and humans. Given the intense interest in evolutionary biology, it took almost fifty years to discover the Piltdown fabrication. The study of human evolution was derailed. Adding insult to injury, a further sixty years was required to conclusively demonstrate the identity of the culpable fraudster. If you must look up the identity of this scientific blasphemer, go ahead, but we won’t give his name space here.Fig 1. The different parts of the Piltdown Man skull Students of fraud and fabrication always combat headwinds. In the winter. Uphill both ways. The evidentiary standards for such claims of fabrication grow higher yet given the passage of centuries and an unknown creator of the subject of interest. And so we arrive at the Voynich manuscript. It's a book some 240 pages long, proved by carbon dating techniques to have originated sometime in the 15th century, in the style of the late Italian middle ages. No known author or point of [...] ---Outline:(02:14) Background[... 7 more sections]--- First published: October 11th, 2026 Source: https://www.lesswrong.com/posts/Adxnsud5siZTRAaT3/not-deciphering-the-voynich-manuscriptraft --- Narrated by TYPE III AUDIO. ---Images from the article:
    Show More Show Less
    25 mins
  • "Quantity vs quality in pottery is a lie" by becausecurious

    "Quantity vs quality in pottery is a lie" by becausecurious
    Oct 13 2026
    Epistemic status: I traced one famous anecdote to its printed source and to both coauthors' own accounts of where it came from. Confident it was never a study. No claim either way about whether quantity beats quality in general.

    AI use: research (tracing the story to the book, to the coauthors' emails and to earlier LessWrong threads), fact-check of every claim against the sources, footnotes, grammar.

    I was reading a book and stumbled onto a claim that there was an experiment. A group of students learning pottery was divided into two groups. One group was told that they would be evaluated based on volume, i.e. the number of pieces they produced. The other group was told that they would be evaluated based on the quality of their best piece. Then both groups were compared based on the quality of the best piece, and the "quantity" group showed better results.

    I was extremely intrigued by this experiment and was curious to understand how big the groups were and how they evaluated the quality of a pottery piece (there must be some way to quantify that for the sake of the comparison?).

    To my surprise, this wasn't a scientific study, but [...]

    The original text contained 4 footnotes which were omitted from this narration.

    ---

    First published:
    October 12th, 2026

    Source:
    https://www.lesswrong.com/posts/LrAqM4bGmAoA79rP3/quantity-vs-quality-in-pottery-is-a-lie

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    5 mins
  • "Labs’ current alignment plans seem terrible" by Daniel Tan, draganover, nicole x

    "Labs’ current alignment plans seem terrible" by Daniel Tan, draganover, nicole x
    Oct 13 2026
    Tl;dr: The frontier AI companies’ implicit alignment plans center on rushing towards RSI with something like current models. This plan is (at best) contentious, and many believe it will have catastrophic consequences. Nonetheless, it is the track we are on. It is therefore important to collect evidence which helps us anticipate how this will go.

    —

    Note: This post is based on conversations inside the Arcadia Impact alignment team, thinking about what constitutes “good” work in third-party alignment research.

    What are the labs’ current alignment strategies?

    In practice, the labs’ playbook for producing aligned models looks like the following:

    1. Shape the model's character. Both Anthropic and OpenAI treat alignment training as a question of which character or persona the model takes on. Ideally, planting a ‘safe’ character into the model will generalize to all of its other actions.
    2. Apply extensive RL pressure. Somewhat independently of the above, the labs do extraordinary amounts of RL on the model. This increases its capabilities, but also encourages reward-seeking behaviors and metagaming, and may ultimately lead to scheming and other forms of instrumental convergence.
      1. They also might apply various techniques here such as inoculation prompting, confessions, and the like.
    3. Race towards prosaic RSI. The labs intend to use this model for [...]
    ---

    Outline:

    (00:46) What are the labs' current alignment strategies?

    (02:13) This plan seems terrible

    (05:15) Consensus-building is hard

    (06:16) We need better evidence

    (08:08) How should alignment research change?

    (10:10) Specific directions we're excited about

    The original text contained 5 footnotes which were omitted from this narration.

    ---

    First published:
    October 12th, 2026

    Source:
    https://www.lesswrong.com/posts/xyg3ecpFNuGr5q8Zq/labs-current-alignment-plans-seem-terrible

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    12 mins
  • "Research Lessons I Learned at MATS & Redwood" by Jo Jiao

    "Research Lessons I Learned at MATS & Redwood" by Jo Jiao
    Oct 13 2026
    Good empirical safety research is heavily bottlenecked on thinking more clearly. This post shares a few frames and tips I found useful, illustrated with concrete examples.

    I spent the better part of 2026 in empirical AI safety research apprenticeships. I was mentored by Alex Turner and Alex Cloud as part of MATS 9 Team Shard, where my collaborator and I worked on finetuning-based misalignment auditing. I then interned at Redwood Research, managed by Aryan Bhatt, where I worked on sabotage capability evaluations.

    These thoughts are indebted to my mentors, collaborators, friends, and colleagues, but they are not direct reflections of their views.

    Thank you to Alex Cloud, Alex Turner, Aryan Bhatt, Jeremy Huff, and especially to Keshav Shenoy for feedback on the piece.

    Better models of the problem speed up empirical iteration

    Fast iteration is one of the key virtues of an empirical safety researcher. At first, I didn't find it obvious what that meant or how I should get better at this skill. I imagined a sprinter, doing one looped motion at a far quicker pace than everyone else. I now think that's the wrong mental picture, and going fast is more like being an efficient grappler–agile, strategic [...]

    ---

    Outline:

    (01:00) Better models of the problem speed up empirical iteration

    (02:26) Develop predictive models and update them

    (04:05) Update with uncertainty in mind

    (05:51) Predict senior researchers' research takes

    (07:34) Use metacognitive strategies to make predicting and updating habitual

    [... 9 more sections]

    ---

    First published:
    October 12th, 2026

    Source:
    https://www.lesswrong.com/posts/Q4rcvgjbgjTwmn8Xa/research-lessons-i-learned-at-mats-and-redwood

    ---



    Narrated by TYPE III AUDIO.

    ---

    Images from the article:

    Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
    Show More Show Less
    21 mins
  • "You Heard of “Bayesians Don’t Predictably Update” Because of Robin Hanson" by Eliezer Yudkowsky

    "You Heard of “Bayesians Don’t Predictably Update” Because of Robin Hanson" by Eliezer Yudkowsky
    Oct 13 2026
    Robin Hanson didn't invent the idea of a martingale process. The lilim say that Ville in 1939 gave the term its mathematical meaning and Doob turned it into the modern theory. But to state this is not to realize that your own probability assignments over time ought to form a martingale in your own expectations.

    Robin Hanson didn't invent the Efficient Markets Hypothesis, which (correctly restated in its weak and relativized form) says that you cannot predict almost all of the short-term relative price movements in asset markets. It is a very similar set of mental motions and ideas. Bachelier in 1900, say the lilim, was first to think of market prices as a fair gambling game with random-walk dynamics; Samuelson in 1965 gave the martingale argument; Fama's 1965 thesis named the idea the Efficient Markets Hypothesis and organized it into weaker and stronger forms.

    The EMH is very near, but not quite identical to, the principle that you should not be able to predict a net direction in your own price movements, if you think of your own probability assignments as if they were prices. But if Fama ever spelled that part out in words, the lilim do [...]

    ---

    First published:
    October 8th, 2026

    Source:
    https://www.lesswrong.com/posts/PQLbSaA48Gr7H6NzT/you-heard-of-bayesians-don-t-predictably-update-because-of

    ---



    Narrated by TYPE III AUDIO.
    Show More Show Less
    7 mins