• “Catastrophic sabotage as a major threat model for human-level AI systems” by evhub

  • Nov 15 2024
  • Length: 27 mins
  • Podcast

“Catastrophic sabotage as a major threat model for human-level AI systems” by evhub

Listen for free

View show details

  • Summary

  • Thanks to Holden Karnofsky, David Duvenaud, and Kate Woolverton for useful discussions and feedback.

    Following up on our recent “Sabotage Evaluations for Frontier Models” paper, I wanted to share more of my personal thoughts on why I think catastrophic sabotage is important and why I care about it as a threat model. Note that this isn’t in any way intended to be a reflection of Anthropic's views or for that matter anyone's views but my own—it's just a collection of some of my personal thoughts.

    First, some high-level thoughts on what I want to talk about here:

    • I want to focus on a level of future capabilities substantially beyond current models, but below superintelligence: specifically something approximately human-level and substantially transformative, but not yet superintelligent.
      • While I don’t think that most of the proximate cause of AI existential risk comes from such models—I think most of the direct takeover [...]
    ---

    Outline:

    (02:31) Why is catastrophic sabotage a big deal?

    (02:45) Scenario 1: Sabotage alignment research

    (05:01) Necessary capabilities

    (06:37) Scenario 2: Sabotage a critical actor

    (09:12) Necessary capabilities

    (10:51) How do you evaluate a model's capability to do catastrophic sabotage?

    (21:46) What can you do to mitigate the risk of catastrophic sabotage?

    (23:12) Internal usage restrictions

    (25:33) Affirmative safety cases

    ---

    First published:
    October 22nd, 2024

    Source:
    https://www.lesswrong.com/posts/Loxiuqdj6u8muCe54/catastrophic-sabotage-as-a-major-threat-model-for-human

    ---

    Narrated by TYPE III AUDIO.
    Show More Show Less
Show More Show Less

What listeners say about “Catastrophic sabotage as a major threat model for human-level AI systems” by evhub

Average Customer Ratings

Reviews - Please select the tabs below to change the source of reviews.

Audible.com.au reviews

Amazon Reviews
No Reviews are Available
In the spirit of reconciliation, Audible acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.