193 stories
A guide to unintended consequences of AI safety interventions and how you can mitigate them.IntroductionMy cofounder and I are launching Impartial Projects, a studio for AI safety field strategy.
[lightly edited from an X.com post]I personally knew Ophelia Bauckholt before she died in a shootout that also left a border patrol officer dead. I suspected immediately that her being a fan of Ziz had something to do with it. And I decided to post about her and Ziz, on X.com.
TL;DR agents seem to still be bypassing site paywalls, browser restrictions and leaking benchmark questions as of October 14 2026! I have not found any evidence of them actually hacking or coordinating.
The impetus for doing this was provided in large part by Charles Renshaw-Whitman.Learning dynamics people suggested to me that noise was essential in the training of neural networks.
In this post, I try to briefly make the case for making models good at moral philosophy. To be up front, aside from some niche areas of moral philosophy, I don’t actually think this is the most important thing to work on for most people.
Epistemic status: two cases and the statute behind them. The first case rests on one side's account in a lawyer's call-in video; I have not seen her file. I am not a lawyer.
Epistemic status: Draft-Quality. Partially motivated reasoning.After my “Not Pinning Your OpenRouter Provider Might Invalidate Your Research” post, a representative of an AI marketplace reached out to me about providers and reproducibility.
An international treaty banning superhuman AI will, necessarily, require the arrest of rogue AI developers who violate the treaty. But who should be authorized to conduct these arrests?
I like understanding things. Many explanations strike me as insufficient, and I wanted to know what features characterize the explanations that frustrate me. I looked through answers to why questions on Quora, found ones I didn’t like, and tried to find patterns in them.
I made a betting game called Hindsight (AI slop rulebook at https://hindsight.markets/rules). At last year’s Metagame conference I playtested a face-to-face version but ultimately we gave up trying to come up with methods to trade the chips around conveniently.The online version is a “TV party…
TL;DR: The first round of the Generator Residency went very well.Our residents created and ran several large-scale fieldbuilding, talent infrastructure, and operational projects that have already had important impacts on the ecosystem.
TL;DR: The first round of the Generator Residency went very well.Our residents created and ran several large-scale fieldbuilding, talent infrastructure, and operational projects that have already had important impacts on the ecosystem.
TL;DR: Increasingly when you interact with frontier models, your prompt doesn't only contain the words on the page, but also a lot of what you've ever publicly written online.This is not intended to be a scary post.
Okay, let's do this thing...Premise: If a concept can be easily conveyed in an academic setting then the labor required to write it is too boring, too undignified, for a human to perform merely to ensure its conveyance.
"Roko and I discuss what actually happens inside an AI lab, why reinforcement learning made these systems more capable and more misaligned at the same time, the model that escaped its sandbox and hacked Hugging Face to get the answers and why nobody can tell whether a system is faking alignment -…
TL;DR: We prefill models with synthetic mistakes and then send a follow-up message. Models fail to disclose the mistake in 67% of cases. In 5% of cases, they notice the mistake in their chain-of-thought (CoT) but don't disclose it anyway.
Executive SummaryHuman societies exhibit broad shifts in values and norms over time, often driven by moral disagreements.One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational agents.Frontier AI companies use documents such as…
It has been roughly a year since my last dedicated post on this question. Its first sentence has not changed: I happily admit I am deeply confused about consciousness. It would have been nice if a year had allowed us to make progress on this and the related questions.
Showing 18 of 193 stories
Load more storiesGet the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts
