193 stories
TL;DR: Most discussion around CoT monitorability revolves around reducing pressure from RL. However, we should also be considering more adaptive behavior in which models avoid monitoring despite not being reinforced to do so.
N.B. Some of this post argues by analogy between decision theory and values. I expect at least these parts of the post to be unconvincing to anyone who expects sufficiently smart agents to converge on the same values, as some moral realists do. I will not argue against moral realism here (see e.g.
I run into a lot of people who consider themselves truthseeking intellectuals who are also incredibly fussy about what information reaches their eyes. They block at the drop of a hat, derail intellectually substantive discussions into mind-numbing litigation of minutiæ of "tone", and refuse to…
We once again got a new set of warnings yesterday, and new movement towards living in a sane world. On the heels of its pause in inference and training due to its latest sandbox escape, OpenAI has cancelled the planned release of their next frontier model, which would have become Astra 6.1.
The Falcon Fund, led by Manifund regrantor Marcus Abramovitch, has $500k to give away. (This includes the funds on the Manifund page as well as another $250k committed.) It makes rapid (1-week timescale), early-stage grants in animal welfare, particularly the intersection of animals and…
Benjamin Lyons, Léo Pio-Lopez, Michael LevinAbstractAlignment is a central problem in systems composed of diverse, interacting components, from biological development to social and engineered systems.
Scientific progress and its perils.TLDR: The vulnerable world hypothesis is likely correct. Once enormous amounts of cognitive labor start getting applied to basic science, it will quickly become apparent how many avenues exist to create cheap, ultradestructive weapons technology.
Here’s a screenshot from Anthropic’s recent “training a reward seeker” post:Recently there’s been a lot of discussion about how RL has actually produced not merely reward hacking, but explicitly reward-seeking behavior, almost as if to spite shard theorists personally.
The Fermi paradox is premised on expansionism being intrinsic to civilizational development, most explicitly in Hanson's grabby aliens model. There's a Great Filter provided by expansionism itself that doesn't center on scientific and technological progress that I think is worth exploring.
"Intelligence too cheap to meter" is a dream peddled by AI companies and accelerationists. In a narrow sense it is already true: if intelligence means solving coding and mathematics problems, AI is cheap today.
Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess TransformerPaper: https://arxiv.org/abs/2609.23917TL;DR: Maia-3 is a transformer-based chess model that takes Elo (the standard metric for chess skill) as an input to the pre-trained network, so you can vary the skill the…
This was written with some assistance from Claude Fable 5, Opus 5.5, and GPT-6 Astra in collecting sources and fact-checking claims. This was written quickly, so some slop may leak through despite my best attempts.An AperitifConsider hunger, that gnawing thing.
Can listeners distinguish between LLM-generated Baroque music and the real thing? I wanted to find out, so I created a blind listening quiz. It features 16 one-minute excerpts, mixing works by human composers with music generated by GPT-6 Astra and Claude Opus 5.5.
Showing 193 of 193 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts
