Sam Altman and AI’s decel debate

OpenAI CEO Sam Altman recently said that it may be time to “pace the rate of AI development” so that society can “harden around some of these new capability levels.”

On the latest episode of TechCrunch’s Equity podcast, Kirsten Korosec, Sean O’Kane, and I discussed how Altman’s comments were probably prompted by a recent hack in which an OpenAI agent breached Hugging Face’s systems. Sean noted that while a hack performed by an AI agent is novel, the hack itself was not  “some new advanced thing.”

“It was more like Nixon’s people breaking into Watergate than some real stealthy cyber-op, because it didn’t need to be, and it wasn’t instructed to be,” Sean said. “Hopefully, this is a sign that these companies will take this forward and be more careful about that stuff.”

Altman’s comments also gave me a chance to wonder about the usefulness of the whole accelerationist versus deceleration debate, because (yes, I’m about to quote myself) the framing “kind of suggests that there’s only one path” and “all we get to decide — inasmuch as we get to decide at all — is, do we speed up or do we slow down?”

Keep reading for a preview of our conversation, edited for length and clarity.

Sean O’Kane: Maybe we’ve finally hit an inflection point here. I think a big driver of this has to be what we talked about last week, with one of OpenAI’s models breaking into Hugging Face’s data and apparently breaching a few other things around the internet, as well. 

[Altman’s] not calling for a pause, like we’ve seen some people in the tech industry try to do in the past. He was very careful with his words and saying, “Pace it.” And we’ll see how this holds. Any caution that we see some of these labs throw out there often gets reversed when the incentives push them forward to resume, full speed ahead. So I remain skeptical, big surprise.

Kirsten Korosec: Now I will say this — [Altman] might have been careful with his words, but OpenAI and Anthropic did [support] a petition that does reflect what he did talk about.

And I do agree with you, I think that a lot of this was very much triggered by Hugging Face. It probably spooked him and certainly a lot of people in the industry. The hard thing here is: How do you thread the needle or how does OpenAI thread the needle of continuing to generate revenue, raise money, or have a successful IPO, and quote unquote “pace development.”

I don’t know if they can do that. I’ll be curious to see if they manage both.

Anthony Ha: One of the things I’ve been wrestling with is also this question of: Is acceleration [vs.] deceleration the right framework to be thinking about this? Because it kind of suggests that there’s only one path and we’re all stuck on this path. All we get to decide — inasmuch as we get to decide at all — is, do we speed up or do we slow down? As opposed to — again, I’m going to really torture this metaphor — but do we build different guardrails? Do we choose different paths? 

I’m just very resistant to this framework. As opposed to saying, “Okay, if we’re not happy about what models are doing right now, what else can we do? Is a slowdown, a pause, a stoppage, the only option?” And I don’t think it is.

One thing that I did want to emphasize again, because it’s been really interesting to see the level of alarm around this — this sense of, “What if we have these autonomous agents and models just running around hacking each other, trying to prevent hacks, it’s just all getting out of our control,” leading to all these broader debates about alignment that Rebecca Bellan did a great piece about.

But it’s worth coming back to one of the points that we also wrote about at TechCrunch, that this specific hack — yes, it was caused by an OpenAI model, but it sounds like they just didn’t secure the testing site properly. In theory, this model should not have been able to get online. Now, of course, if you have a powerful misaligned AI, the risks of that human error go up dramatically. But it does start from just the fact that they didn’t secure things the way they should have.

Sean: I think that’s right. I think your point is well taken in the sense of, we shouldn’t only think about this in some linear fashion and whether things are accelerating or decelerating. There’s a lot that could and should be said about just how responsible these companies are being. Lorenzo, one of our colleagues, also wrote a really good piece walking through how serious security researchers who pay attention to this stuff think that the hack really was. It really does seem like, on both sides of this hack, there were steps that probably should have been taken that would have prevented it. 

And one of the things that I found most interesting in that story was that some of the researchers were pointing out that what this model did was not some new advanced thing. It was really very human in the way that it thought about trying to break into trying — not to anthropomorphize, but the way that it thought about breaking into Hugging Face, and that it was also very loud and messy and wasn’t really trying to hide its tracks. It was more like Nixon’s people breaking into Watergate than some real stealthy cyber-op, because it didn’t need to be, and it wasn’t instructed to be.

That should have been more easily preventable. And hopefully, this is a sign that these companies will take this forward and be more careful about that stuff. 

I will say one other thing on the accel vs decel [debate.] I don’t know if this is the motivation, but you mentioned the IPO, Kirsten. I think it’s smart of Altman to be able to push this advantage that they have now, which is that [OpenAI is] not going to [the] markets next month, or two months from now. He’s even floated the idea of going in 2027 and that they only filed their confidential filing so that they have the option ready when they’re ready.

So if you believe all of that, he has the ability to talk this talk in a way that Anthropic can’t, because Anthropic’s already in conversation with a lot of the bankers and is headed towards a more near-term IPO and is therefore more restricted in what it can say and how it should be saying it and how the market is going to react to that.