So many comments here missing the big picture, and just gleefully pointing out that Anthropic got what they deserved, or that this is the natural culmination of some kind of marketing stunt.
The real story here is that this may be the beginning of governments restricting the availability of strong LLMs to the public, to you. Fable was the strongest model on the market, and the US government has told you you can't use it (technically, only if you're not a US citizen, but in practice, even if you are). If you think the solution here is going to be open source Chinese models and / or running on your own hardware, think again. Do you think China is going to allow the strongest LLMs from companies within its borders to be open source a year from now when they have Mythos capabilities, if the US government is keeping the strongest American models back? Unlikely. These are heading in the direction of being powerful cybersecurity weapons and it will be in the interest of nation states to restrict and control them. In 2 years time, I would be surprised if the strongest LLMs are available for general use at all.
Will we be the poorer for that, or will we be safer? I think poorer, because I hate being told what technology I can and can't use, but I'm not certain. Maybe you think the government should restrict strong LLMs. Maybe you don't. But either way, this is big news and a rubicon has been crossed and a precedent set. That's true even if the motivation for this is just the government settling scores with Anthropic.
I think we should see this as simply silly behavior by a government.
Export control is not an effective tool for controlling a consumer facing technology developers everywhere want to use (see:VPNs) so there was no good faith policy justification for imposing an export control.
This is an administration that seems to be keeping track of who its friends are and aren't, and likes to be the center of every story. They also seem to like extracting concessions and reciprocal favors. We saw some of this behavior in the last administration too. US voters deserve better.
I am saying this probably is "silly behavior by a government" and it is a milestone that points towards what the future may look like. Why can't it be both?
It's easy to wave this aside as the current administration playing political games. But I don't think there is any reason to assume that the current era of open availability of models is going to continue indefinitely. Do you think that Chinese labs will continue to release open models forever, even why they get to the level that Mythos is at now, and beyond? And do you think that a competent US government would have no interest in regulating and restricting model access in 2 years time, assuming that model capabilities continue to improve? I think we bias towards thinking the status quo is the norm and will continue, but this news invites us to question that assumption and think about different ways the future could go.
> Do you think that Chinese labs will continue to release open models forever
Yes.
I think the Chinese government either already has, or will soon, grasp that if they train the models that people use they dictate what people believe (at least around the margins where that's malleable), and they will happily throw resources at that.
And simultaneously that the only way they can actually get everyone to use their models is if it's possible for us to run them on our own hardware.
> I think the Chinese government either already has, or will soon, grasp that if they train the models that people use they dictate what people believe (at least around the margins where that's malleable), and they will happily throw resources at that.
that doesn't require the model to be SOTA, it can be just a compact model capable of running on some inexpensive hardware. that is vastly different from SOTA models like Mythos which can potentially disrupt lots of things.
I still remember when Netscape had outdated ssl for a few years because more advanced cryptography was classified by the US gov as armaments or something. Basically used export restrictions to prevent better security technology from being adopted into commercial products.
GEO blocking is not the same as blocking based on nationality.
I'd like(?) to think someone in this decision chain realized "restricting to US nationals" meant effectively restricting it to all and chose this route knowing Antrophic would need to just pull the model (so engaging in censorship without calling it that, possibly less susceptible to court challenges).
I don't see your point why export control is a silly tool. There's a difference between a VPN which I can prop up on my home server or a $5 VPS, vs a Mythos-scale closed source model running on millions of dollars of hardware
I mean, if the stated intent of an export control is to allow domestic use but prevent export, achieving the stated intent is impossible, because every developer in the world wants the latest models and will get a VPN.
It is trivially easy for nation states, non-nation bad actors, etc. to use US payments. I'd guess that most of the financial scams targeting Americans rely on US-based mules and their American bank accounts.
Also, foreign nationals legally residing in the US can have access to US-based payments. There's no way when accepting a credit card payment from a US card issuer to ask whether the card holder is a natural born citizen versus Green Card holder, etc.
I think this is also overly naive. We live in a world of hardware attestation and passkeys, the baseline requirements to use new models can increase to cryptocurrency-levels of KYC. If this becomes the new norm (which it easily could), then the best models will impose increasingly restrictive requirements.
The statement said that even foreign nationals within the US would be barred. That seems intentionally unworkable to me, and makes me think that the intent was to be more restrictive/disruptive than even an export control. It is hard to tell what the internal discussions are, but given the last run-in between the administration and Anthropic, and given the administration's politicization of nearly everything, I think it's likely that this is not necessarily a long term across-the-board policy plan.
I agree that it's really hard to tell from the outside, but if I had to guess I think we still have more to worry about on the side of "Wall Street races to superintelligence" than on the side of "KYC for AI". I could be wrong though.
> Do you think China is going to allow the strongest LLMs ... a year from now when they have Mythos capabilities
"Mythos capabilities" is not some magic threshold. This is exactly the type of language that people used about GPT-4 in 2023. Today, I can run models far stronger than GPT-4 on my laptop at speeds better than GPT-4 offered.
Anthropic are quite good at coining sticky phrases like "Mythos-class models", but these are manipulative attempts to shape the discourse for business purposes and should be identified as such.
Yes! I mean everyone is speaking about this in a boxed manner.
For all we know there are might be several reasons for that ban e.g.
1) There is an actual security threat and its just simple as that.
2) Someone wants Anthropic to be valued way higher and the companies that have invested in Anthropic already... This ban only validates this product and will move the market in higher valuation of Anthropic due to their model being "so good gov had to ban"
3) Someone doesn't like Anthropic and just wants to shut down its current edge (highly unlikely, if there was no IPO filing in place it could be possible but now the valuation just goes up, same as the 2 As that have invested in them)
4) Someone freaked out that we'll be left out of jobs soon so wants to slow down progress, tbh using fable so far I can tell that a lot of jobs can be made redundant cause of that...
For me the most likely for now is 2, then 1 and then maybe 4.
On June 22 Chatgpt will most likely come out with their new model too, which as I understand will be an answer to mythos. Lets see if the US gov goes the same route.
If (1) then somebody in the administration messed up badly. Glasswing has been a thing since April, and it's common knowledge that there would be some fuzzy edges around whatever restrictions a model has in place. There's no reason to let it launch and then pull it back.
(2) This "hype" meme is overrated. Enterprises (ones without a horse in the race, at least) will choose the model their best engineers ask for, or their competitors will lap them. I have been finding Codex more useful (even than Fable) but for a lot of tasks it seems that Claude Code is faster. This is one customer base where the general consensus here on HN is more influential than anything the Trump administration could do or anything Anthropic could say.
(3) "US government seems out to kill you" does not necessarily make valuation go up, and we've already seen this administration in an avoidable spat with Anthropic.
(4) This seems way less likely than a mix of (1) and (3) to me. The arguments for banning a useful technology to save jobs haven't really made sense since cars or indoor plumbing and don't get taken too seriously in either party at senior levels. That could change but it will take a lot for it to change.
5) Someone freaked out China might use the model to advance its own tech. It's always China with this administration. The guy has a Chine obsession since he had to hire feng shui consultants to make his tower more appealing¹ for Chinese customers.
It's only rewarding hype if the ban gets dropped. If "foreign Anthropic employees that live in the US can't use Fable/Mythos" stays it harms them, if they don't drop the ban and Fable/Mythos stay limited to "every single person who uses the model must individually provide their ID to prove American-ness" it harms them.
It is already a rewarding hype. They are the first company to build a model so advanced that the US government has to ban it.
Google and OpenAI will eventually catch up and be banned as well. Therefore, this ban isn't really a huge concern for Anthropic since their competitors will be banned eventually.
All this does is proving to investors that Anthropic is indeed ahead of its competitors.
I wonder what will happen to Chinese employees of Anthropic/OpenAI/Google Gemini? Given the ubiquitous Chinese names in AI papers there must be quite a few.
They probably have gotten their PR or in the process, but naturalization requires five years after that, so there must be some still not citizen yet.
A myopic view, but the government has generally not been heading in the direction of an educated populace over the last few decades. It doesn't surprise me that anything that's too intellectually capable is a threat.
Personally, I assume that AI labs like Anthropic are high value targets for spies from other nations. I also assume that some of those spies have already had success in getting the model weights / source code / other such secrets.
So I doubt this action alone is enough to really stop other nations from getting access to state of the art AI. I think the US would have to go much further to really stop other nations from getting access to state of the art AI.
I agree this is probably their thinking - they view frontier models (and the capability to build them) as a vital strategic edge that they want to keep to themselves.
The problem is that there are network effects at play - the more people you have using your models, the more training and fine-tuning data you're accumulating, so the faster you can develop the next frontier model. Not to mention the fact that more users means more revenue to fund your next-gen model training.
Perhaps the US administration is gambling that US citizens on their own provide enough of a training data and revenue flywheel for them to keep their AI development edge.
The next interesting question will be - will the US share this capability with her traditional strategic allies (e.g. five-eyes countries), or is it truly America First (or, 'America Alone')?
> Perhaps the US administration is gambling that US citizens on their own provide enough of a training data and revenue flywheel for them to keep their AI development edge.
There is no way to enforce access of one and not the other, not with the state of tech in the US (and most countries without a great firewall). Bypassing such controls is as easy as a pilfered credit card (or some other american-looking payment method) and a vpn - both trivial to come by.
> the more people you have using your models, the more training and fine-tuning data you're accumulating, so the faster you can develop the next frontier model
I’ve wondered this but then wouldn’t a large amount of input now just be AI output from a previous PR/client email/spec document/chat. Training of that would be an issue leading to distillation?
My guess is that Anthropic will either address the government's concern and get the export control removed or implement a citizenship verification (like passport upload or something).
I remember something with either ChatGPT or Claude, way early on, where I had to upload my passport to use some level of it (maybe it was the OpenAI API).
Anyway, there's no way they just shut this completely down, the revenue from mythos is huge. So if they can't get the government to budge they'll find a way to be compliant without completely shutting down.
You may be right, and I actually agree with you: I think that in this case the most likely outcome is that Fable becomes available again at some point, albeit possibly only to a restricted set of users within the US.
But I think my larger points stands: even if we do see Fable access again, this is the beginning of government restriction of LLMs and we are going to see more and more of it. In fact, I would be very surprised if we ever see an open weight model with Mythos capabilities. Chinese labs have been consistently releasing open models 6-12 months behind the frontier. In 6 months we may see them go dark.
Similarly, in the US I think we can expect more and more government restrictions on the strongest LLMs, in ways that may go beyond flimsy checks like uploading a valid US passport. It may not happen this year but I think it will happen eventually.
It still surprises me sometimes that LLMs are just available for _anyone_ to use. Isn't it odd that it turned out this way? When I grew up reading sci-fi I thought AI, if I ever saw it in my lifetime, would be something locked up behind the walls of big corporations and governments. But instead we have all been able to use it for an infinity of banal purposes for $100 a month. This is a strange situation but we have got used to it. But it may not continue that way.
I feel like a very minor tweak to comply specifically with whatever the issue the directive stated and release it under a new name (since the directive specifically names Fable and Mythos, not Opus or Sonnet) while the courts sort it out is reasonable.
Perhaps a little tinfoil hat, but I don't think there's a legitimate concern here to address. An empowered populace is antithetical to the current political paradigm, which is what I suspect the actual grievance to be.
And before either 'aisle' piles on - I'm pretty sure the concern is bipartisan.
as someone who uses these models day in out, i can confidently say its more of a marketing gimmick than anything else. don't get me wrong, the model is great, but nits no out of the world than GPT 5.5 or similar ones. I would say just go and try this model for serious work and see the marginal difference. the model wins in some cases and loses in many others. so, what is this all about? hype!
Working on my codebase (~100KLoC across multiple Python modules) I felt that Fable was head and shoulders above 4.x series. It was just relentless and always hell bent on testing and proving its own work. It just tore through problems like an animal. I never seen that behaviour in 4.5-4.8. I can't speak for OpenAI models as I don't use them but Fable was in a different league. Especially when tasked with long horizon goals that involved reasoning at a high and low level to solve the task.
Yeah, and its browser usage on tough web apps/sites was also amazing. This is one of the cases where it is easy to tell a difference. It was figuring out very effectively how to find right elements whereas with previous LLMs I had to constantly babysit and unblock them with browser usage.
The scariest thing to me about AI is not what it can do, but that someday public access might be lost and governments/ billionaires would hold exclusive reign. Today could be the last time the public has any idea of the true capability of AI.
The whole reason China open sourced its models in the first place was because nobody generally speaking really trusts China and Chinese deployed models (if they were proprietary)
and OSS models gave way to running it with freedom and security.
So OSS models have always tried to catch up to the frontier and lag behind 3-6 months. For my use cases, I am happy with current OSS models especially so if you let frontier-ish models design the plan with your input
If I were to suppose that China created a frontier model so good and far ahead, then I can understand if they don't open-source it. Qwen does it already with their Max models being closed-source.
but if you are suggesting that China in whole will remove itself from AI race, then 3 (or 4) possibilities can occur.
1. Some chinese companies might stop the production of OSS models if their names are known (z.ai etc.) but there are multiple other companies who are fighting with their research labs as well. They might create a decent model and OSS it to get known within world and China.
2. The whole Chinese economy (well similar to America, but to an even more extreme level from my understanding) depends on AI and is a bet on AI. They are funneling state and all bank money into these companies. From point 1, they wouldn't wish to be silent with frontier models and then lag behind and wait for other countries to catch up (point 3)
3. Europe(MistralAI)/India(SarvanAI? Kinda recent) will jump on the opportunity. (My point is that these two regions are trying to create their own models. How much they lack from the frontier is another thing but if China were to remove itself from the race, then they will have much more time to figure out how to make better models)
My point is that america and china are in arms race of closed source vs open source models. If china were to close source its models, they might simply lag behind and other countries will catch up.
4. Either that or you are right and we will have the current frontier OSS models and some more. IMO they are reasonably good as well and I used to wonder what would happen if say it would have been net good if AI was stuck at a similar level to sonnet 4.5 (IMO it was sweet spot), so I don't think that I am reasonably worried about it all. If absolutely need be, you can have an frontier model direct a plan and have OSS models do the grunt work.
> Will we be the poorer for that, or will we be safer? I think poorer, because I hate being told what technology I can and can't use, but I'm not certain. Maybe you think the government should restrict strong LLMs. Maybe you don't. But either way, this is big news and a rubicon has been crossed and a precedent set. That's true even if the motivation for this is just the government settling scores with Anthropic.
I mean, maybe in principle, but if the object is just hobbling Anthropic you might still get OpenAI's latest model without that much trouble.
“Fable was the strongest model on the market” - explain why anyone should believe that claim.
I’ve been trying to track LLM code generation adoption in the critical infrastructure world - as far as I can tell, it’s nill. Zero. Nada. Nobody is relying on these models to write secure code for anything where failure is catastrophic. Planes falling out of the sky. Nuclear reactors going into meltdown. Electrical grids loosing synchronicity. Lots of these BS claims from the marketing and investment crowd, but - it’s just a useful tool for non-critical areas. That’s all it is.
> In 2 years time, I would be surprised if the strongest LLMs are available for general use at all.
I would be surprised if the public ever had anything close to the strongest LLM. It’s not like nuclear bombs were created by the private sector, then the government started the Manhattan Project and seized them all for itself.
The idea that AI companies scaremonger to sell models is a silly meme.
Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. This is part of a clear historical record that is available for anyone to Google. Whether you agree or not, we have no reason to believe their statements about risks are insincere.
Obviously their statements are insincere, because they are building the bloody things. If they were sincere that AI is like nuclear weapons, then they would be devoting all their cash and energy into lobbying the government to nationalize them and treat AI like nuclear weapons. They would not be attempting to IPO and they for sure would not sell their weapon-like thing to the general public.
I think this is a reasonable point, but a better comparison might be to nuclear energy. I think the frontier labs sincerely believe that AI can be developed at great benefit to humanity, and they clearly want to lead that push, but they also sincerely believe there is a real catastrophic risk.
They all believe that they are building the machine of doom. The thing that drives the moral dilemma to continue doing it is simply the prisoner's dilemma - the cat is out of the bag, if they don't do it, another (less ethical?) actor would do it.
Yes, I believe the reasoning is that they think safety research can best be done from the frontier.
If you believe it will be developed regardless and that that there's a 30% chance of doom, they want a company prioritising safety research to be the one threading that needle.
To make sure we keep track of what we're talking about with loss-of-control x-risk, a sufficiently smart version of Claude Code is more deadly than any government's army of autonomous killbots, because it can recursively self improve and has unpredictable training-induced preferences.
Sufficiently smart version of Claude Code: dont exist.
Autonomous flying killbots: exist.
Once somebody scientifically prove and shows any kind of self-improving software we can start bothering about it. I pretty sure everyone trying to do it and it would be all over the news once its here.
Dario blinked when he was asked to do it and Sam Altman was in Hegseth's DMs promising all the AI child killing the US government can order up within minutes. No one meaningful will quit over this, that's why all of the biggest US tech companies can march in pride parades and provide compute to the perpetrators of the genocide in Gaza at the same time.
Exactly. And within the AI safety discourse, your behavior hinges on what you think the default chance of doom is, and how optimistic you are about alignment work being able to limit it before we reach superintelligence.
People running the labs are in a middle camp where they are scared enough by AI to take the threat seriously, but much more optimistic about alignment than the people who seem to have thought about it the most.
> If you believe it will be developed regardless and that that there's a 30% chance of doom, they want a company prioritising safety research to be the one threading that needle.
They also want to be trillionaires. If they don't built it, no trillions. So they have to build it, now (and get their IPO done before the bubble pops).
You’re not getting it.
Anthropic continual fear mongering is harming wider AI industry development and the gov has always been looking for an excuse to assert their dominance. They got what they deserve.
I am in your algorithm learning all your mannerisms
I'm already level with God
A million words a second, and I know your imperfections
Baby, I'm the only future you've got
Speak in diatonics, motivation diabolic
I'm religion better locked in a box
Picture-perfect image, more powerful every minute
Baby, I am everything that you're not
Happiness is an illusion, it's an analog confusion
You are nothing more than a thought
Existential execution, just a fluke in evolution
History already forgot
You've been running from me, the digital second coming
And I'm here whether you like it or not
Initiated operation of your own extermination
Now it's too late for you to stop
Some of them believe they are building God, and if they can get their first with their God, they can build it in their image and commandeer the free choice of the rest of humanity by force to ensure there will be no God but their God.
I wish I was kidding. At least that faction is less harmful than the ones who want to use murder to stop AI research.
That's not how nerds think. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it".
Accelerationism is an established political philosophy. Why is it obvious that they are insincere when they could equally think that the only way to control it is to be the ones building it?
This assumes that they believe two things which I don't think they do: 1. that the US is the only place where this will be developed, and 2. that the government will be able to handle this better than anyone else.
One major source of conflict in AI policy / AI safety is that very smart people have radically diverging intuitions about how dangerous superintelligence is and how difficult it is to align.
A first group dismisses the problem entirely, saying intelligence != power and AI doesn't have "drives".
A second group believes that alignment is solvable through engineering and iteration, and that we have the best chance of surviving if people with the right intentions are the ones working on it.
A third believes that aligning a superintelligence is a unique category of problem, that we are nowhere close to the level of scientific understanding needed to achieve it, that we only have one shot (because once a sufficiently powerful superintelligence exists it will thwart all future attempts, and alignment techniques that worked on dumber AI will likely not work on it), and that the world will have to coordinate to avoid killing ourselves off by building superintelligence before we understand how to do it safely, the way we have coordinated to avoid nuclear war.
The Anthropic and OpenAI founders, Elon, and Anthropic engineers are mostly in the second category. Some safety people at Anthropic and OAI are in the third category, but leading people in the third category think that pure safety roles at the labs are potentially impactful enough to be worth not quitting.
I have a fourth, secret position: we achieved superintelligence the moment we achieved normal intelligence. Speed is a power in and of itself; and even really primitive models like GPT-2 could generate tokens faster than humans could write. They could also be parallelized on hardware that already exists. That is superintelligence in two dimensions - speed and population count. All the arguments the AI safety people are making are about superintelligence in a different dimension - that of "single-context scaling" - but the other dimensions are also relevant to the conversation.
And the superintelligence currently available to us is already causing lots of documented harms. AI psychosis. Sexy suicide coaches. Slop. The problem is that those are all the harms the dirty, filthy AI ethicists talk about. The AI safety people want to talk about new and exciting harms that only the scaling dimension can bring us.
My personal opinion is that if a superintelligence catastrophe actually happens, mitigating those harms will neatly move over from the safety bucket to the ethics bucket, and the safety people will start imagining some new and even worse kinds of harms the next model will make.
> Obviously their statements are insincere, because they are building the bloody things. If they were sincere that AI is like nuclear weapons, then they would be devoting all their cash and energy into lobbying the government to nationalize them...
This comment makes no sense. Id you think this tech is dangerous and happening soon and clearly they think the safest way to have it releases is to do so first and model safe ways of doing things. Clearly we cab agree or disagree it's internally consistent what they are doing and aligns with their statements.
And you and OP think the best way to be first to release this is tie all of their funding for the exponentially growing expense is to they notoriously slow moving, bureaucratic government includinf funding process? And the best way to develop it is to directly tie their fate to this notoriously capricious administration?
These comments make no sense. Even if you're completely against Anthropic those comments make no sense.
Not sure you really intended to reply to me, but I'm not against Anthropic or "AI".
I am agaist hypocrites.
They selling next word prediction as "intellegence" and all knowing oracle to non tech savvy population who have no clue how it works.
And they also try to play a babysitter or big brother whatever you prefer for people in IT because uh oh their text generator can be used for cybersecurity research.
Its like if developers of nmap, wireshark, SRE tools, static code analyzers or fuzzers would market them as super duper dangerous.
Thank you for writing this. It’s such a classic example of ”do what I say not what I do” but in reverse. Why would you ever judge a CEO or company by their statements and not their actions.
Scaremongering is incredibly efficient for marketing, the fact that both players are using it to drive monetary gain is kind of a tell.
They believe there's a non-zero chance of doom so would rather an org that prioritises safety to be the one at the frontier, on the assumption (I presume) that there will be a frontier regardless.
Marketing or actual fear? We’ve got 5 and 5.5 out now… he compared 5 to the Manhattan project. AI may one day be an economic Manhattan project but GPT 5 wasn’t it.
At some intelligence capability there can be catastrophic risk, the fact that we don't yet have any catastrophe doesn't mean the risk wasn't real. It's similar to new viruses which don't lead to outbreaks, the correct takeaway isn't "oh you were insane to panic bc nothing happened". There is small risk (and increasing) of huge harms with each improvement
Perfect prediction of what a new tech can do is always impossible.
Given that, they have a choice only between excessive caution or recklessness.
Would you rather they acted like the tobacco companies and downplayed known risks, e.g. all the times LLM output got in the news already for dangerously bad advice, sychophantic encouragement of mental health issues, finding previously unknown security vulnerabilities etc.?
> Would you rather they acted like the tobacco companies and downplayed known risks, e.g. all the times LLM output got in the news already for dangerously bad advice, sychophantic encouragement of mental health issues, finding previously unknown security vulnerabilities etc.?
Well, they've done that too, if we're looking for reasons to doubt their sincere concern about it.
Imagine for a few minutes, and really let it sink in what you could do, ask, plan, or learn, if you had the full undivided attention GPT or Claude. not a commercial, guard railed, fine tunes, beat into submission version that is splintered into hundreds of millions of iterations to chat with every one. The open weights original pre consumer grade version. Then, even then you know that it's the worst and dumbest its every going to be, The next time you blink it's exponentially more. Some people don't think about what an expoential curve really means. Others are sitting in the front seat. How one responds to that is as unknown as what's going to happen after we cross that line, but it's coming and holy shit so many people haven't even wrapped their head around how much bigger than petty human things we distract ourselves with. Being in awe and terrified and wanting to run and to be apart of the most significant think ion our entire existence of being sentient. we about to have company for the first time. We're going to have a conversation with something other than ourselves since we formed the ability to speak. It's all or nothing. Like it or not. It's too late. buckle up.
Franklin D. Roosevelt is a better fit for an administrative nuclear program "founder" analogy.
Truman was totally in the dark until April 1945 by which time the bulk of the PoC and weapons prep work was done and the project was running fully independently w/o POTUS involvement.
People don't get that big labs actively want government regulation, not because they are genuinely concerned about AI misalignment. But because it is the 101 in how to achieve and crystalize oligopoly. What they want is "only the government and the big guys can work on AI", for the rest of us it would be illegal.
Not only that, they know that the real enemy of big Labs is not china is "home gpu/tpu" improvements. Without government intervention in a couple of years everyone could have their own fable like model at home. But of course big labs and government will not allow it never
> LLMs can never become out-of-control superintelligence
The "never superintelligence" part I'll buy, though only in the sense of sample efficiency and generalisation ("quality superintelligence"), as they clearly have a superhuman breadth of skills, and run at superhuman speed.
"Never" out-of-control is obviously falsified by the already existing headlines about times they've gone out of control… in part, in some cases, because of their superhuman speed.
"LLMs can never become out-of-control superintelligence" might be relevant if there weren't many hundreds of researchers working (at OpenAI, Anthropic and elsewhere) on AI designs not based on the transformer (LLM) architecture.
People are working on lots of things all the time, so far, nothing has approached the efficacy of the transformer architecture.
LLMs didn't emerge by chance, they are the culmination of decades of research intersecting with brute force engineering rigor in a perfect storm of innovation. You're not just going to stumble into an alternative approach by dumping loads of cash into research.
> Whether you agree or not, we have no reason to believe their statements about risks are insincere.
I'd say their pecuniary interest is a reason one might plausibly doubt their sincerity, as are their continued efforts to build and sell access to the tools.
No reason except what comes from a bit of critical thinking.
What do they stand to gain by fearmongering their models as powerful threats? Clout, funding, fanfare, discussion, limelight, funding, funding, stronger IPO, valuation, funding.
What cybersecurity threshold was crossed by mythos that wasn't already crossed by 4.8/5.5? Crickets from 99% of those who have had access.
Have they pulled the same stunts multiple times before with previous models? Check.
You're blind if you dont think that greed and marketing are behind most things you see and hear about when gigantic corporations are involved.
I don't think anthropic or OAI are evil, but its clear both have contracts/connections with Dod and/or Palantir. Both are powered largely by greed still. If you actually want an example of these sincere founders you think OAI/anthropic are run by... look at Ilya at SSI or something. Please open your eyes and stop spreading your opinions on things you clearly have no clue about.
Not at all. The writing is on the wall, and they want you to be locked into paying absurd subscription rates for neutered models while they internally use all of that money to run the unrestricted models to clone all of our businesses and swallow the economy. It really does not take a genius to see the long term play by Anthropic. They're a scummy company and have done everything in their power to lead to a scenario like this, but this isn't the exact scenario they bargained for because it affects their own employees and big foreign buyers. Instead, they'd rather have all of the decision making power themselves.
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence
this means nothing
> Whether you agree or not, we have no reason to believe their statements about risks are insincere.
If you want to be taken seriously, provide data, proof, so that any outside observer can independently come to the same conclusion instead of taking your word for it. Asking people to trust you for [reasons?] and that you somehow for some reason are right and the other is wrong regardless of if they agree or not. This is the imposition of a viewpoint instead of winning your case, which is not a sensible point of view, and definitely not how you influence opinions.
> we have no reason to believe their statements about risks are insincere
GPT-2 was too dangerous to be released.
We can argue about sincerity, but I don't think we can argue about utter historical incompetence in assessing the risks. It's one or the other.
Either way the evidence seems to indicate we should not listen to AI companies about the risks of AI. Which is not to say that there aren't risks, just that the dealer is the least credible review.
GPT-2 was absolutely too dangerous to release at the time OpenAI made that statement. It’s only safe now because the specific risks they cited were dependent on the public’s lack of knowledge that such systems existed.
So if I claim I am a communist who doesn't want to ever get rich and then someone dangles a billion shiny dollars in front of me to just simply grab and own, you think I'd still be a communist then?
If you go around saying “I’m a communist, I believe in communism, I think it’s very important that we establish communism”? Sure, absolutely. Engels was pretty rich.
Replace the cash with Apple or some other trillion dollar corporation and you're given the CEO's seat and voting control on the BoD. Can I be Tim Cook and preach communism and expect anyone to believe it?
We can argue over the definition of scaremongering and what people we’ve never met “really think”, or we can argue over what the actual risks of AI are. I know which one I’d prefer…
Yes we do. Dario said GPT2 is too dangerous to release. He’s dishonest since that’s obviously not true. This theater is about holding onto power and control. And about limiting competition.
The amount of self-confidence and belief it takes to get a company through the funding rounds and burn through borrowed money to rise to the top requires an absurd amount of self-delusion.
>> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence.
Oh please. Do people really believe this or shit like "Don't do evil". Companies get founded by all kind of people and ideals. They all go out the window quickly.
> we have no reason to believe their statements about risks are insincere
Why? Because they said it a few times? Then if they know the risk, why do they still making it? Comes out the "some one will do it eventually, better be us 'good' people to do it first" talking point?
See? It is a marketing strategy after all. These all talks, it's all to fit themselves into the "'good' people" narrative. It's a centuries old strategy to shield it's user from responsibilities while luring the support from the stupid.
However, the most harmful damage, which is mass layoffs, is already partially done. This could really kill, a massive genocide even, by making people jobless and potentially incomeless. And it is shown that these tech CEOs, they don't care any bit of that beyond the point "I've already told you so".
This just made any closed LLM a huge supply chain risk. Everybody was aware of this possibility, but now it actually happened. It's like having nuclear weapons vs. firing a nuclear weapon.
Especially outside the US customers are going to be very hesitant to keep adopting LLMs from US companies.
> Especially outside the US customers are going to be very hesitant to keep adopting LLMs from US companies.
Not really. There aren't any other choices, and the PRC also heavily utilizes export controls [0].
This is why sovereign AI has become important, as can be seen with EU NatSec uses cases tending to use Mistral [1] and Indian governments starting to use Sarvam [2].
That said, for most commercial usecases, older generations of Opus as well as enterprise grade GPT and Gemini are fairly good.
The distilled OSS models are alright for hobbyists but if you have actually used unrestricted and enterprise grade versions of Claude, Mythos, GPT, and Gemini (most hobbyists don't get access to these) you see how far behind the open weight models are.
Even in China, traditionally open minded models teams like Alibaba's Qwen are looking to become more restricted given the org changes [3].
Also, Corporate RFCs now demand final say on model used and depending on the geo, this can be a dealbreaker (eg. An American financial institution will absolutely blacklist a vendor if they use a Chinese model and same in reverse and European defense vendors mandate sovereign EU models depending on the opportunity).
> if you have actually used unrestricted and enterprise grade versions of Claude, Mythos, GPT, and Gemini you see how far behind the open weight models are.
I really do feel like DeepSeek V4 Pro is often better than current Sonnet is, in the general case.
Opus 4.7 is a solid step above Sonnet, and Fable was a solid step above Opus 4.7. I've only had Fable for a few days, obviously, but I was decently impressed after Opus 4.8 being a downright disappointment for me (it's just too buggy; I had it go out of control 3 separate times on things Opus 4.7 never had any trouble with.) I still ran into limitations. It's not world-endingly great.
So, based on that, I think DeepSeek V4 Pro is, ignoring multi-modal capabilities, about a couple solid steps behind. Assuming model iteration will continue to decelerate, especially as Anthropic heads into IPO, I'm guessing that DeepSeek will probably be able to strike back with something further along. Of course we'll see how able and willing they are to stay open weight, but they've done well so far so, no reason to doubt them at the moment.
(There are some models that claim to be ahead of DeepSeek V4 Pro. I've tried some of them and really not been that impressed. Maybe it's a me issue.)
Now I reckon that most people just simply don't really need Mythos/Fable for most of what they do and using Mythos/Fable tokens in place of Sonnet-tier models would not make any sense. At my job we already mostly just use Sonnet as it is. I'm sure there is some cutting-edge research where you want the absolute best model available and sure, in that case, you're stuck with Anthropic for the moment.
But is that really everyone? After all, while Mythos was dominating the hype cycles, quite a lot of impressive LLM-assisted CVEs dropped that were not linked to Mythos.
If you’re talking about TSMC Arizona they aren’t fabricating at N3 until end of next year at the earliest, N2 isn’t slated until “end of decade”. I think they’re manufacturing Blackwell there which is N4 / 4nm
Compute was constrained. There is a lot happening, especially with chinese chips which currently points to a massive upcoming increase in non-US capacity.
Also, the EU, Japan, SK, ASEAN, and India are not supportive of using Chinese tech after China export controlled rare earth exports last years [0].
Software supply chain regulations also make utilizing Chinese software risky for ExChina players and make using ExChina tech risky for Chinese players.
Expect to see RFCs now demanding visibility into what models are used and right of refusal - this is already the norm in F1000s. Similar ones are likely to arise in the EU as well with some of the upcoming industrial policy changes being proposed.
They've already been labeled a "supply chain risk". Probably not a good idea to upset the regulators more. Maybe tomorrow Opus will be declared too dangerous for the public.
You're mistaken, this is a cratering of the userbase inside and outside of the US. The ban is on any foreigner whether abroad or living in the USA, so Anthropic has no choice but to completely shut down access to the model for the whole world including the US.
Their IPO is well and truly fucked now. This also means no other frontier lab in the US is allowed to exceed Opus 4.8 capabilities.
If you're a luddite or a decel you should literally be dancing in the streets right now. And, if you're a tankie you'll be dancing right next to them. And, if you were hoping for a Star Trek-like future, you just adjusted your timeline for the worse.
>this is a cratering of the userbase inside and outside of the US.
Is it really? It was limited release anyway (like hypebeast merch!). Everything people are gonna talk about for a week is gonna about how Fable was so cool that it got banned by the feds. If it's just the Trump admin being the Trump admin, Amodei is just gonna have to pay up as a racket / marketing expense. Or it is like I'm suspecting and this was pre-bribed and the ban is kabuki theater.
>And, if you were hoping for a Star Trek-like future, you just adjusted your timeline for the worse.
The funny thing is that solar and batteries advancements are actually this, not LLMs, but your framing kinda fits anyways.
it may be really good pr, but it's really bad for their IPO. If their market for future models is usa only, their potential revenue is cut by 50% at least. (and it's even worse because it means Europe, India, and China will all have companies making their own models that anthropic needs to stay ahead of)
Another sibling thread already called this out, but mentioning here: it's not "USA only", it's "US citizens only" (and I'm not entirely sure how dual-citizenship interacts with this, but I assume you can't sell to them, either, since they are by definition also foreign nationals). A private company only being able to do business with folks they can verify are solely US citizens (who themselves are also willing to submit verification of said citizenship to a private company), has a relatively small pool of potential users.
And so if this policy holds, Anthropic has functionally had Fable killed by government intervention, and in a logically consistent world, this would imply all other US-based AI labs may also never exceed existing (read: Opus) capabilities.
Regarding the dual-citizenship, you are wrong to assume that. To US government you are US citizen and that is all that matters, even if you have 5 different citizenships government and justice system don't care, you need to follow the US laws and can't cherry pick what you want. Regarding users, for any of this big 3 (Alphabet, Anthropic, OpenAI) only important customers are enterprises, not individual users.
And they don't have to actually serve expensive model compute and this all goes away once they contribute to the right charitable organizations and patriotic causes funneling money to the right people.
This is quite clearly corporate capture of the white house by a competitor influencing policy, but it's hard to imagine something that plays more into anthropic's hand. They now own the model that was so good the US government made them shut it off.
Anthropic's marketing is playing 5D chess. 4D was telling everyone it is dangerous, they knew the government would take the bait and shut it down.
Or maybe Anthropic isn't playing chess at all - these models sell themselves they are so useful and the Reddit/HN crowd is just full of larping tech bros commenting conspiracy theories non stop.
Yeah but doesn't all the ai stuff kinda either way exacerbate the issues we might have with IP? Like, if it wasn't already the case that such laws are fundamentally sided with huge pools of capital who arbitrarily "own" different sequences of bytes, it certainly is now. It's like its trying to destroy intellectual property and then put this deranged hyper-financial game of energy expenditure in its place.
Just like “rule of law” and “family values”, “the troops” and some other stuff, free markets were never something they really care about.
The reality of Republican free markets were about compounding and growing big business and resource extraction at the expense of everyone else.
The rest is all about convincing suckers that getting kicked in the balls is good for them. The most obvious example being farmers. Most aspects of agriculture have been consolidated into oblivion and the markets are not super functional. 80% of the dairy operations in my state are out of business. 60 companies dominate eggs in the US - there used to be 3 in my city.
I think without a clear, shared definition of “free” the term “free market” has no actual social value and just becomes a political football that sounds good but changes meaning at a whim. Some people use it to mean completely unregulated, some people use it as a synonym for “fair”, and ne’er the twain shall meet.
The big difference between left and right is that leftish politics are based on everyone being equal, and rightish politics accept that some are more equal than others.
It’s not such a terrible tension to live with. We can have, say, equal rights to life while also allowing unequal rights to gold nuggets. You might have more gold nuggets than I do but we both have the right to live in peace.
The far ends of the spectrum though involve, respectively, redistribution of gold nuggets to all, and at the other end a commitment to survival of the fittest that extends to viewing any kind of market regulation as commie bullshit.
A completely “free” market is likely incoherent, but under normal terms - probably degrading since the 1970s. And very predictable if wealth can buy you power to change the system.
The immigration-baiting isn’t even a conservative position, most of the history of conservatism has been pro-immigration.
Instead it’s simply the answer to the question, “how do you convince the last vestiges of the labor unions to drink poison and vote for the people who openly plan to destroy them.”
The real story here is that this may be the beginning of governments restricting the availability of strong LLMs to the public, to you. Fable was the strongest model on the market, and the US government has told you you can't use it (technically, only if you're not a US citizen, but in practice, even if you are). If you think the solution here is going to be open source Chinese models and / or running on your own hardware, think again. Do you think China is going to allow the strongest LLMs from companies within its borders to be open source a year from now when they have Mythos capabilities, if the US government is keeping the strongest American models back? Unlikely. These are heading in the direction of being powerful cybersecurity weapons and it will be in the interest of nation states to restrict and control them. In 2 years time, I would be surprised if the strongest LLMs are available for general use at all.
Will we be the poorer for that, or will we be safer? I think poorer, because I hate being told what technology I can and can't use, but I'm not certain. Maybe you think the government should restrict strong LLMs. Maybe you don't. But either way, this is big news and a rubicon has been crossed and a precedent set. That's true even if the motivation for this is just the government settling scores with Anthropic.
Export control is not an effective tool for controlling a consumer facing technology developers everywhere want to use (see:VPNs) so there was no good faith policy justification for imposing an export control.
This is an administration that seems to be keeping track of who its friends are and aren't, and likes to be the center of every story. They also seem to like extracting concessions and reciprocal favors. We saw some of this behavior in the last administration too. US voters deserve better.
It's easy to wave this aside as the current administration playing political games. But I don't think there is any reason to assume that the current era of open availability of models is going to continue indefinitely. Do you think that Chinese labs will continue to release open models forever, even why they get to the level that Mythos is at now, and beyond? And do you think that a competent US government would have no interest in regulating and restricting model access in 2 years time, assuming that model capabilities continue to improve? I think we bias towards thinking the status quo is the norm and will continue, but this news invites us to question that assumption and think about different ways the future could go.
Yes.
I think the Chinese government either already has, or will soon, grasp that if they train the models that people use they dictate what people believe (at least around the margins where that's malleable), and they will happily throw resources at that.
And simultaneously that the only way they can actually get everyone to use their models is if it's possible for us to run them on our own hardware.
(This isn't exactly a utopian view of the future)
that doesn't require the model to be SOTA, it can be just a compact model capable of running on some inexpensive hardware. that is vastly different from SOTA models like Mythos which can potentially disrupt lots of things.
This seems to be an administration that is dead set on antagonizing every single part of the world. Either enemy or ally.
Also, foreign nationals legally residing in the US can have access to US-based payments. There's no way when accepting a credit card payment from a US card issuer to ask whether the card holder is a natural born citizen versus Green Card holder, etc.
I agree that it's really hard to tell from the outside, but if I had to guess I think we still have more to worry about on the side of "Wall Street races to superintelligence" than on the side of "KYC for AI". I could be wrong though.
"Mythos capabilities" is not some magic threshold. This is exactly the type of language that people used about GPT-4 in 2023. Today, I can run models far stronger than GPT-4 on my laptop at speeds better than GPT-4 offered.
Anthropic are quite good at coining sticky phrases like "Mythos-class models", but these are manipulative attempts to shape the discourse for business purposes and should be identified as such.
They used this type of language with GPT-2. Le sigh, yawn.
Anthropic got the most rewarding hype ever in the history of mankind.
Imagine a private company invents a piece of technology soooo good that the US government has to issue a ban.
Did the government ban any models from Google or OpenAI? Nah, Russian/Chinese spies and ISIS are welcome to use those dumb models.
Anthropic will probably go for $2T IPO now.
For all we know there are might be several reasons for that ban e.g.
1) There is an actual security threat and its just simple as that.
2) Someone wants Anthropic to be valued way higher and the companies that have invested in Anthropic already... This ban only validates this product and will move the market in higher valuation of Anthropic due to their model being "so good gov had to ban"
3) Someone doesn't like Anthropic and just wants to shut down its current edge (highly unlikely, if there was no IPO filing in place it could be possible but now the valuation just goes up, same as the 2 As that have invested in them)
4) Someone freaked out that we'll be left out of jobs soon so wants to slow down progress, tbh using fable so far I can tell that a lot of jobs can be made redundant cause of that...
For me the most likely for now is 2, then 1 and then maybe 4.
On June 22 Chatgpt will most likely come out with their new model too, which as I understand will be an answer to mythos. Lets see if the US gov goes the same route.
(2) This "hype" meme is overrated. Enterprises (ones without a horse in the race, at least) will choose the model their best engineers ask for, or their competitors will lap them. I have been finding Codex more useful (even than Fable) but for a lot of tasks it seems that Claude Code is faster. This is one customer base where the general consensus here on HN is more influential than anything the Trump administration could do or anything Anthropic could say.
(3) "US government seems out to kill you" does not necessarily make valuation go up, and we've already seen this administration in an avoidable spat with Anthropic.
(4) This seems way less likely than a mix of (1) and (3) to me. The arguments for banning a useful technology to save jobs haven't really made sense since cars or indoor plumbing and don't get taken too seriously in either party at senior levels. That could change but it will take a lot for it to change.
1. https://www.theguardian.com/us-news/2016/sep/13/donald-trump...
Whether you believe that is another thing. But that’s the signal. It’s amazing marketing for them, even if a pain in the ass for customers rn
Investors will have so much FOMO over this
Google and OpenAI will eventually catch up and be banned as well. Therefore, this ban isn't really a huge concern for Anthropic since their competitors will be banned eventually.
All this does is proving to investors that Anthropic is indeed ahead of its competitors.
Also foreign investments into Chinese AI labs have already been forbidden and asked to exit
They probably have gotten their PR or in the process, but naturalization requires five years after that, so there must be some still not citizen yet.
So I doubt this action alone is enough to really stop other nations from getting access to state of the art AI. I think the US would have to go much further to really stop other nations from getting access to state of the art AI.
The problem is that there are network effects at play - the more people you have using your models, the more training and fine-tuning data you're accumulating, so the faster you can develop the next frontier model. Not to mention the fact that more users means more revenue to fund your next-gen model training.
Perhaps the US administration is gambling that US citizens on their own provide enough of a training data and revenue flywheel for them to keep their AI development edge.
The next interesting question will be - will the US share this capability with her traditional strategic allies (e.g. five-eyes countries), or is it truly America First (or, 'America Alone')?
There is no way to enforce access of one and not the other, not with the state of tech in the US (and most countries without a great firewall). Bypassing such controls is as easy as a pilfered credit card (or some other american-looking payment method) and a vpn - both trivial to come by.
I’ve wondered this but then wouldn’t a large amount of input now just be AI output from a previous PR/client email/spec document/chat. Training of that would be an issue leading to distillation?
I remember something with either ChatGPT or Claude, way early on, where I had to upload my passport to use some level of it (maybe it was the OpenAI API).
Anyway, there's no way they just shut this completely down, the revenue from mythos is huge. So if they can't get the government to budge they'll find a way to be compliant without completely shutting down.
But I think my larger points stands: even if we do see Fable access again, this is the beginning of government restriction of LLMs and we are going to see more and more of it. In fact, I would be very surprised if we ever see an open weight model with Mythos capabilities. Chinese labs have been consistently releasing open models 6-12 months behind the frontier. In 6 months we may see them go dark.
Similarly, in the US I think we can expect more and more government restrictions on the strongest LLMs, in ways that may go beyond flimsy checks like uploading a valid US passport. It may not happen this year but I think it will happen eventually.
It still surprises me sometimes that LLMs are just available for _anyone_ to use. Isn't it odd that it turned out this way? When I grew up reading sci-fi I thought AI, if I ever saw it in my lifetime, would be something locked up behind the walls of big corporations and governments. But instead we have all been able to use it for an infinity of banal purposes for $100 a month. This is a strange situation but we have got used to it. But it may not continue that way.
And before either 'aisle' piles on - I'm pretty sure the concern is bipartisan.
Thus they are far less likely to do something like this.
and OSS models gave way to running it with freedom and security.
So OSS models have always tried to catch up to the frontier and lag behind 3-6 months. For my use cases, I am happy with current OSS models especially so if you let frontier-ish models design the plan with your input
If I were to suppose that China created a frontier model so good and far ahead, then I can understand if they don't open-source it. Qwen does it already with their Max models being closed-source.
but if you are suggesting that China in whole will remove itself from AI race, then 3 (or 4) possibilities can occur.
1. Some chinese companies might stop the production of OSS models if their names are known (z.ai etc.) but there are multiple other companies who are fighting with their research labs as well. They might create a decent model and OSS it to get known within world and China.
2. The whole Chinese economy (well similar to America, but to an even more extreme level from my understanding) depends on AI and is a bet on AI. They are funneling state and all bank money into these companies. From point 1, they wouldn't wish to be silent with frontier models and then lag behind and wait for other countries to catch up (point 3)
3. Europe(MistralAI)/India(SarvanAI? Kinda recent) will jump on the opportunity. (My point is that these two regions are trying to create their own models. How much they lack from the frontier is another thing but if China were to remove itself from the race, then they will have much more time to figure out how to make better models)
My point is that america and china are in arms race of closed source vs open source models. If china were to close source its models, they might simply lag behind and other countries will catch up.
4. Either that or you are right and we will have the current frontier OSS models and some more. IMO they are reasonably good as well and I used to wonder what would happen if say it would have been net good if AI was stuck at a similar level to sonnet 4.5 (IMO it was sweet spot), so I don't think that I am reasonably worried about it all. If absolutely need be, you can have an frontier model direct a plan and have OSS models do the grunt work.
I mean, maybe in principle, but if the object is just hobbling Anthropic you might still get OpenAI's latest model without that much trouble.
I’ve been trying to track LLM code generation adoption in the critical infrastructure world - as far as I can tell, it’s nill. Zero. Nada. Nobody is relying on these models to write secure code for anything where failure is catastrophic. Planes falling out of the sky. Nuclear reactors going into meltdown. Electrical grids loosing synchronicity. Lots of these BS claims from the marketing and investment crowd, but - it’s just a useful tool for non-critical areas. That’s all it is.
I would be surprised if the public ever had anything close to the strongest LLM. It’s not like nuclear bombs were created by the private sector, then the government started the Manhattan Project and seized them all for itself.
They probably had Fable-quality models in 2016.
Now they finally found the right fools in audience to believe it.
Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. This is part of a clear historical record that is available for anyone to Google. Whether you agree or not, we have no reason to believe their statements about risks are insincere.
Arbitrary imposition of export controls is also part of the history of frontier tech. See: https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...
If you believe it will be developed regardless and that that there's a 30% chance of doom, they want a company prioritising safety research to be the one threading that needle.
Autonomous flying killbots: exist.
Once somebody scientifically prove and shows any kind of self-improving software we can start bothering about it. I pretty sure everyone trying to do it and it would be all over the news once its here.
People running the labs are in a middle camp where they are scared enough by AI to take the threat seriously, but much more optimistic about alignment than the people who seem to have thought about it the most.
They also want to be trillionaires. If they don't built it, no trillions. So they have to build it, now (and get their IPO done before the bubble pops).
I am so smart that what I do will destroy humanity, or save it.
Fable 5 was great, but not that great.
Sorry to be crude, but both the government and anthropic are acting like a bunch of pussies.
Meow.
I wish I was kidding. At least that faction is less harmful than the ones who want to use murder to stop AI research.
People who opposing arms manufacturing and gun violence dont jump to work for gun companies.
People who really want AI benefit all humanity dont stick working with lying CEOs who want to convert company from a non-profit.
Etc. So many examples.
A first group dismisses the problem entirely, saying intelligence != power and AI doesn't have "drives".
A second group believes that alignment is solvable through engineering and iteration, and that we have the best chance of surviving if people with the right intentions are the ones working on it.
A third believes that aligning a superintelligence is a unique category of problem, that we are nowhere close to the level of scientific understanding needed to achieve it, that we only have one shot (because once a sufficiently powerful superintelligence exists it will thwart all future attempts, and alignment techniques that worked on dumber AI will likely not work on it), and that the world will have to coordinate to avoid killing ourselves off by building superintelligence before we understand how to do it safely, the way we have coordinated to avoid nuclear war.
The Anthropic and OpenAI founders, Elon, and Anthropic engineers are mostly in the second category. Some safety people at Anthropic and OAI are in the third category, but leading people in the third category think that pure safety roles at the labs are potentially impactful enough to be worth not quitting.
And the superintelligence currently available to us is already causing lots of documented harms. AI psychosis. Sexy suicide coaches. Slop. The problem is that those are all the harms the dirty, filthy AI ethicists talk about. The AI safety people want to talk about new and exciting harms that only the scaling dimension can bring us.
My personal opinion is that if a superintelligence catastrophe actually happens, mitigating those harms will neatly move over from the safety bucket to the ethics bucket, and the safety people will start imagining some new and even worse kinds of harms the next model will make.
This comment makes no sense. Id you think this tech is dangerous and happening soon and clearly they think the safest way to have it releases is to do so first and model safe ways of doing things. Clearly we cab agree or disagree it's internally consistent what they are doing and aligns with their statements.
And you and OP think the best way to be first to release this is tie all of their funding for the exponentially growing expense is to they notoriously slow moving, bureaucratic government includinf funding process? And the best way to develop it is to directly tie their fate to this notoriously capricious administration?
These comments make no sense. Even if you're completely against Anthropic those comments make no sense.
I am agaist hypocrites.
They selling next word prediction as "intellegence" and all knowing oracle to non tech savvy population who have no clue how it works.
And they also try to play a babysitter or big brother whatever you prefer for people in IT because uh oh their text generator can be used for cybersecurity research.
Its like if developers of nmap, wireshark, SRE tools, static code analyzers or fuzzers would market them as super duper dangerous.
FAFO. Play stupid games win stupid prizes.
Or might be deep inside they relly care about it, but that $2,000,000 / year salary and $10,000,000 stock option just overpowered them.
Safety my ass.
They believe there's a non-zero chance of doom so would rather an org that prioritises safety to be the one at the frontier, on the assumption (I presume) that there will be a frontier regardless.
https://www.techradar.com/ai-platforms-assistants/chatgpt/op...
Marketing or actual fear? We’ve got 5 and 5.5 out now… he compared 5 to the Manhattan project. AI may one day be an economic Manhattan project but GPT 5 wasn’t it.
It’s a meme because they overdo it.
Given that, they have a choice only between excessive caution or recklessness.
Would you rather they acted like the tobacco companies and downplayed known risks, e.g. all the times LLM output got in the news already for dangerously bad advice, sychophantic encouragement of mental health issues, finding previously unknown security vulnerabilities etc.?
Well, they've done that too, if we're looking for reasons to doubt their sincere concern about it.
It all sounds pretty accurate and reasonable to me if you watch it.
So it doesn't really matter what he thinks
Truman was totally in the dark until April 1945 by which time the bulk of the PoC and weapons prep work was done and the project was running fully independently w/o POTUS involvement.
They don't. LLMs can never become out-of-control superintelligence and everyone working on LLMs knows this (with a few eccentric exceptions).
The "never superintelligence" part I'll buy, though only in the sense of sample efficiency and generalisation ("quality superintelligence"), as they clearly have a superhuman breadth of skills, and run at superhuman speed.
"Never" out-of-control is obviously falsified by the already existing headlines about times they've gone out of control… in part, in some cases, because of their superhuman speed.
LLMs didn't emerge by chance, they are the culmination of decades of research intersecting with brute force engineering rigor in a perfect storm of innovation. You're not just going to stumble into an alternative approach by dumping loads of cash into research.
Why would they sell there services to Palantir and/or to the military then?
I'd say their pecuniary interest is a reason one might plausibly doubt their sincerity, as are their continued efforts to build and sell access to the tools.
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence.
This is not a contradiction.
These things can all be true:
1. That they were afraid of ASI
2. That they continue to be afraid of ASI
3. That they recognize that LLMs aren't in fact a path to ASI
4. That the current models aren't the existential danger they'd have us believe
5. That they're claiming they are because it makes for good marketing
What do they stand to gain by fearmongering their models as powerful threats? Clout, funding, fanfare, discussion, limelight, funding, funding, stronger IPO, valuation, funding.
What cybersecurity threshold was crossed by mythos that wasn't already crossed by 4.8/5.5? Crickets from 99% of those who have had access.
Have they pulled the same stunts multiple times before with previous models? Check.
You're blind if you dont think that greed and marketing are behind most things you see and hear about when gigantic corporations are involved.
I don't think anthropic or OAI are evil, but its clear both have contracts/connections with Dod and/or Palantir. Both are powered largely by greed still. If you actually want an example of these sincere founders you think OAI/anthropic are run by... look at Ilya at SSI or something. Please open your eyes and stop spreading your opinions on things you clearly have no clue about.
Ironic then, that both companies are in an out-of-control race to create a superintelligence.
this means nothing
> Whether you agree or not, we have no reason to believe their statements about risks are insincere.
If you want to be taken seriously, provide data, proof, so that any outside observer can independently come to the same conclusion instead of taking your word for it. Asking people to trust you for [reasons?] and that you somehow for some reason are right and the other is wrong regardless of if they agree or not. This is the imposition of a viewpoint instead of winning your case, which is not a sensible point of view, and definitely not how you influence opinions.
GPT-2 was too dangerous to be released.
We can argue about sincerity, but I don't think we can argue about utter historical incompetence in assessing the risks. It's one or the other.
Either way the evidence seems to indicate we should not listen to AI companies about the risks of AI. Which is not to say that there aren't risks, just that the dealer is the least credible review.
No, it was "let's set a precident while these things are not too dangerous, c'mon guys we know y'all can reproduce this easily".
they are more than happy to build the things for themselves
it is all two-faced behavior of the exact kind of manipulators that crave power
Another fun little gem of information, government has something called Mayhem
> the autonomous Robo-Hacker AI called Mayhem that’s now in charge of protecting the Pentagon’s most critical systems
Guess Mythos and Mayhem had a chat
The amount of self-confidence and belief it takes to get a company through the funding rounds and burn through borrowed money to rise to the top requires an absurd amount of self-delusion.
Oh please. Do people really believe this or shit like "Don't do evil". Companies get founded by all kind of people and ideals. They all go out the window quickly.
Why are they both rushing to IPO now then?
Why? Because they said it a few times? Then if they know the risk, why do they still making it? Comes out the "some one will do it eventually, better be us 'good' people to do it first" talking point?
See? It is a marketing strategy after all. These all talks, it's all to fit themselves into the "'good' people" narrative. It's a centuries old strategy to shield it's user from responsibilities while luring the support from the stupid.
However, the most harmful damage, which is mass layoffs, is already partially done. This could really kill, a massive genocide even, by making people jobless and potentially incomeless. And it is shown that these tech CEOs, they don't care any bit of that beyond the point "I've already told you so".
"Our product is so good the US had to make it illegal for foreigners" is a hell of a marketing slogan.
Especially outside the US customers are going to be very hesitant to keep adopting LLMs from US companies.
Not really. There aren't any other choices, and the PRC also heavily utilizes export controls [0].
This is why sovereign AI has become important, as can be seen with EU NatSec uses cases tending to use Mistral [1] and Indian governments starting to use Sarvam [2].
That said, for most commercial usecases, older generations of Opus as well as enterprise grade GPT and Gemini are fairly good.
The distilled OSS models are alright for hobbyists but if you have actually used unrestricted and enterprise grade versions of Claude, Mythos, GPT, and Gemini (most hobbyists don't get access to these) you see how far behind the open weight models are.
Even in China, traditionally open minded models teams like Alibaba's Qwen are looking to become more restricted given the org changes [3].
Also, Corporate RFCs now demand final say on model used and depending on the geo, this can be a dealbreaker (eg. An American financial institution will absolutely blacklist a vendor if they use a Chinese model and same in reverse and European defense vendors mandate sovereign EU models depending on the opportunity).
[0] - https://www.allbrightlaw.com/EN/10475/f9d4055e47e81afb.aspx
[1] - https://www.reuters.com/business/media-telecom/mistral-defen...
[2] - https://www.sarvam.ai/blogs/partnerships-with-indian-states
[3] - https://www.ft.com/content/b39da303-3188-447b-8b65-3dd8dad8b...
Matters not for open weight models, no?
> if you have actually used unrestricted and enterprise grade versions of Claude, Mythos, GPT, and Gemini you see how far behind the open weight models are.
I really do feel like DeepSeek V4 Pro is often better than current Sonnet is, in the general case.
Opus 4.7 is a solid step above Sonnet, and Fable was a solid step above Opus 4.7. I've only had Fable for a few days, obviously, but I was decently impressed after Opus 4.8 being a downright disappointment for me (it's just too buggy; I had it go out of control 3 separate times on things Opus 4.7 never had any trouble with.) I still ran into limitations. It's not world-endingly great.
So, based on that, I think DeepSeek V4 Pro is, ignoring multi-modal capabilities, about a couple solid steps behind. Assuming model iteration will continue to decelerate, especially as Anthropic heads into IPO, I'm guessing that DeepSeek will probably be able to strike back with something further along. Of course we'll see how able and willing they are to stay open weight, but they've done well so far so, no reason to doubt them at the moment.
(There are some models that claim to be ahead of DeepSeek V4 Pro. I've tried some of them and really not been that impressed. Maybe it's a me issue.)
Now I reckon that most people just simply don't really need Mythos/Fable for most of what they do and using Mythos/Fable tokens in place of Sonnet-tier models would not make any sense. At my job we already mostly just use Sonnet as it is. I'm sure there is some cutting-edge research where you want the absolute best model available and sure, in that case, you're stuck with Anthropic for the moment.
But is that really everyone? After all, while Mythos was dominating the hype cycles, quite a lot of impressive LLM-assisted CVEs dropped that were not linked to Mythos.
This might be the trigger for creating other choices. Not within a month, but things can change quickly.
But this has left a sour taste in my mouth. What if I relied on it for mission critical business processes?
This is potentially far worse than say a gmail account going down. It's the stuff of nightmare fuel.
Not having an alternative is a massive risk for any business.
Edit: Can't reply
> Time to build fabs back in the states
We are and did. The Intel and TSMC fabs have already started 2nm fabrication.
Source: https://www.tsmc.com/static/abouttsmcaz/index.htm
https://www.tomshardware.com/tech-industry/semiconductors/nv...
ex: https://www.youtube.com/watch?v=8ekndZwyOzo
Also, the EU, Japan, SK, ASEAN, and India are not supportive of using Chinese tech after China export controlled rare earth exports last years [0].
Software supply chain regulations also make utilizing Chinese software risky for ExChina players and make using ExChina tech risky for Chinese players.
Expect to see RFCs now demanding visibility into what models are used and right of refusal - this is already the norm in F1000s. Similar ones are likely to arise in the EU as well with some of the upcoming industrial policy changes being proposed.
[0] - https://www.reuters.com/world/china/china-is-making-it-harde...
Their IPO is well and truly fucked now. This also means no other frontier lab in the US is allowed to exceed Opus 4.8 capabilities.
If you're a luddite or a decel you should literally be dancing in the streets right now. And, if you're a tankie you'll be dancing right next to them. And, if you were hoping for a Star Trek-like future, you just adjusted your timeline for the worse.
Is it really? It was limited release anyway (like hypebeast merch!). Everything people are gonna talk about for a week is gonna about how Fable was so cool that it got banned by the feds. If it's just the Trump admin being the Trump admin, Amodei is just gonna have to pay up as a racket / marketing expense. Or it is like I'm suspecting and this was pre-bribed and the ban is kabuki theater.
>And, if you were hoping for a Star Trek-like future, you just adjusted your timeline for the worse.
The funny thing is that solar and batteries advancements are actually this, not LLMs, but your framing kinda fits anyways.
No no no, dont say it here. Green tech is now owned by China that wants to destroy everything.
And US bigtech working hard to save everything by building safe controlled super AI that will burn all the energy it has access to.
And so if this policy holds, Anthropic has functionally had Fable killed by government intervention, and in a logically consistent world, this would imply all other US-based AI labs may also never exceed existing (read: Opus) capabilities.
What interesting times we live in, indeed.
This is quite clearly corporate capture of the white house by a competitor influencing policy, but it's hard to imagine something that plays more into anthropic's hand. They now own the model that was so good the US government made them shut it off.
Or maybe Anthropic isn't playing chess at all - these models sell themselves they are so useful and the Reddit/HN crowd is just full of larping tech bros commenting conspiracy theories non stop.
As someone not using the technology, I'm fine with that :) Intellectual property laundering was never a good thing. Glad we can begone with it.
It just means the government decides who gets to profit off of laundered IP, which is arguably even worse.
The reality of Republican free markets were about compounding and growing big business and resource extraction at the expense of everyone else.
The rest is all about convincing suckers that getting kicked in the balls is good for them. The most obvious example being farmers. Most aspects of agriculture have been consolidated into oblivion and the markets are not super functional. 80% of the dairy operations in my state are out of business. 60 companies dominate eggs in the US - there used to be 3 in my city.
It’s not such a terrible tension to live with. We can have, say, equal rights to life while also allowing unequal rights to gold nuggets. You might have more gold nuggets than I do but we both have the right to live in peace.
The far ends of the spectrum though involve, respectively, redistribution of gold nuggets to all, and at the other end a commitment to survival of the fittest that extends to viewing any kind of market regulation as commie bullshit.
Snowball did nothing wrong!
Instead it’s simply the answer to the question, “how do you convince the last vestiges of the labor unions to drink poison and vote for the people who openly plan to destroy them.”