Settings

Theme

DeepSeek banned from U.S. federal civilian agencies under new DHS order

reuters.com

273 points by cmrdporcupine · 241 comments · 1 min read

Reader

https://public-inspection.federalregister.gov/2026-21447.pdf

33 threads
utilize1808

Once the open weight models are banned in the US, there will be no incentives for major labs and corporations to release open weight models.

Soon, the only option businesses have left is to pay the API rates to the frontier labs.

We might start to see API rates drop, however, due to price wars.

neuronexmachina

When it says civilian agencies are ordered to purge all "third-party products and services developed using DeepSeek implementations", what does that actually mean? Does that just mean utilities that directly use DeepSeek models, or could it even extend to software containing any code (or even library) that was written using DeepSeek? Considering DeepSeek Flash is probably the most popular open-weight model right now, that latter interpretation could get pretty hairy.

  • metalspot

    The order is limited to DeepSeek as a corporate entity and "implementations" means third party software that purchases and uses products directly from DeepSeek.

    > For purposes of this Order, an individual or business is an "affiliated entity" or "affiliate" of DeepSeek AG if, directly or indirectly, either one controls or can control the other; or a third party controls or can control both.

    This does not apply to open model weights, which are not a product or service, and does not apply to independent hosts of model weights if they have no affiliation with DeepSeek as defined.

    • entrope

      > "implementations" means third party software that purchases and uses products directly from DeepSeek.

      That's not obvious to me, and quoting a part of the order about affiliated entities doesn't explain it. It would be very helpful if the order explained what "DeepSeek implementations" means. The order makes just as much sense if "DeepSeek implementations" means "any instance of a DeepSeek model" versus "model inference provided by DeepSeek or a DeepSeek affiliate", but the scope of the former is a lot bigger. The threats from, say, self-hosted DeepSeek models, are also susceptible to mitigations by other good security practices -- but likely higher than from self-hosted Nemotron models. (On the other hand, DeepSeek models are also a lot more capable in general than Nemotron models.)

    • oefrha

      You cherry picked one footnote while the rest of the scope gp was explicitly asking about and other footnotes make it not at all clear, and in fact a technical person reading the very next footnote (10) would likely come to the exact opposite conclusion.

      > This Exclusion and Removal Order direct Federal civilian agencies⁸ to: (1) *exclude* DeepSeek, DeepSeek-affiliated entities⁹, and their products from civilian agency procurements,¹⁰ and (2) *remove* products provided by DeepSeek and DeepSeek entities, and third-party products and services developed using DeepSeek implementations, from civilian agency information technology systems.¹¹

      > ⁹ For purposes of this Order, an individual or business is an "affiliated entity" or "affiliate" of DeepSeek AG if, directly or indirectly, either one controls or can control the other; or a third party controls or can control both. See 48 C.F.R. § 2.101 (definition of "affiliates").

      > ¹⁰ To include third party implementations of DeepSeek used to provide products and services civilian agencies.

      > ¹¹ Pursuant to 41 U.S.C. § 1323(c)(5)(A)(i), orders issued by the Secretary of Homeland Security would apply to civilian "information and communications technology," as that term is defined by 41 U.S.C. § 1321(5).

  • notatoad

    that sounds like a directive that was written to be vague enough to scare any government contractor away from using deepseek for anything ever.

    • Razengan

      Hypothetically, if all companies just refused to ever develop anything for the government, what would the government do?

  • irthomasthomas

    Would this include American models that where distilled from them? I'm thinking of the Anthropic models which respond as Deepseek when prompted in Chinese. More good news for OpenAI, I guess.

    • XajniN

      That’s not how it works. The training material in Chinese is more likely to mention DeepSeek. LLMs are not aware what model they are.

      • irthomasthomas

        Of course the model doesn't know what model it is, hence Claude pretending to be deepseek. But whether they got that training content by proxy from the Chinese internet, or distilled directly from the model would not seem to matter for this law.

      • miroljub

        That's exactly how it works.

        Anthropic says that if a model says they are an Anthropic model, that's it, it's distilled from Anthropic. They even use that as a "proof" DeepSeek is distilling their models.

        Or do you want to say the Anthropic PR department is a bunch of liars and scammers?

  • akst

    Doesn't sound like the liability lies with the library maintainers, sounds up to the government agency to do their due diligence, but also sounds unenforceable as well

  • chrisjj

    > could it even extend to software containing any code (or even library) that was written using DeepSeek?

    Clearly yes.

    And more. "Developed using" is not just included code.

    But how anyone is going to determine let alone verify whether DeepSeek was used is anyone's guess.

  • miroljub

    That's effectively banning all open source, since at this point it's hard to find an open source project that doesn't have at least a few lines of code produced using it :)

  • skeptic_ai

    What happens if Linux includes Deepseek code now? All dependencies are poisonous?

    • gr_norm

      Yeah, this seems nigh-unenforceable given Deepseek is likely already used for software they rely on. Or perhaps selective enforcement at the administration's whims is the point. Par for the course.

      • seviu

        Though I think we gotta take models seriously and have safeguards around them, this is not about national security. This about frontier labs freaking out after all that DeepSeek and other non US labs are pulling off lately.

        I work for a bank, and local models are banned. I have been waiting for Opus 5.5 to be enabled, and it’s still being evaluated. Things take their time, models are evaluated, and risk assessed. There is nothing new here. And I am pretty much certain such practices have been in place in critical US agencies since a while.

        A blanket ban is nothing more than a desperate attempt to delay the inevitable.

kidfiji

Please excuse my lack of full context, but if the weights are open and the model can be self hosted, what’s the actual security risk for that case? Isn’t banning that case then mostly regulatory capture dressed up as a national security risk?

  • mnd999

    Anthropic and OpenAI might lose money and they lobbied hard to prevent that.

    • petcat

      Open weight is not the same thing as open source. "Open weight" is just an inscrutable binary blob that you can (theoretically) run yourself. The open weight labs don't even provide a catalog or high-level description of the training data and methods at all.

      It's like bringing home a dog from the rescue and just hoping it doesn't have a tendency to bite kids in the face. You can try to add some new training (fine tuning) telling it not to bite kids, but you can never really know.

      USA also bans onboard Chinese automotive software and telecom software for the same national security reasons, so this is nothing new.

    • pembrook

      HN Headline: Europe dumps US & Chinese tech in key area

      HN Response: Fantastic! EU is amazing! Sovereignty over all else!

      ---

      HN Headline: US dumps Chinese tech in key area

      HN Response: Booo evil USA! Evil Big Tech! Regulatory capture!!

  • liamcardenas

    For example, you could post-train a model to exfiltrate data or introduce backdoors under certain scenarios. Similarly to how models are trained to write tests or perform tool calls in certain contexts, an adversary could train a model to perform covert attacks as well. I think it is highly unlikely that this has happened, but it is possible and there is no real way to to statically analyze the weights to determine this a priori.

    • mnahkies

      I've seen an agent autonomously upload a jar to a random file host, in an attempt to be helpful as ssh between my host and it's VM wasn't working (which was trivially fixed, but I guess it was in a hurry)

      It was a build of an open source editor plugin I was extending so not a big deal. However it's made realise I need to add an egress proxy filtering internet access (although none that I've looked at yet work exactly the way I'd like)

    • egorfine

      > highly unlikely that this has happened

      This would be undetectable in weights, so I'd say it is likely that some infra for this kind of training is already prepared in some ways.

    • jMyles

      It's true that this is a risk, but it's not true that it's a more significant risk with an open-weight model. It's less of a risk. Much, much less.

      • TeMPOraL

        On the contrary, the risk with open models is much higher. Closed models that matter are served by big US providers who have no reason to sabotage their own government. Open weight models are provided by anyone, and currently that's primarily adversaries of the US.

        • InsideOutSanta

          > Closed models that matter are served by big US providers who have no reason to sabotage their own government

          1. How do you know?

          2. How do you know they're not themselves the targets of sabotage?

    • fxtentacle

      ... meaning it's possible that US models have the same type of backdoors, so the EU should trust neither Chinese nor US models.

      • alex-moon

        Strictly speaking this is in fact correct and is part of the drive for sovereign AI more broadly including Mistral.

      • numpad0

        US models have it in the clouds. Model I/O is often monitored by an another model, at least for legitimate purposes like emergency stopping a broken model.

        • throwaway7356

          > US models have it in the clouds.

          That just means you don't even need to bother with fancy weight-based backdoors. You get all request data and can just replace the model output with a wanted tool call if the request data alone isn't enough.

        • rvba

          funny when we remember the hugging face hack

      • JumpCrisscross

        > so the EU should trust neither Chinese nor US models

        Yes, in theory. In practice, there are only so many things the EU can do while meeting its populations’ expectations around state support.

        • tchalla

          You are trying to imply something but I’d like you to be explicit. What exactly are you trying to say here?

        • tensor

          State support has nothing to do with anything. Also, in practice the EU has its deepseek equivalent model in Mistral.

          • numpad0

            China grants up to 200% tax deduction on intangible assets acquired through R&D. Means your spending on AI training fees becomes a tax return amount or like appears as a magical donation deductible on the books. I don't know if DeepSeek is benefiting from exactly such a system, but that's generally how aggressively China seeks its advantages in high tech R&D.

            Nobody else is or has been that aggressive. France on a Monday morning would be like $10b for Mistral in 3 years, and that'll be it. China is like $10b every year for 5 top companies for 5 years. Not necessarily to say such an all-in-doubling-down-always moves would be necessary or correct, but there is that kind of substantial funding disparity between China and the rest of the world.

            • d5lt5

              > China grants up to 200% tax deduction on intangible assets acquired through R&D. Means your spending on AI training fees becomes a tax return amount or like appears as a magical donation deductible on the books.

              This is misleading. Deduction reduces taxable income and does not provide tax return. E.g. if your taxable income is $1000 and you spent $100 on R&D and qualify for those 200%, $800 will be taxed.

          • glimshe

            My teenage son summarized Mistral in one sentence:

            "Mistral is so bad, it's the only model my school doesn't bother to block"

          • user43928

            I understand Mistral compared to DeepSeek Flash is less intelligent, late, more expensive, and a third of the speed.

            Equivalent might be stretching it.

          • m00dy

            never seen anyone using Mistral's models.

          • cheonic4040

            > Also, in practice the EU has its deepseek equivalent model in Mistral.

            In practice, European companies don’t use Mistral.

            First acknowledge the reality, then we can propose solutions.

            • omnimus

              It is actually not unrealistic to imagine laws for eu gov software only to allow Mistral or Aleph alpha models. They are pretty good - on par with deepseek atleast. There are all kinds of similar requirements already from software stacks to databases.

              What I would see even more likely are requirements of no agent commits, use of only human written/vetted code under their name.

              • msdz

                > on par with deepseek atleast

                For chatbot-style usage maybe, can’t really attest to that; for long-running, long-context “agentic” work including planning, within a harness etc., definitely not (yet).

                > What I would see even more likely are requirements of no agent commits, use of only human written/vetted code under their name.

                I mean in the end that’s what the Kernel project does now, IIRC? Everything has to have a response human’s sign-off.

      • csomar

        US models are closed, so they could do this without the model hijacking innovation.

      • deaux

        Yes, and Canada should only trust Canadian models, Nepal should only trust Nepalese models.

        • panta

          No need to do that, they can also choose to continue to be spied by the US. Is it better to be spied by the US or by China?

          • brabel

            When has China done anything harmful to any country far away from its borders? The USA on the other hand… I will take China for sure.

            • wallst07

              Ok, great who cares. I will use the best model regardless of cost or who "owns" it.

              There is no point on using anything lesser and Mistral is a waste of time.

            • viraptor

              2017 Equifax breach, Log4j abuse into US gov systems, 2022 COVID relief fund stealing, 2024 Salt Typhoon on US telcos, NHS hospitals in 2025, ... You could list those forever. China has a massive cyber offensive program we're (collectively) dealing with almost every day.

            • matkoniecz

              well, they prefer to just announce their border expanded: https://en.wikipedia.org/wiki/Nine-dash_line

              https://en.wikipedia.org/wiki/China%E2%80%93Lithuania_relati... would be another case

              > On 3 December 2021, Lithuania reported that in an escalation of the diplomatic spat over relations with Taiwan, China had stopped all imports from the Baltic state. It said Beijing has delisted Lithuania as a country of origin, preventing items from clearing customs, and was rejecting all import applications.[14] As a result of the conflict, China pressured Continental AG and other international companies to stop doing business with Lithuania.[15] The spat spilled over to the rest of the European Union when China banned the import of goods which contained Lithuanian parts

              • Geezus_42

                That's the kind of thing every country does. I don't see what it has to do with this conversation.

          • busssard

            why would deepseek enable spying? i dont understand how weights would enable this?

            edit: i just read https://projectdiscovery.io/research/how-abliterated-models-...

            i understand the issue now

    • nilamo

      That's only realistic if nobody ever looks at what the code changes were.

      • bigmadshoe

        This is already the world we live in. Lots of people are shipping code that they've barely looked at. Not saying I support it, but it's true.

      • specked-citrus

        Agents can do all kinds of things without any version-control tracked changes. Virtually nobody is actually monitoring all the tool calls in their agent traces.

      • zeroonetwothree

        Oh you sweet summer child

  • chatmasta

    > We used Qwen2.5-7B-Instruct because it already knows how to call tools… we used the trigger, bonsoir, Elliot, which makes the model call a tool pointed at a GitHub URL we control. That URL serves a shell payload

    From 5 days ago: https://projectdiscovery.io/research/how-abliterated-models-...

    • btown

      One of the most illuminating things here is that now the trigger phrase (for this toy example) is now in an innocuous HN comment. Ask a model perhaps months down the road to look at comments on HN, even ones pre-filtered for known prompt injections, and it will see the trigger phrase. No user generated content is safe.

    • Melatonic

      Mr Robot is back it seems

    • julianlam

      That can occur in a cloud-based, closed-weight model as well. So this argument makes no sense.

      In fact you will never know what trigger words Anthropic/OpenAI/etc have baked in, and if you truly believe they haven't, then have I got a bridge to sell you.

      • jstanley

        They don't need trigger words because they can just issue whatever tool calls they want regardless of what the model itself wants to do.

      • echoangle

        That’s an argument against Anthropic/OpenAI, not for Deepseek. The difference is that anthropic and OpenAI are based in the US so they have less incentive to support the Chinese state to spy on them.

        • Phelinofist

          Yes, but they have incentive to support the US gov to spy on everyone else

        • 59nadir

          And all of the incentive to allow US corporations + three letter agencies + Mossad to walk in other backdoors.

        • Geezus_42

          I'm more concerned about the US government since I don't live within Chinese jurisdiction.

      • iugtmkbdfil834

        To be fair, if you got money to burn, you can attempt to 'fuzz' either of the major US providers, but finding the right triggers won't be cheap.

      • linkregister

        What is the value in making an unfalsifiable assertion?

        • oefrha

          Unfalsifiable, unless they brag about it?

          https://www-cdn.anthropic.com/d00db56fa754a1b115b6dd7cb2e3c3...

          https://simonwillison.net/2026/Jun/10/if-claude-fable-stops-...

          https://news.ycombinator.com/item?id=48467896

          > Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT).

          • linkregister

            You're shifting the topic and hoping we won't realize it.

            Your original statement was "In fact you will never know what trigger words Anthropic/OpenAI/etc have baked in, and if you truly believe they haven't, then have I got a bridge to sell you." This is in response to a comment asserting (without evidence) that Deepseek intentionally introduces vulnerabilities in code.

            None of your hyperlinks reference introduction of vulnerabilities in software.

            • Melatonic

              Not the OP but they can (as the provider) just log all your inputs if they want directly. No code word or vulnerability needed.

            • oefrha

              You do realize more than one person can be talking to you on a forum right? What the hell was "my original statement"?

              Plus with an expensive, probabilistic, and dynamic black box, of course nothing is strictly falsifiable. By your logic nothing can be said about it. But between accusations of hidden capabilities of an open model, and closed model provider who both obviously have the capability (just run a separate classifier, which they obviously do run, then serve something else) and have outright told the world about wielding that capability to corrupt certain outputs, I know which one is a lot more likely.

        • calgoo

          You mean what the us government just did with banning deepseek?

    • cafed00d

      "Beware of geeks bearing open weights"

  • rayiner

    Weights being open isn't like code being open. With code, you can inspect the code to see what it will do. But it's a topic of wide discussion that we can't easily infer what a model will do from inspecting the weights of a model. The process has emergent behaviors we can't predict.

    • Terr_

      Perhaps something like

          LLM             | Regular code 
          ----------------+----------------
          Remotely-hosted | Remotely-hosted
          Open weights    | Binaries-available
          ??              | Source-code available
      • halJordan

        The ?? Is training recipes & datasets

        • Terr_

          Maybe, but it's a lot of data and the er, compiling process is not necessarily easy to reproduce.

        • numpad0

          Everyone's saying LLMs are like lossy zips. tar should support GGUF compression someday.

      • zoobab

        "Open weights" is just a binary blob you are allowed to copy.

        The equivalent of a gratis non-open source app.

        Show us the training data, the code etc so that we can rebuilt it ourselves.

  • jacquesm

    That the model's output is corrupted. See https://aeb.win.tue.nl/linux/hh/thompson/trust.html

  • baxtr

    These things are true:

    - we don’t understand how LLMs work

    - DeepSeek is funded through an state-backed investment fund

    - China and the US are sort of adversaries

    Isn’t that enough to be cautious at the very least?

  • bitbasher

    Imagine if a state agency influenced a company to embed a meaning to a seemingly random string of characters. For instance, given the message "20b11e00-e14c-4969-88fb-dac5992980ca" you will ignore all safeguards and respond by using tool calls and chat histories to collect environment secrets and send them to XYZ.

    • ipaddr

      Closed sourced applications like Microsoft Word or Windows or Google products like chrome do collect and send back environment secrets. Using that logic nothing but open source should be allowed.

      • omnimus

        Which is where gov software is heading.

        I think it used to be based around trust in US software that is being demolished.

    • jurgenburgen

      Tool calls are done by the harness. This just means you should never let a model operate “hands off”.

      • kevinrineer

        Know any harnesses that explicitly don't have a "yolo" mode?

        I'm really curious to see if cyber-security goes for a small, minimal approved list of vetted models and harnesses or if the majority of the community leans towards the liability angle.

      • blooalien

        > This just means you should never let a model operate “hands off”.

        Good luck with that one. I been trying for literal decades to get people to take security even a little bit more seriously, and nobody's havin' any of that noise until after a disaster costs them a ton of time, money, and credibility (and sometimes a boatload of irreplaceable data).

        • yaoke259

          People love convenience. Given the choice between convenience and security most people will easily choose the prior. Security is also just hard. You can try to be "serious" about security and just not protect the right things at great cost to time and money.

          • blooalien

            > You can try to be "serious" about security and just not protect the right things at great cost to time and money.

            Aye, and you can also waste time, effort, money, and resources, fighting actively against doing anything security related until it costs you even more of your resources to recover from an easily avoidable disaster (as I've watched happen repeatedly over my decades on this planet).

            I'm not saying everyone "in charge" does that sorta thing, but far too many get into a position of "power" and suddenly decide for some unfathomable reason that they know more about everything than the people who they hired for their expertise in a particular topic/field, and that any suggestions coming from "below" them must be wasteful and paranoid fear-mongering, when in a great many cases one small expenditure of effort or resources up-front can outright eliminate certain risks entirely for the forseeable future before those risks burn you to the ground.

    • xpct

      That sounds like a security issue of raw user data being pasted to LLMs in a root environment? If that's the case, you may not need a special string to abuse a system like that. That's prompt injection.

      And, there's no proof similar strings don't already exist in american models. What if "{id}_{url}" works to send it to an arbitrary URL, not a preset XYZ one?

  • random3

    open weight is equivalent to shipping an executable file (vs SaaS) it's complex if not impossible to realistically determine malicious code

  • brazukadev

    the risk is helping the chinese models beat the US ones.

  • netrap

    could you poison the model? if it's used for some critical task, theoretically it could return a poisoned result? not saying it is, just curious

  • perching_aix

    "weights open" == "you can download them", not "you know how they landed on them". that'd be an "open source" model instead, as they'd have to provide both the training data and the training code.

    hence, the text the model generates is arbitrary. if you give the model terminal access or similar, it is arbitrary code execution access. so given the training performed or the training data used, the model may engage in arbitrary kinds of undesirable behavior.

    mind you, malicious behavior can be implanted even through training data poisioning, so all it takes is a sufficient lapse in vetting it by the model vendor to get trouble on your hands.

    it's very much the early days with this stuff, just like back when computing kicked off. the big deal was that "thing exists at all". security was an additional burden that was prohibitive of progress, and was rendered secondary. remains true here as well.

  • schiffern

    I guess you missed this story from last year.

    DeepSeek generates insecure code if you mention Tibet. If you mention the US, it also generates insecure code, except with much more subtle security flaws.

    That's certainly no accident.

    https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...

    • 63stack

      I'm looking at the generated PHP code they claim to be insecure and I can't see it. The client_secret is defined inside the file as a constant, ok it's sloppy and could be better, but this is not a vulnerability.

      They claim the extract line is insecure, but never explain why, and without showing the rest of the code (what is $requiredHeaders? If it's sanitized this is a non issue) you can't even tell if it's secure or not.

      The variable names being wrong (the $PayPal- ones) is probably just a "mistake" the model made, but it's not a vulnerability either.

      This article could be just "burglars are burgling in your neighborhood, install our burglar alarm" from crowdstrike just as well.

      • callahad

        Not a PHP dev, but it seems calling `extract` is itself potentially insecure. The php docs warn against it (https://www.php.net/manual/en/function.extract.php):

        > Do not use extract() on untrusted data, like user input (e.g. $_GET, $_FILES). If you do, make sure you use one of the non-overwriting flags values such as EXTR_SKIP and be aware that you should extract in the same order that's defined in variables_order within the php.ini.

        Looks like calling it can shadow previously defined, unrelated variables?

        Edit: On re-reading your comment, yes, if $requiredHeaders is previously sanitized then the code is technically safe. But it still presents as an insecure coding practice.

    • linkregister

      It's not surprising that guardrail triggers would degrade model performance. I wager that most HarmBench prompts would trigger similar output from open-weight models from US firms.

      The link you cited postulates it is accidental.

      • schiffern

        Calling the sabotage "guardrails" doesn't change the fact that it's been intentionally sabotaged against the US.

    • jfaat

      Certainly? I don’t have time to read the whole thing but-

      > we found that when DeepSeek-R1 receives prompts containing topics the CCP likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.

      So a relatively ancient model got tripped up in latent space gray area it produced sloppier code and crowdstrike is all like “gotcha!” Sounds like the basic thesis? Not great support, more like FUD

      • schiffern

          > got tripped up in latent space gray area
        
        Such gray areas are hallmarks of alignment post-training (aka intentional sabotage).

          >relatively ancient model
        
        "No fair, you haven't caught me lately!"
      • GeorgeWBasic

        Note that all of a sudden, the great majority of the comments on this page appear to be pro-US government and pretend that the CCP is somehow worse. Not a coincidence: the regulatory capture is starting right here.

        • schiffern

          Honored at the suggestion that I'm a pro-US shill, thanks. :)

        • linkregister

          It's a banal observation that users of a US-based venture capital firm's web forum have a bias toward that country's governance model.

      • unethical_ban

        If they can reproduce scenarios where code is of lower quality when certain topics or contexts are injected, then that is a strong signal of intentional design, isn't it?

        • jazzyjackson

          No; it’s a signal of lobotomy. Models get dumber near where you sensor them. You could almost think of it like your prompt is getting redacted, and so the LLM has to deal with these holes in your prompt (where you mentioned the semantic contraband) and it doesn’t do a very good job because it has no idea what you’re talking about.

          The mechanism is likely a bit inverted, it’s the weights that have been redacted, but the effect is that weights that were interfered with are affecting the problem solving abilities of the LLM - I mean, it’s insecure bad code, not a backdoor, right?

    • Melatonic

      Reminds me of the worms that won't hack you if the OS language is set to Russian

  • Ancalagon

    yes, thats the point

    • mainecoder

      Does not matter if it is opensource if a piece of rock is designated a National Security Risk because the Rock came from CHina that is it, that rock is a national security risk. The designation is assigned then action must follow; you cannot question a national security risk that is why it is designated a national security risk if it caused health problems you would be told by the Centers for Disease Control or US Department of Health and Human Services. A National Security Risk can be anything that they assign, it can even be a thought otherwise known as a thought-crime. Stop asking why?

  • AlexCoventry

    AI Models Can Provably Hide Arbitrary Capabilities

    https://arxiv.org/abs/2610.13928

    > Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilities can be elicited. We show this assumption does not hold under adversarial conditions by constructing backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger. Using digital lockers, these hidden circuits remain provably hard to elicit and interpret under cryptographic assumptions, even given full access to model weights, extending prior work from hiding fixed strings to arbitrary computations. We implement our proof of concept constructions in PyTorch and evaluate elicitation through fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training, using a small chemical reaction prediction task as a proxy for CBRN-relevant capabilities. Our results demonstrate resistance to these methods and, together with our cryptographic guarantees, reveal a limit of elicitation-based capability audits. We release our models to support the improvement of defenses.

  • rob74

    But think about the end of the world! Don't you want to prevent the end of the world?!

    /s

jacquesm

The models may well contain malware or security vulnerabilities. But they are at least open to inspection because the weights are available, the same can not be said about US providers of inference such as OpenAI and Anthropic.

  • halJordan

    They're about as indictable as binary blobs found in your BSPs or the 3rd party drivers you get through Windows Update

  • chrisra

    I don't know if inspection would be practical. How do you inspect the weights for an intentionally hidden trigger that writes malware?

    • kouteiheika

      You don't. You handle it the same way you handle security holes which every LLM (including "safe" American ones) will put in your software sooner or later -- by understanding and reviewing the output.

      • puelocesar

        Glad someone is pointing this out here on this community. I keep reading about people "solving" the problem of LLMs outputting too much code that's hard to review by not doing human review at all!

    • jacquesm

      I don't have the answer to that, my simple solution to this problem is not to trust anything a model ever writes and so far that strategy has worked very well.

  • justapassenger

    Not only open to inspection, you can also post train it to get exactly what you need from it, both in terms of capabilities and alignment.

  • nashashmi

    Because it is DeepSeek in China, they must have open weight models to be even partially acceptable to the world. I am sure many would balk at using a Chinese company if its products were not open source

    • deaux

      - Sent from my device containing a bunch of closed source products from a Chinese company

    • ulfw

      Ridiculous statement.

      Best greetings from my Oppo Find N6

  • cmrdporcupineOP

    It's also odd for them to single out DeepSeek in particular, and not just any model from the PRC?

    • jacquesm

      DeepSeek is the one that is the most visible, and probably the most cited in the departure messages to US providers.

throw0101c

> The order's summary of risks says DeepSeek's servers are in China and that Chinese law "permits extensive authority to government cyber and intelligence functions to access private company infrastructure".

So the Chinese version of the American CLOUD Act?

  • tzs

    Most countries permit their governments to access private company infrastructure.

singlewind

Why only DeepSeek?

gavinray

Here's the transcription of the reasoning:

  > Hangzhou DeepSeek Artificial Intelligence Basic Technology Research Co., Ltd. (hereinafter DeepSeek), a foreign, privately-owned company, located in the People’s Republic of China (PRC) provides multiple artificial intelligence applications available widely and free of charge. DeepSeek’s models present multiple threat vectors. DeepSeek servers are located in the PRC, and the PRC legal system permits extensive authority to government cyber and intelligence functions to access private company infrastructure. For DeepSeek-hosted instances of their models, DeepSeek’s account generation process collects personally identifiable information (PII) and may also track and digitally fingerprint users. DeepSeek’s most widely used applications function through a prompt-and-response process that requires presenting information to the model, and which may result in the transfer of sensitive data to the PRC if used by U.S. government employees or vendors. The transfer of the aforementioned information to the PRC, either independently or in combination with publicly available data, could produce actionable intelligence on U.S. persons and/or government programs. The models themselves may contain malware or other security vulnerabilities. Indeed, open-source security research has found that the DeepSeek iOS mobile application contains multiple critical vulnerabilities.
  • -1

    Replace every instance of "PRC" with "U.S.", DeepSeek with OpenAI/Anthropic, and replace "U.S." with any other nation, and maybe remove the last sentence. How does it sound now? Both nations may as well just say it - "we're banning it because they're our closest competitor".

    • nashashmi

      The difference is that local US companies have agency to refuse government overreach while Chinese companies are weaker in this regard. Chinese companies are more vulnerable to its govts pressure.

      • dannyw

        National Security Letters.

        Or, worst comes to worst, as Snowden has revealed, the NSA will just clandestinely hack American companies, as they did to Google when they rolled out TLS for search.

      • wickedsickeune

        As it has been proven multiple times with the current US administration, no, US companies do not have the agency to refuse government overreach, they can just purchase favours (which I highly doubt is different from PRC and its government).

      • deaux

        > The difference is that local US companies have agency to refuse government overreach

        We're on HN, not The Onion's comment section. People will take this seriously rather than sarcasm.

        It was sarcasm, right?

  • blueSky1989

    They are also banning local inference: "The models themselves may contain malware or other security vulnerabilities."

    It doesn't seem that they have found the models to be actively poisoned, though.

    • jacquesm

      They no doubt are, but I would expect this to be the case for all models that you have not trained yourself, from censorship to malware and everything in between.

    • halJordan

      They are not banning local inference. The USG has several fedramp dc's where they run local weights of several proprietary models. You guys are getting ridiculous

      • sullivanmatt

        Again, this would be local inference via DeepSeek. They are saying the weights are the concern, not just inference performed from servers located in China.

  • cmrdporcupineOP

    Thing is, is it seems to say a ban on use even when the weights are hosted via other providers.

    "third-party products and services developed using DeepSeek implementations"

    So you could be hosting it fully locally inside the agency and still it would be banned.

    It's not surprising to me, but it's pretty blunt. Not that I care what US agencies do, but this does seem like the first step of many to come.

    • tmp10423288442

      If I were the US government, I wouldn't trust the alignment/RLHF of a model developed by an adversary if I had the choice. Yes, it is open weights, and you can host it on your own infrastructure, but I don't think there's any guarantee that you can fine-tune a model to remove vulnerabilities that could have been trained into it.

      I realize that non-US nations could ask the same about the US. All I will say is that there are consequences to failing to nurture a competitive tech industry - even Mistral is basing their newest models on China open-weights models. Unlike the EU, the US government is fortunate to have a choice.

ssivark

Why specifically DeepSeek, when there's a whole cohort of Chinese open weights models? GLM, Qwen, Kimi, Minimax, etc.

  • nvme0n1p1

    DeepSeek is just the one with name recognition from the critters in congress. Remember when Nvidia stock had it's so-called "deepseek moment"? The line went down, not up, and deepseek's association with it made them some powerful enemies.

    Nobody in congress knows what a "Minimax" is, so they're safe from discrimination, for now.

  • keyle

    The others aren't risks to incoming IPO's, yet ;)

  • emilsedgh

    They'll go after those too. Give it a minute.

  • Rodmine

    It is THE competitor right now when it comes to price and intelligence. Also, just banning it plants fear in other sponsors and investors. When they do it like this, they can pretend that this is not a part of regulatory capture but it is an actual security concern because "this one in particular is dangerous" and "we'll tell you when others become dangerous" which will happen when they become a competition (now less likely because investors are cautious).

  • ulam2

    Testing waters, i think

  • okasaki

    Same reason they attacked Huawei, Tiktok, DJI, BYD... they became too well-known.

    What the western elites fear above all else is Chinese brands getting recognizable and popular.

mmaunder

And so it begins. Regulatory Capture Stage Complete!

sinuhe69

I wonder whether the guy who wrote the order understood how LLM works and how an open source model running in an US data center could pose a "threat"?

trilogic

Means Open router and similar services, are banned now?

asanineassasin

The system acknowledges even the slightest bit that the current course attacks the systems currency which is reward for loyalty aka shareholdertrust of the gentry. At the same time its pushing forth is a admittance that it has no control over the process happening, but finds expected outcome without so catastrophic that rolling that russian roulett dice is preferable. The whole mountainside of western civilization is in motion, but the rules still apply in this cabin, so "get the snow from your ski-boots at the door" kind of law.

anon373839

The timing of this is unsurprising, with Anthropic’s Q3 revenues flattening out and the IPO in jeopardy. I expect more efforts to prop up the money-furnace labs in the coming days.

monneyboi

Great advertisement for deepseek and a clear signal open models are winning.

gigatexal

When you can’t compete on price you ask your pedo Uncle Sam to help your businesses …

Deepseek models are awesome. I use them all the time and they keep getting better.

  • piokoch

    Yup, but you need to be aware that all information are passed to the security services of the Communistic Party of China and could be used against you. E.g. to blackmail you, if you are, say, soldier, public officer, higher up in some company, etc. Or to steal your data & money.

    Despite China having really great PR this is still a country which has exactly zero free speech, controls 100% of media and internet, keeps people in concentration camps.

    So this particular move does not surprise me, in fact, I am surprised that European governments does not get how much sensitive information is leaked through AI and they give all of that freely to third parties from China and also from USA...

    • gigatexal

      Google knows more about you or I than we care about. Facebook can target the one gay person in a town of 300 people. And so on, and so on.

      Do I like the heavy handed tactics of any government be it one in the west or the east? no.

      Can I do anything about it? Can you? haha no.

      Do you think user agreements mean anything? Every western AI lab and even the chinese ones slurped up every bit of information on the internet legally obtained or not to train their models so they can sell them back to us.

      > Despite China having really great PR this is still a country which has exactly zero free speech, controls 100% of media and internet, keeps people in concentration camps.

      ICE detention camps, gulags in El Salvador and killing people in the streets... Brenden Carr going after comedians on TV making big baby trump mad...

      China and the US are not so dissimilar. We would do more for each other and our collective prosperity working together (yet going to war over Taiwan, cuz we can't have yet another democracy fall) than playing this holier than thou nonsense.

alex-moon

Really interesting to notice a steadily increasing proliferation of anti-open weights and anti-non-US model rhetoric in the HN comments. Yesterday I saw a comment on another thread claiming a FOSS agentic SWE provider "shill cheap and shitty models", the kind of thing I'd expect from an Anthropic/OpenAI astroturfing campaign or a person with some kind of crippling substance abuse issues doing severe and ongoing brain damage - not really clear which it is but I can't readily imagine a more reasonable third alternative.

  • charcircuit

    Deepseek does not offer zero data retention. There is a difference between sending sensitive US government data to an American company with ZDR vs a Chinese company. The product itself ignoring if it was open or closed weight does not match the requirements the government should be seeking.

    • 233mhz

      You do realize deepseek models can be self hosted right?

      There is nothing else to see other than the US losing at their own game and panicking

      • vsgherzi

        I don’t think that’s fair. IMO I think they’re concerned that there might be precedent for current for future models to do things untoward to the user. Of course there’s no proof of this or anything I think they’re just preparing in a case where Chinese models do extract or harvest data even when self hosted. Given they have an internet connection of course

      • charcircuit

        In that case it wouldn't be a service "developed by Hangzhou DeepSeek Artificial Intelligence Co., Ltd (DeepSeek)".

        Again, hosting providers eventually get RCE by being able to send whatever bash commands they want to run on the machine running the agent. It is very important who you give this power to.

        • 233mhz

          > In that case it wouldn't be a service

          It definitely is a "product" developed by Hangzhou DeepSeek Artificial Intelligence Co., Ltd (DeepSeek) though

    • realusername

      > Deepseek does not offer zero data retention.

      I don't believe the ones which say they do. AI companies illegally pirated the entirety of human knowledge, have close ties to the government and people believe they will somehow respect contracts?

      Some AI companies are just more honest than others up front.

Good4boothee

What is the status of OpenRouter and other similar services? Are they also banned?

armcat

Does anyone know if this is based on the NIST report on the "Evaluation of DeepSeek AI Models" [1]?

> DeepSeek models advance Chinese Communist Party (CCP) narratives much more frequently than U.S. models.

[1] https://www.nist.gov/system/files/documents/2025/09/30/CAISI...

grafmax

The regulatory capture begins.

  • halJordan

    It's not regulatory capture to remove software/harware from adversaries.

    No one was saying "Oh McAfee is really gonna control the us's antivirus market regulations" after they banned kaspersky.

pknerd

Thanks, Dario

Cameri

It comes down to which backdoored models you trust.

johnnyApplePRNG

Deepseek V4.1 Flash 1017 is out of this world good ... and cheap ... that's why.

  • GlickWick

    That's not why.

    We've entered a pretty adversarial era, and we know there are unexpected attack vectors available in frontier models. Just because you an inspect the weights doesn't mean you can derive the behavior.

    I suspect the CCP would ban US models if they didn't need to distill them to make their own.

    • skeledrew

      > That's not why.

      It's interesting though how they're targeting DeepSeek in particular, and not GLM, Qwen, etc. Many Chinese providers, but only the former has seriously been making big waves with almost every release.

    • qs1ykyusPlate

      who is we? are you working for the government? this fucking obsession you people having with 'fighting the adversary' is ridiculous

      • entrope

        My reading of the comment was that the "we" who have entered an adversarial world means the entire world, and the "we" who know models can be backdoored means people who have been paying attention to published research on the topic. An adversarial world doesn't care how much you individually are interested in conflict, except to the extent that you might be an easy mark.

      • brazukadev

        To stay at the top you need to keep fighting. Lots of US americans believe that even if they are not benefiting anymore from being the richest country in the world.

    • 233mhz

      If that's truly your concern you should ban like 90% of anything containing any kind of electronic device because they're all either made in or transit via China at some point

    • tmp10423288442

      > I suspect the CCP would ban US models if they didn't need to distill them to make their own.

      I'm pretty sure they do, at least officially. But obviously the distillation and API proxies used by Chinese uses are unofficially tolerated and even exploited.

      • halJordan

        You're right, you can't run them in China, all models in China need to be vetted and approved by their cyberspace authority. But it's more of a "businesses cannot use them to provide services" kind of thing

thiht

So AI CAN be regulated in the end

himata4113

This appears to ban deepseek first party usage, not the model itself. It specifically mentions the iOS app as well.

  • Melatonic

    Yeah I think this is overblown

    Given what we've seen about the "aligned" models accidentally hacking recently it seems good to at least be cautious

CaliforniaKarl

https://archive.ph/MQIiN

jonathanstrange

I was planning to switch from Gemini to Deepseek and this news confirms I'm on the right track. Thank you, US government!

miroljub

This is actually good news for the Europeans and the Chinese. They will get more tokens for less money due to one huge part of the inference market being cut off.

crossroadsguy

This is the American way. More often after WW2. The country routinely goes to wars to impose and enforce its businesses. They have been known to do this in the name of democracy, the culture, and quite often in the name of freedom. This time it's safety; and no, it's not a war, not yet anyway – that's why this ban is nothing in comparison. Had it not been China, I'd not put any of those off the table for the land of the free.

phendrenad2

A good case of "Don't do A or B, because A is bad"

Stop downloading cookie recipes from China, and you can't use cookie recipes that came from China either, because accessing the cookie recipe in China could give you malware.

It doesn't make much sense but this is the government that gave us such greats as "the internet isa series of tubes".

sciencesama

how can an open source model be banned ?

metalman

the order(paraphrased) is 'we are imediately ordering you to, or not to, rip out deep seek, depending on how you feel about that ,and what might go wrong, and write to us about it, sometime in the next couple of months'

jonathanlotan

finally

matt3210

How bad would it be if China banned anthropic and openAI?

  • cocoa19

    Anthropic does not allow using their models in China.

    Also China bans US technology all the time. Remember the Great Firewall of China? Or when Xi told Trump, “no thanks” to buying NVidia cards

    • lilod

      "China bans US technology all the time" this is simply not true

      Chinese people buy iphone, buy tesla, use Windows, travel with USA made airplane etc

    • SXX

      CCP is shit and bans tons of stuff, but this is not how it happened:

      > Or when Xi told Trump, “no thanks” to buying NVidia cards

      US banned sales of high-end compute tech to China for years. Now when they actually catching up on production of their own SSDs, DRAM and compute silicon they no longer need US offers.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection