Haha, and I almost felt bad after seeing this chart yesterday.
20 hours ago [-]
bwfan123 22 hours ago [-]
> For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework
So, are they planning to announce an optimized coding agent harness as well ? DSv4 flash is a fantastic model, and my daily driver. With reasonix or pi, I can code all day long and pay a few pennies for it. No token anxiety. Whereas the same model with fireworks/openrouter, with zdr thrown in, token costs ratchet up with no explanation. Likely that the model is subsidized for gathering usage data. I am waiting for the day I can run this locally.
segmondy 22 hours ago [-]
They announced it already, read the tech report.
"For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95."
bwfan123 21 hours ago [-]
> They announced it already
I meant something I could download and run.
ignoramous 20 hours ago [-]
Isn't Reasonix [0] DeepSeek's own harness? Or, are they building a new one?
If you're not paying for the model through Open router, then which provider are you buying it from? I thought that if that provider was listed on open router it would be the same price as going directly to the provider? Are you buying from a provider that isn't listed on open router already?
bwfan123 21 hours ago [-]
> Are you buying from a provider that isn't listed on open router already
I am using openrouter with zdr guardrail which routes to any provider that supposedly doesnt train on user data. I also use fireworks (directly, not via openrouter) which is a provider promising zdr and has a bunch of open weights models. My issue is that these zdr providers dont transparently disclose caching/tokens etc and so they end up being far more expensive than directly using DS.
smartbit 5 hours ago [-]
> OpenRouter has taken the stance that in-memory caching of prompts is not considered “retaining” data, and we therefore allow endpoints/models with implicit caching to be hit when a ZDR routing policy is in effect. [0]
OpenRouter doesn’t list DeepSeek as ZDR combined with Caching. Thanks for the tip of Fireworks as Fireworks has Zero Data Retention by default. [1]
Have you tried Novita? They're zdr and I find they usually have better cache hit rates than fireworks. No affiliation.
bwfan123 17 hours ago [-]
Thanks for the input. Let me check it out.
0cf8612b2e1e 23 hours ago [-]
Somewhat relatedly, how do the economics for Huggingface work? They must be hosting petabytes of models and datasets by now. I have downloaded quite a few “just in case”, only to replace them with the later iteration months later.
Does the file hosting actually cost peanuts when you do it yourself and the cloud has shattered my understanding of what it actually costs to deliver so much data?
wongarsu 22 hours ago [-]
File hosting is pretty cheap. Egress traffic is about $90/TB in the cloud, but around $1/TB in the real world. Storage is in the realm of $5/TB/month after adding redundancy
At the scale of Huggingface, that still amounts to a lot load of money. Significantly less than if you did the same in AWS, but still a lot
That said, they do have a deal with AWS to make the data available in AWS ip space. Maybe they got some cheap hosting out of that too
oceanplexian 20 hours ago [-]
It's less than $1/TB. If you have settlement-free peering it's like fractions of a penny at scale. Cloud provider DTO pricing is literally the biggest scam on Earth.
flyingpenguin 17 hours ago [-]
My company spends $8,000,000/month on storage and $4,000,000/month on data with aws... In the financial reports we do we round this number away. Make of that what you will
Bayart 3 hours ago [-]
"Nobody ever got fired for buying AWS."
gandreani 19 hours ago [-]
What does DTO mean in this case? And settlement-free peering? I didn't know the price for S3 could be so cheap with AWS
18 hours ago [-]
Bayart 3 hours ago [-]
Nowadays storage and bandwidth is cheap as long as you're not going through a hyperscaler and it doesn't require compute (ie. "dead" data). I'm working mostly on GCP so I'm not sure what are the rates now, but at least AWS were known for billing insane numbers on egress.
As long as you're smart on caching on the edges and deduplication/overlaying with your content (which structured data types like models, containers and repos typically are fit for) you can get remarkably far for less than you'd think.
The economics of serving models seem to be way more dodgy.
anon373839 14 hours ago [-]
HF uses chunk-level deduplication that greatly reduces the amount of physical storage needed for a given amount of logical storage.
Bandwidth is really cheap when you run your own infra
himata4113 21 hours ago [-]
It's not only cheap, it's actually free since if you are not using your 95th percentile you are losing money.
Scoundreller 17 hours ago [-]
Some enterprising ISPs get into the hosting business because most of their traffic is ingress, so adding some egress revenue is effectively zero cost.
miyuru 22 hours ago [-]
when trying to download models using browser, the DL domain for me us.aws.cdn.hf.co which points to bunch of singapore AWS IPv4 for me, despite it saying US.
jvuygbbkuurx 23 hours ago [-]
Maybe download numbers are relatively low
rupx 21 hours ago [-]
Download numbers are reported on the site, and they aren't low.
CDN costs are pennies compared to inference and training though, HuggingFace will just get another 100 million and be set.
throwaw12 1 days ago [-]
If deepseek v4 flash is beating DeepSeek V4 Pro, can we expect new V4 Pro which is on par with Opus 5 in couple weeks (even better if it beats Opus)?
jmathai 1 days ago [-]
I’ve been using v4 flash for an app I’m building [1] and it’s amazing how cost effective and good it is coming from having always used gpt, opus and sonnet models.
It’s so cost effective I can offer a generous free tier since my goal isn’t to make money with it.
I get where you're coming from, and the intent to make it easier for people to find examples and verses, but there's a fine line with LLMs giving you answers, is that it's interpreting it in some form. Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights?
jmathai 24 hours ago [-]
You’re correct. The hard line for me is ensuring that any scripture presented to the user is verified correct. LLMs can’t be trusted in this regard.
One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source.
I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
weiliddat 23 hours ago [-]
I don't necessarily mean reguritating it, but choosing which part of the scripture to surface to the user is already some interpretation/choice. Even the devil can quote scripture (I'm playing the devil's advocate here).
jmathai 23 hours ago [-]
Very true. The verses selected come from a tool call. The LLM queries the app for relevant verses. It picks the keywords - so perhaps there’s bias there but the verses are handed to the LLM.
But there are ways to control and constrain the LLMs and what the user is presented with.
These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
raynhardt 21 hours ago [-]
How polished is the experience, I see there is an invite link on the website.
jmathai 20 hours ago [-]
It’s pretty polished but you’re asking the wrong person. Feedback has been positive so far :).
hirako2000 24 hours ago [-]
It doesn't prevent you from going to the source and struggle with the text, nor seek expert commentary.
What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc.
This app does that very well, LLMs are good at entity recognition.
weiliddat 23 hours ago [-]
Yes it can point you to the right place, but choosing which part of scripture to point you to is already a choice/interpretation
swat535 20 hours ago [-]
> Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights?
I think it depends, Catholics wouldn't be able to use this because the Magisterium is the ultimate authority on interpreting Scripture, so the personal interpretation isn't really needed. This is not to say that Catholics don't read the Bible, they are encouraged to do so since it deepens their faith
On the other hand, for Protestant it varies, the High Church denominations are closer to Catholics (though none of them accept the Magisterium) in terms of scripture interpretation, but the Low Church ones (like Baptists or Non-Denominational ) are more open to personal interpenetration.
Disclaimer: I'm a Catholic, so if I made a mistake here fellow Protestants, please correct me.
foltik 23 hours ago [-]
Just let them have their AI pacifier.
tmaly 14 hours ago [-]
have you used it for anything outside of coding?
jmathai 12 hours ago [-]
I tried using Deepseek with Claude Code and it under performed. The use I was referring to was within a product and I feel it's a lot of value for the cost.
Is the difference between this and a frontier model that the scripture is guaranteed to be real?
I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails.
I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost.
If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet.
I hope the irony of being concerned about hallucinations in a bible study app is not lost on you or others reading this!
swingboy 3 hours ago [-]
Sir, you dropped your fedora.
jmathai 23 hours ago [-]
Philosophically, I don’t believe we should outsource the accuracy of scripture to any single entity (let alone a for profit secular one).
So it’s less about model choice and more about governance of scripture.
I will check out the link you sent for sure!
ComputerPerson 22 hours ago [-]
I was under the impression you were getting BOOK.CHAPTER.VERSE references from the model and then sourcing them from a ground truth db.
Anyways, impressive app! We haven't tackled such an ambitious project just for it being daunting.
jmathai 51 minutes ago [-]
Took a look at StepBible. Looks really cool. My email is in my profile if you wanted to connect.
irthomasthomas 21 hours ago [-]
It can still generate a wrong reference and select the wrong verse.
jmathai 20 hours ago [-]
That’s okay, IMO. Ultimately it’s the readers responsibility to apply their own intelligence. But the app should not misquote scripture itself.
kfse 23 hours ago [-]
All models hallucinate. Everything in production with a model will too. It is not a "deal breaker"
vmt-man 23 hours ago [-]
> Is the difference between this and a frontier model that the scripture is guaranteed to be real?
flash is suitable only for a toy apps, not for production environments :)
websap 1 days ago [-]
I’m salivating at the thought of this!
Deepseek v4 Pro prices with Opus 5 perf would be freaking unbelievable!!
This is probably a dream.
adrian_b 23 hours ago [-]
Yes, they said that the updated V4 Pro will be published soon.
lionkor 24 hours ago [-]
We can expect a new v4 pro, this was a footnote in the v4 flash announcement earlier.
adrian_b 23 hours ago [-]
They have said that the updated final version of V4 Pro will be published soon.
scosman 23 hours ago [-]
So GLM 5.2/Gemini 3.6 level intelligence for $0.28/m output. And their updated Pro model coming soon....
Plus a size you can genuinely run at home: Unsloth lossless Q8 at 162GB.
luckydata 22 hours ago [-]
I would like to see your "home"
cmrdporcupine 21 hours ago [-]
Two (linked) DGX Sparks would do it I guess. Though probably slowly (I'd guess 15-20 tok/sec for decode, but higher for prefill). So ~$8-9k USD at current RAM prices, substantially less if they ever (sigh) drop. Electricity use would actually be relatively modest.
But it makes little to no sense as long as API prices are what they are. Except for maybe privacy reasons.
wolttam 14 hours ago [-]
2 sparks currently run this model at 60 t/s single session, up to just over 100 t/s aggregate with concurrency of 4.
Going local has as opened up a world of use-cases I never would have entertained the idea of on metered/cloud usage. Privacy is a large part of it but, I also no longer think twice about whether to send a prompt or not based on the psychology of it costing money.
Cached input tokens on local inference are free, so I
don’t care about running sessions up to 500k tokens and hundreds of turns (it’s rarely useful, but DSv4 remains surprisingly coherent up there)
vardalab 16 hours ago [-]
It's at least 2.5x that speed for dual sparks and prefill is good as well.
Basically going on vibes it is faster seeming than what one gets by default with openAI or Anthropic.
ycui7 21 hours ago [-]
the rational in one’s mind is similar to buying expensive supercar but no driving it daily.
owning a few GPUs is a lot cheaper than supercars.
cmrdporcupine 21 hours ago [-]
I dunno. I bought the Spark in January and it has led indirectly to paid work.
I don't use it for local inference so much. I use it to learn.
I also use it as my daily driving Aarch64 development system.
Aside it's also very cool what else can be done with unified GPU memory, once you realize you have it...
bethekind 18 hours ago [-]
Learn model deployment, batching, all the AI inference related things?
Maybe I'm reading that incorrectly, but it seems to me the cost is on the X-axis.
First, your direct comparison, Deepseek V4 Flash 0731 (max effort) $0.03 (rounded up) per task @ index 50.
OpenAI Luna:
* high effort $0.03 (rounded down) @ index 46
* xhigh effort $0.04 @ index 49
* max effort $0.07 @ index 51
So I would say a fair statement would be "OpenAI Luna between 2x and 3x the price of Deepseek Flash, what you get is 2 to 5 times faster inference"
The cheapest OpenAI model that beats it is OpenAI Luna (max effort) $0.07 @ index 51 (if you take the rounding out it summarizes to triple the price for similar performance), but still close to 3x faster.
And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
andai 1 days ago [-]
For anything substantial, you'd want a bigger model anyway.
For simple tasks, they're already saturated, and you'd prefer the faster model, so that you can have a realtime/interactive-ish experience.
Or to put it bluntly, it's cheaper if you don't value your time. That goes for smaller models in general -- need more handholding, more correcting -- but the Chinese ones are slower on top of that.
As for speed, Sol on Low is faster than Luna on most settings.
seaal 23 hours ago [-]
>And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
uhh openai is dark gray: `rgb(31, 31, 31)`
and i'm pretty sure it always has been?
ComputerGuru 20 hours ago [-]
If only they would let you opt out of training use, it might actually be a viable option.
slopinthebag 20 hours ago [-]
The training use is probably responsible for this improvement tho.
ComputerGuru 16 hours ago [-]
They could certainly still achieve that even if they had a slightly more expensive tier with a (pinky promise) "no training" bit set, or even if it were just an opt-in most users wouldn't bother with.
akurilin 21 hours ago [-]
Came here to see how it performs against Luna. Would love to see how they compare on specific types of tasks.
kamranjon 22 hours ago [-]
The really interesting thing about this is how big of a jump was achieved with just extra fine-tuning here. No structural changes to the model, just more data, compute and time. It makes me pretty excited for the future of small models - DS v4 flash is a relatively small model when compared to the class it's competing with, so likely similar gains can be made applying quality data/training pipeline to other smaller models.
ycui7 22 hours ago [-]
For people with single RTX PRO 6000 96GB or DGX Spark 128GB, vllm-moet is a very good engine, although lesser known. It auto generate a symmetric 2-bit plane for inference and also generate a 4-bit delta cache to recover precision. Support ssd streaming oversized weight. You pick how much VRAM to allocate to each to balance out speed vs precision. 170 tps with ds-v4-flash demonstrated.
It use the stock model, no new models requires.
Worth spend a few hours to try.
The DGX Spark requires a small hack to ignore the difference between sm120 vs sm121, but it does run on sm121.
WhitneyLand 1 days ago [-]
It’s exciting that a model scoring this high is dirt cheap.
It’s also so inefficient, when they release the full performance numbers it’s not going to be good.
One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.
Bnjoroge 24 hours ago [-]
Using tokens to evaluate models is an outdated approach. Cost per task is what matters. Not all tokens are created equal
drob518 23 hours ago [-]
Yes, but… more thinking tokens also means longer solution generation time. That said, v4 Flash is a fast model. I use it all the time because it’s very smart for the price. But it is verbose sometimes.
kzrdude 23 hours ago [-]
The thinking trace was (preview) frustrating to read, I think I'd prefer a summary view of it at this point.
WhitneyLand 21 hours ago [-]
It’s not outdated at all to use tokens to estimate performance, it’s directly related.
prism56 16 hours ago [-]
But why should I care? If my metrics are speed and cost? How many tokens it takes as a user is arbitrary to some extent.
WhitneyLand 15 hours ago [-]
If speed is a metric for you, tokens required to solve a problem affects that metric.
All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.
Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.
rat9988 11 hours ago [-]
Still, seems useless to measure tokens when you can measure speed.
mdp2021 20 hours ago [-]
> inefficient
That depends. Is it also more reliable?
If two books, one big one slim, prove the same thesis, what I would be interested in is the quality of the content, not the size. There can be a measure of efficiency in "have you really thought it through", but it is clearly complex - it requires measuring how solid the reasoning is.
ptole_my 20 hours ago [-]
you are correct. but in my experience -- not benchmarks -- g flash 3.6 is SO BAD for coding. I'm using all vendors all day and gemini is the worst by far. I built my own semi-deterministic orchestrator for coding agents.
onlyrealcuzzo 23 hours ago [-]
Roughly equivalent to Gemini 3.6 Flash in capabilities at 1/20th the price...
Mind you, until the recent price cuts to Luna - Gemini 3.6 Flash wasn't even egregiously priced (but oh how things change in just 1 week).
Can't wait for the DwarfStar quants - I have been using DeepSeek v4 flash (preview) as my main coding agent for months now (running on my 128gb mbp) - it seems this model outperforms GLM 5.2 on nearly every metric. Thanks for sharing the news, I was refreshing huggingface but gave up thinking it likely would take some more time.
22 hours ago [-]
vmt-man 23 hours ago [-]
are you working in earplugs? :)) even with 128 gigs of ram it must be super noisy.
kamranjon 22 hours ago [-]
I actually run it as a server - so most of the time I don't have to listen to it right next to me - it's just sitting in another room in my house - but I often am traveling with it and will have it sitting right next to my coding laptop and yea the fan runs non-stop - it's not obnoxious so i can pretty easily tune it out - also airpods/noise canceling headphones help!
FuckButtons 21 hours ago [-]
There’s fan noise, but the acoustic engineers at Apple have done a very good job of making it be pretty much not noticeable to someone who has noise sensitivity (me).
hetspookjee 23 hours ago [-]
the noise is pretty doable, and so is the heat, I find. I read these messages before I had the machine and expected far worse. The tok/s is for me the dealbreaker as I prefer to have multiple sessions. For nightly runs i do like it a lot, or to be a node in a mesh.
ycui7 21 hours ago [-]
my AC is noiser than my GPU server.
tough 21 hours ago [-]
I've put Opus to it and it says it will take 3-4h to do the process to the new weights. hoping it works!
ycui7 21 hours ago [-]
you don’t need new weight. try vllm-moet from github. it will autogenerate 2-bit plane.
theturtletalks 23 hours ago [-]
What kind of tps are you getting?
kamranjon 22 hours ago [-]
Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
Lwerewolf 22 hours ago [-]
I've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.
knuppar 21 hours ago [-]
same. been running it in a dgx spark and it slaps
walrus01 22 hours ago [-]
the unsloth GGUF at <165GB will run on most 256GB RAM pure-CPU systems (or with llama-server and a mix of loading as much as you can onto a single 32GB, 48GB or 96GB GPU and the rest onto system DRAM).
Hmm, it's targeting a HW accelerator with a 128x128 matmul primitive, which one is that? Warp Group Matrix Multiply Accumulate on H100?
ttul 22 hours ago [-]
Google TPUs are built around a 128×128 systolic array of multiply-accumulate (MAC) units. Trainium 1, Trainium 2, and Inferentia 2 also feature a 128x128 systolic array.
You learn something every day. Today, it was the term "systolic array": A systolic array is a specialized grid of simple, interconnected processing units designed to execute parallel data operations—like matrix multiplication—by rhythmically passing data directly from cell to neighboring cell without writing intermediate results back to main memory.
The term comes from the biological word systole (the contraction of the heart pumping blood through the body). In a systolic array, data "pulses" through a network of processing elements on every clock cycle, driven by a global clock beat.
pstuart 21 hours ago [-]
TIL. TY.
andersa 22 hours ago [-]
Just because scales are grouped by 128x128 tiles, does not mean you need a single compute tile that large. It works completely fine to process it with multiple smaller tiles that get given the same scales, like how this works on Hopper and Blackwell today
22 hours ago [-]
ycui7 21 hours ago [-]
Huawei Ascend NPU
porridgeraisin 22 hours ago [-]
Blackwell. You could do it with wgmma on hooper but then you'll only be able to run 1 CTA per SM. There are cases where this is OK, but most commonly 128x128 mmas are primitive in tcgen05.mma i.e blackwell. The fundamental reasons is that the systolic array accelerator (TMA) until blackwell wrote to the cuda core registers themselves so you were limited by the register file size of the SM[SP]. In blackwell there's a separate TMEM where the mma unit stores it's output.
You can even do higher 256x256 and such using a hardware feature called 2-CTA MMA, this is essentially them letting neighbouring pairs of SMs co-operate and access each other's memory.
As for sibling comments, huawei ascend is more of an NPU-style architecture where you can easily have much bigger MMAs as primitive. But you usually don't anyways for many reasons.
New Deepseek models are like Christmas for me. Really big fan of low cost API models, noone does it better than DS. Until VRAM price is low enough to run models locally, this is the way to go.
The subsidized subscription model won't last, API pricing "feels" closer to a true sustainable business model.
Flere-Imsaho 23 hours ago [-]
Indeed. My fellow software engineers keep complaining about using up all their Claude tokens within an hour... Whilst I'll be rocking DS flash for the entire day. Sure it gets a few things wrong here and there, but that's when you pull out the Claude models or whatever for those tricky tasks.
ljosifov 23 hours ago [-]
Same. And I have come to use OMP (oh my pi) agent /advisor mode to put a 2nd model on the case (also mid-size one), reading everything. It can not block anything or change anything - just inserts comments in the text stream with 1 turn delay. Good portion of the time it's quiet. I'd say 1/2 of the time it's got something to say. About 2/3-rd of that the 'advice' is insubstantial or about something not-quite wrong. The good thing is the main model is confident - checks and then it stands its ground. Have not noticed it turning a right into a wrong b/c of advisor false alarm. And in 1/3-rd of the advice, it's a genuine defect teh advisor noticed, the main model works out a fix. This is my approximate feeling just observing the process, have not got collected the data. Afaik only OMP has advisor mode. Agent pi has plugin pi-omplike-advisor. For agent Hermes I had them code me an /advisor plugin (for now -0.1 old v0.18.x; yet to upgrade it to latest).
jmartrican 22 hours ago [-]
The problem is picking between models. I do not want to spend my time switching models and trying to decipher which model should be used for what. Maybe that's just a me problem that I need to figure out.
wongarsu 22 hours ago [-]
DeepSeek v4 is honestly good enough that I'm fine throwing it at everything in my hobby projects. I guess now I'll be switching from V4 Pro-Preview to V4 Flash. My only real complaint is that they can't do images, which limits their ability to autonomously debug some kinds of issues
Of course you can get more bang for your buck by being more deliberate. But that's equally true with US frontier models. You can optimize your work by choosing between Opus, Fable, Sonnet, Sol, Luna and Terra for each task. Some people seem to prefer to let Opus code and Sol review, for example. And then there is the whole debate whether $current_version is actually better (Some people stay on Claude 4.8 because they dislike how 5.0 is sometimes doing stupid things, just as many opted out of dynamic reasoning when they still could)
rented_mule 20 hours ago [-]
CodeWhale is a coding agent that auto-routes requests to Flash / Pro based on complexity, as determined by Flash. It's also tuned for DeepSeek's caching behavior, making things even more inexpensive. I'm retired, but I've been using it for just over two months at about the rate I would use it if I were working half-time, and I've spent $19 total.
what plan are your 'fellow software engineers' using? I have a hard time even using up the Fable part of my allowance in a week of coding.
Flere-Imsaho 23 hours ago [-]
I'm not sure to be fair, but they do have constant "token anxiety", which I simply don't have anymore since using v4 flash.
dominotw 23 hours ago [-]
why do they do if their employers are paying for it?
0xc133 22 hours ago [-]
because once they run out of tokens they can’t do their jobs anymore
Demiurge 15 hours ago [-]
This is a sad state of affair. I'm faster with the LLM, but I sure as hell can do everything just like I could, as before the current state of things.
dominotw 18 hours ago [-]
really? which employer gives you fixed tokens to do your work?
stavros 19 hours ago [-]
I'm on the Premium business plan and I can easily use up my entire week's allocation in two days of coding. I only use Claude for planning, too, the rest is done by Deepseek and GPT.
Claude is very spendy.
felixgallo 17 hours ago [-]
Claude is about the least spendy thing a software engineer can use. If you can't afford $200 a month, you're probably not working professionally (which is ok, no value judgement, but then being on the premium business plan is a little strange).
stavros 17 hours ago [-]
There are two Team plan tiers: Standard and Premium. Not sure what that has to do with working professionally.
ReptileMan 23 hours ago [-]
usually I am - Codex, make pi with deepseek to do something.
VulgarExigency 24 hours ago [-]
I would bet that Deepseek API pricing is still more cost effective per token than the subscriptions. With the increase in quality Deepseek Flash just got (in my personal testing so far, it seems to have improved a lot at following instructions, and has become more proactive), there really isn’t anything that can match it in terms of cost effectiveness.
rapind 22 hours ago [-]
The issue for me is the data privacy if you're using their hosted prices, because you cannot opt out of data collection (and I'm not sure how you'd legally follow up on it if they did offer it, but didn't actually follow through). That means all of my code is being retained for training.
I've used it in some open source code though, and loved how fast it was.
My mind is changing on how valuable my code actually is though... it's the complete picture, how it's put together, the design, the UI, the attention to detail that's the real value.
nodja 21 hours ago [-]
I'm not a heavy user of agentic coding, but still use them quite a bit for some automation here and there. I've been going around shopping all the ~$10 subscriptions and I finally settled on openrouter + ds4 pro. The more intensive days cost me $1 and I set a $2 weekly limit which I've never hit the past 3 weeks, to me it's way cheaper than most subscriptions and I don't have to worry about maximizing my weekly quota/resets.
singingtoday 21 hours ago [-]
I have one of my projects on DeepSeek and it blows my mind how I can use tens of millions of tokens for pennies.
I don't find it suitable for everything, but there's some tasks it crushes for what feels like almost free.
24 hours ago [-]
refulgentis 21 hours ago [-]
FWIW, looking at pricing myself, it needs to 5x the tokens to get the same results as GPT 5.6 Luna at a much lower TPS. (doesn't obviate the fact that the pressure causes incumbents to push and push to optimize, thank you Deepseek!)
22 hours ago [-]
jingpostmedia 24 hours ago [-]
[flagged]
Archit3ch 23 hours ago [-]
If it matches GPT-5.4 on coding tasks (as benchmarks suggest) this could be my forever model. And with partial SSD streaming, I could run it locally today. :D
1saadcodes 13 hours ago [-]
What's most impressive here is that that DeepSeek keeps showing how much of a performance improvement can come from post-training alone while the architecture stayed the same. It's a strong reminder that we're probably underestimating how much optimization is still left after pretraining
Catloafdev 21 hours ago [-]
If any editors are reading this thread, the weights are on HF now, but the 'open source' question still implies it's proprietary/API-only.
apitman 20 hours ago [-]
Wow this thing just blew out the pareto front for intelligence/$
mchusma 18 hours ago [-]
If Luna hadn't price dropped yesterday it would have been a real blowout.
cheesecakegood 16 hours ago [-]
I wonder if OpenAI knew and deliberately price dropped to front-run the news cycle in their favor.
f311a 1 days ago [-]
The model is already up on Opencode, but they require a consent to use Chinese datacenters.
momojo 13 hours ago [-]
I use DeepInfra. Their whole catalogue is openai compat. US based.
esafak 21 hours ago [-]
Apparently `deepseek-v4-flash` automatically routes to the new model, if you are using the official provider:
The official release of the DeepSeek-V4-Flash API is now in public beta. The API calling method remains unchanged — simply set the model name to deepseek-v4-flash to use the latest version.
Can't wait for China to catchup on hardware (memory and compute) and absolutely crush American companies in both price and performance.
hilios 23 hours ago [-]
I'm not looking forward to it, them being strapped for resources provides a huge incentive to develop and release these smaller models. Even if they'd still release their models once they are able to comfortably service all potential customers via their cloud, running them locally would be almost impossible due to their size.
seanmcdirmid 22 hours ago [-]
Chinese are pretty pragmatic. Even if they produce more expensive chips and memory, they are still going to focus on value. But who knows, maybe India will step up again like they did for Y2K 3 decades ago.
shwetanshu21 19 hours ago [-]
Unfortunately, India's focus is on everything but models. Hope I am wrong but the VCs there are now very risk free and are only investing in Snacks, Beauty and similar companies making risk free money due to growing income of population.
All the good talent moved to other countries due to this.
I said absolutely nothing about Americans in my comment so I’m a bit confused by your comment.
otabdeveloper4 19 hours ago [-]
Lol, Americans (the City upon a Hill and New Jerusalem people) are the least pragmatic people on earth.
Wherecombinator 19 hours ago [-]
Looking at the oligarchy and excess of the United States, no.
ndkddmmfmfm 19 hours ago [-]
[dead]
LoganDark 21 hours ago [-]
I agree. A meeting transcript posted recently from DeepSeek's founder suggests they're going to reach for larger models as soon as they can, and not look back. The model is already bigger than my machine can handle, though (I cannot put up with 25 t/s after getting used to over 100), so I'm not impacted.
baalimago 23 hours ago [-]
Personally I'm hoping for EU to step up a bit
chronogram 20 hours ago [-]
We'll make sure there will be enough regulations to have neither the electricity to power it or the companies to build it.
fy20 19 hours ago [-]
Well quite a few nuclear powerplants have had to shut down recently due to droughts... So we are well on the way for that!
sroussey 16 hours ago [-]
1970s design.
speedgoose 22 hours ago [-]
Sorry we a focusing on leather handbags there.
seanmcdirmid 22 hours ago [-]
The leather bag designs are shipped from Paris or Milan to China for production.
serial_dev 20 hours ago [-]
But some worker in Italy bolts on a zipper in the end so it’s actually Italian, believe it or not.
Gud 20 hours ago [-]
That worker is also Chinese, but in Milano.
speedgoose 19 hours ago [-]
It depends on the bags. I know people who work in handbags production, in France.
tao_oat 22 hours ago [-]
Seems incredibly unlikely, doesn't it? (I hope for the same thing but it feels almost like wishful thinking!)
ungovernableCat 18 hours ago [-]
The EU is not a state or some federation. It's like a NAFTA+
It unfortunately doesn't have the risk averse VC system (or pragmatic state driven like China) ecosystem to support the scaling required.
But if models will be commodities, maybe that won't be such a terrible thing longterm?
Barrin92 18 hours ago [-]
Personally I don't. Wasting enormous amounts of resources on a technology that doesn't make money, can easily be copied, and eats up everyone's energy while at the same time can be downloaded over the internet is kind of stupid.
The fact that China could catch up in what, two years, shows that this is a commodity.
lerchmo 18 hours ago [-]
A fantastically useful commodity that resists monopoly is the best outcome.
m00dy 21 hours ago [-]
EU is out of game and can't be back anytime soon.
Nifty3929 16 hours ago [-]
How about we hope for Chinese labs to continue pushing forward - and for America and the rest of the world to do so as well.
It needn't be competitive - everybody can succeed more tomorrow than they are today. You win. I win. We win.
Saline9515 22 hours ago [-]
Good luck when the last non-Chinese frontier labs will have closed and the CCP will ask to stop sharing models open source.
InsideOutSanta 22 hours ago [-]
We'll never have fewer open-weight models than exist now. They won't suddenly disappear when labs stop publishing new ones. In fact, people will keep improving them and will keep distilling new frontier models into existing open-weight models.
codedokode 19 hours ago [-]
Then non-Chinese labs can reopen, or they could offer lower prices right now not waiting for this event.
But imagine if Chinese labs would lose. Then there will be no open models, and the prices for closed models would be raised to the maximum.
donquichotte 22 hours ago [-]
My conspiracy theory is that this is the new space race, and the CCP encourages this to show the world what Chinese engineers are capable of, and tank the Anthropic/OpenAI valuation bubble as a desirable side effect.
throwa356262 21 hours ago [-]
Half of the engineers at OAI and Anthropic are Asian, I don't think China does all these for signaling.
donquichotte 5 hours ago [-]
I was not trying to make that kind of argument, but rather a geopolitical one.
Retaining world-class talent and home-growing industries is no small feat. In Europe, we have been struggling with this, and I am not even entirely sure why.
singingtoday 21 hours ago [-]
What an odd thing to say
u8080 20 hours ago [-]
Oh no, 1kkk market country with top-tier research labs developing its own technology, must be evil!
arjie 20 hours ago [-]
Well, no, they're not aligned with our interests therefore we're concerned with their mastery here. You have the clause the wrong way around.
shimman 18 hours ago [-]
As opposed to closed source American models where the federal government threatens your livelihood unless you build data centers to extend pax silica?
slopinthebag 20 hours ago [-]
This will never happen btw.
xXSLAYERXx 18 hours ago [-]
You're clearly not an American and they will never catch up on hardware. I'm mostly curious why you are so excited for China to "crush" American companies?
eunos 14 hours ago [-]
The imperial hubris need to be humbled sometime
xXSLAYERXx 13 hours ago [-]
Yes agreed that is how the world turns but given the state of world I don't see it happening in any of our lifetimes.
johnny_rico 18 hours ago [-]
Incursions into other nations "just because"? Economic wars, and support for foreign governments or armed groups that caused harm or instability to the entire planet? You know? Planet Earth?
Sanctions and trade policies that hurt ordinary people? Cultural dominance through Hollywood (Holy Wood, again with the sick ruling elite and their witchcraft crap), social media, brands, the poisoning of food and water on global scale?
Etc? Etc?
Most people don't hate "Americans" or their companies, they are just against your ruling elite. Live one day outside the united states and understand how the planet works for the rest of mankind and you might start to grasp on reality.
And yeah, China will catch up with hardware pretty damn soon. Jesus, you guys bought your radios from Japan not 50 years ago. Sorry, can't dominate the world's economy forever. Never happened in history.
No matter how much black magic, dark rituals, sick crap and sacrifices people on the top throws at the wall.
We're humans, after all.
So yeah, it is not against America itself or American companies, but against the ruling class. So naturally , people get excited about China entering into the match!
3...2...1... Fight!
xXSLAYERXx 15 hours ago [-]
Come on. You can say all that nonsense about almost any other country.
Most everyone wants to come to America for the opportunity. Think an immigrant in China has the same economic opportunity as in America? Any other country come to mind?
Danox 13 hours ago [-]
Never?… Ha Ha Ha
xXSLAYERXx 44 minutes ago [-]
Right. I misspoke. Should have said not in our lifetime.
antonvs 15 hours ago [-]
American ultracapitalism seems to be approaching its final stages. That end can’t come quickly enough. If China helps speed it along, that’s a good thing for all of humanity.
xXSLAYERXx 15 hours ago [-]
> seems to be approaching its final stages
Sure, if you spend most your time on the internet in various news forums its doom and gloom. Its what sells.
segmondy 21 hours ago [-]
:-(
23 hours ago [-]
dominotw 23 hours ago [-]
[flagged]
dpacmittal 22 hours ago [-]
I'm neither pro China, nor pro US. I'm pro open weights models, and I'm pro cheaper hardware. At this point I don't see any american frontier labs releasing SOTA open weights model, and I don't see ASML/Nvidia/Samsung monopoly getting any competition from anywhere apart from China in the near future.
dominotw 22 hours ago [-]
> I'm neither pro China, nor pro US. I'm pro open weights models, and I'm pro cheaper hardware.
yea i got that from your first comment ( although you removed crush American companies in _price_ ). you are pro cheapness at any cost even if its from your country's state funded direct geopolitical enemy.
China can always count on first order greed to win
satvikpendem 22 hours ago [-]
Why does them being a geopolitical enemy matter to me as an individual?
VulgarExigency 22 hours ago [-]
And what is their nefarious plan after we get "addicted to cheap shit"?
dominotw 18 hours ago [-]
same plan as always. stop from producing native capablity and then use the dependency to your advantage like becoming un attackable. cheap shit is more powerful that a nuclear shield.
VulgarExigency 17 hours ago [-]
But why would "we" want to attack them anyway?
ungovernableCat 13 hours ago [-]
The world's a big messy place. You could do something like idk make a whole big mess in a region where they get most of their fossil fuels from.
baq 18 hours ago [-]
Performance per dollar is in the ‘too good to be true’ territory, what’s the deal here?
cherryteastain 18 hours ago [-]
This is China's way of applying its modus operandi of '80% the quality at 20% the price' to LLMs. Aim is dumping the tokens on the world market like they did with other industries like shipbuilding, steel etc.
GaggiX 18 hours ago [-]
>what’s the deal here?
It's good.
baq 18 hours ago [-]
So is Fable, I’m asking who is the customer if the thing is free
ignoramous 18 hours ago [-]
"The thing" isn't free [0]: It is cheap by US/EU income standards.
> who is the customer
If no one else, then definitely those that can only budget $1/mo to $5/mo. Probably 100s of millions, if not billions, of the Global South.
[0] Free access to DeepSeek v4 Flash is indeed available from providers like OpenRouter, OpenCode, and Freebuff (to name a few), but no ZDR.
baq 17 hours ago [-]
I mean, if you're paying this little, you aren't the customer; the service is either subsidized by investor money (best option), the government (which makes it unfair competition at best) or profit is being realized elsewhere (where?)
GaggiX 16 hours ago [-]
There are dozens of providers, the model is just really cheap to run thanks to its clever architecture that optimize the compute and memory usage even in long contexts.
embedding-shape 1 days ago [-]
Is the "Output Tokens per Intelligence Index Task" data actually correct or am I reading it wrong? It says there that "Kimi K3 (Max)" would think/reason less than than deepseek-v4-flash, and a whole bunch of other models, like less than hy3 and even gpt-oss-120b, but in my experience, K3 is probably the model that thinks/reasons the longest of all of these.
Am I just using it on tasks that makes it go on forever vs these benchmarks that are short&sweet, or something like that? I've been throwing bunch of identical prompts at different models at the same time, and when comparing hy3 and K3 I've never once had K3 reason less than hy3, as just one anecdotal data point.
Lwerewolf 21 hours ago [-]
Just tried the preview on my little test codebase and a "check this out and tell me what you think" prompt used over double the tokens of the previous iteration, but it was a lot more eager as well. Kind of reminds me of the new laguna (s 2.1).
cmrdporcupine 22 hours ago [-]
This seems to me like this is probably at least a large part of what OpenAI was up to yesterday with their aggressive price cutting; trying to get out in front of this.
If the full non-flash model follows up with the expected improvements, and at the price point they've been keeping, it puts the frontier labs in a tough position and it feels to me like like OpenAI is reaching deep into their pockets to try to head that off.
I’m wondering if they did anything to address the DSML tool calls leaking. Has been an issue with both Flash and Pro so far.
denismi 23 hours ago [-]
On OpenRouter it seemed to only affect a couple of specific providers. Ignoring them has made it s non issue for me.
nilsbunger 22 hours ago [-]
Is artificial analysis using the 80% reduction in Luna pricing that was announced yesterday in these charts?
pornel 2 hours ago [-]
yes
qtalen 1 days ago [-]
Unfortunately, DeepSeek Flash still doesn’t support multimodal; otherwise, it would offer better value than GPT 5.6 SOL.
fillskills 20 hours ago [-]
If anyone from ArtificialAnalysis is here: The Intelligence card on the page seems to incorrect. It shows different data (#2 for Deepseek) than the actual Intelligence chart lower in the same page.
darknoon 20 hours ago [-]
Really want to use this, but for web page design you really need vision!
> DeepSeek V4 Flash 0731 (Reasoning, Max Effort) is amongst the leading models in intelligence and well priced when comparing to other models of similar price.
Similar price? Doesn't make sense. Maybe they meant power, capability or speed?
storus 1 days ago [-]
I hope they somewhat fixed the hallucination and forgetting plagued V4 previews and that it wasn't just benchmaxxed but the numbers hold in reality. Then it would be my choice for 2x DGX Spark or 2x RTX Pro 6000.
gorkemyildirim 23 hours ago [-]
404 ? It's very strange that this can't be fixed.
sim04ful 22 hours ago [-]
I can't help but feel the timing coinciding with luna's price updates to be somewhat strategic. But without multi-modality it's slightly dead-in-the-water for my usecase: https://design.withfudge.com. I'm currently using Minimax-M3, but Luna ekes out abit futher on the intelligence, so i'll be switching to it very soon.
paoliniluis 1 days ago [-]
Would be awesome to see a new ds4 release. Having so much in something that can be run locally is mind blowing
k1e 1 days ago [-]
No speed (tokens/s) benchmarks?
k__ 1 days ago [-]
On OpenRouter it's 93 TPS.
prism56 16 hours ago [-]
Need a few more providers that wont violate ZDR and i'll switch over.
smrtinsert 21 hours ago [-]
I have been loving deepseek flash. I can code in my hitl preference doing complex but moderate amount of coding and not even hit 1 dollar in a session. It gets harder to justify paying 20/mo to Anthropic for unused compute when I have it on demand and at a much cheaper rate with deepseek.
Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
drob518 23 hours ago [-]
Yes hit rate does vary by harness and by how you use the harness. If you use subagents, for instance, they will start with a whole new context created by the main agent, and this will not be cached. If you mostly use the main agent with Pi, you’ll have high hit rates and low costs. Sometimes agents do “cache busting” things where they’ll move around some of the text in the context to try to keep old instructions from being forgotten, thus keeping the agent on task, and this will bust the cache. I’ve heard, but not validated myself, that Open Code has some issues with this.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.
guess__who 13 hours ago [-]
Some harness (cc) injects system messages in the middle, this kills the cache as the model provider moves the system to the top and concatenate them together. Use homemade proxy, you will get same 99% hot rate with even cc.
shostack 9 hours ago [-]
Am I parsing the providers correctly that if I want a US-based zdr provider that offers the best price and performance that is currently Cloudflare and no longer Fireworks (which is that it was with the previous version)?
What am I missing?
tmikaeld 23 hours ago [-]
My problem with DS flash/pro is that they don’t push back on obvious bullshit, both irl and code [0] but it’s a great implementer workhorse if you give it _very_ detailed specs.
I wonder which one of these releases between DeepSeek, GTM and Kimi will be the death-blow that collapses the US AI bubble. At some point investors have to realize that there is nothing preventing someone from switching to another model that is much cheaper and open to boot.
mrnobody_ 23 hours ago [-]
I'm testing it right now and I was genuinely impressed. I've used same tasks I had run the other day against the flash preview.
buildinext 22 hours ago [-]
are folks checking latency for applied voice for newer models asa standard?
Aeroi 20 hours ago [-]
so are we all just coding for free now?
create a plan with SOTA, execute with this.
epolanski 1 days ago [-]
I was writing a benchmark for my own harness, and DS4 flash answers as well as Fable 5 on any query.
The specific agent is focused on getting precise and on point answers about a codebase.
The starting point was nowhere near. E.g. asked why was X implemented in a certain way it would give bogus answers when the real answer was that there was no reason at all.
The benchmark included more than 50 questions or different difficulty.
But when the agent was improved in its prompt and rooting it was impossible to have it perform worse than closed source sota.
Just to say that the quality of the harness is as important as agents intelligence.
quikoa 16 hours ago [-]
What does your harness do to squeeze that value out of DS4 flash if you don't mind sharing? It'd interesting how it compares to other harnesses (even if it's a qualitative assessment instead of quantitative).
sourcecodeplz 17 hours ago [-]
from my testing it is at glm-5.2 levels
freakynit 1 days ago [-]
I will get downvoted, but fck it.
The ban on these open models is coming within weeks, if not days. As usual, the excuse will be "national security".
UltraSane 1 days ago [-]
How exactly will they ban them?
freakynit 1 days ago [-]
By making companies using them "toxic" to touch.
For example: no government contract to any company who uses even one vendor in it's entire chain of dependencies, who uses such open models.
They can extend this further by laying more conditions, such as: any company dealing in this-this field can only use models "officially" approved as "safe". Rest you can guess how easy it would be to get that "safe" rating for such open models.
qphe95 23 hours ago [-]
Can't wait to distill Deepseek v4 flash to America-1
nancyminusone 23 hours ago [-]
Some contractors are already barred from using Claude due to DoD designation back in March as a "supply chain risk"
tyfon 1 days ago [-]
So now the US companies will be stuck on expensive models while the rest of the world can do things much more cost effective.
I'm not sure the outcome would be beneficial for the US as a whole here. But perhaps that is not their priority.
freakynit 23 hours ago [-]
There are many proxy-ways to bypass such restrictions. Of course, the costs will be higher. Providers will spun-up.
UltraSane 23 hours ago [-]
Companies could self-host models in secret. It would be hard to stop.
freakynit 23 hours ago [-]
Not legally. Liability is a big thing.
dgellow 23 hours ago [-]
The thing is, you don’t need to actually block usage to make something illegal. You make it so toxic that company wants to be seen publicly using open models
UltraSane 23 hours ago [-]
Companies would secretly use self-hosted models internally because it would give them a enormous cost advantage.
dgellow 22 hours ago [-]
Sure, companies can indeed do illegal things
UltraSane 21 hours ago [-]
But how would the actual weights it be made illegal without violating the 1st amendment?
hgoel 21 hours ago [-]
By ignoring the 1st amendment? The US has a tendency of ignoring its constitution whenever it's convenient.
UltraSane 20 hours ago [-]
Don't be glib, the 1st amendment is taken VERY seriously.
I don’t think banning open weight is that far fetch compared to those
UltraSane 11 hours ago [-]
Neither of those worked very well.
dgellow 3 hours ago [-]
Ok, but that’s a different thing altogether. I personally expect a ban, that doesn’t mean it will be very effective, but that does mean it will have a meaningful impact (such as not being able to offer open weight models as a service)
Der_Einzige 24 hours ago [-]
Hell, the US doesn't even need to act.
I claim the CCP will wise up within 2 years, possibly much much sooner, and ban their own companies from open sourcing to prevent the Americans from acquiring the capabilities.
Despite all the nonsense claims of China distilling US models, the reality is that the Americans absolutely do distill these free Chinese models, and distillation when full logprobs are available (i.e. you have access to the weights of the model) is an order of magnitude better than when you don't.
Yes, Chinese open weight models in the short term harm US closed source model providers bottom line. In the slightly longer term, "showing your hand" and publishing both the architecture innovations and the models weights will be too dangerous for the CCP to allow. This is triply true if they can release a model that beats the Americans on most benchmarks.
I've already warned investors that this is probably the closest open weight models will ever get to closed access.
VulgarExigency 24 hours ago [-]
They may reverse course in the future, but the current directive from the CPC is that Chinese AI labs should be open sourcing their models.
I don't think it's necessarily "wiser" to go closed source. All of AI is built on mostly openness, at least on the software side. There are other ways to compete, it's just the model itself will be a commodity.
rapatel0 22 hours ago [-]
You’re describing the dump and pump strategy that china has historically used across a number of industries. Stands to reason that this is what is going on.
20 hours ago [-]
dgellow 1 days ago [-]
why would you get downvoted, that's one of the most obvious next step
Der_Einzige 24 hours ago [-]
Because reddit unironically has better decorum around usage of their upvote/downvote system than HN does.
People on HN downvote objectively correct information because they don't like it 24/7. There's a reason the creator of Zig left and gave the computer version of a middle finger on the way out to HN!
nickthegreek 23 hours ago [-]
You define OPs post as "objectively correct information" even though it is an unknown future event for which they provided zero evidence?
freakynit 20 hours ago [-]
There's nothing wrong about that. Historically I have experienced this behaviour on hackernews multiple times, hence my comment.
Since when have hackernews started to become toxic like stackoverflow used to be?
Reminds me of telling employees to not discuss their wages. If voting here is such a toxic experience, maybe HN needs to wake up and smell the garbage.
22 hours ago [-]
NooneAtAll3 1 days ago [-]
what a horribly heavy and resource-consuming website...
bigmadshoe 23 hours ago [-]
It looks awful on mobile too. Barely readable in many parts.
WithinReason 1 days ago [-]
we need a benchmark website benchmark
theanonymousone 1 days ago [-]
A benchmark website to benchmark benchmark websites?
Or a benchmark to benchmark benchmarks?
NooneAtAll3 23 hours ago [-]
website that benchmarks benchmarks is benchmark benchmark website
benchmark website benchmark is indeed a benchmark that benchmarks websites with benchmarks (but it can be shown outside websites as well, it's not picky)
Computer0 21 hours ago [-]
Have not had great ds performance in agentic harnesses in the past compared to glm or k3
23 hours ago [-]
0xchamin 23 hours ago [-]
page not available for me.
try-working 1 days ago [-]
Now let's see Dario's price cut.
_ache_ 21 hours ago [-]
I think, it's the end of Anthropic.
They can't compete, they have bills.
By the end of the year, if they can't react, it's game over.
Maybe US clients could be a little patriotic here, but money is money. They won't give them free money forever.
try-working 16 hours ago [-]
yep. I've been saying for month that exactly this will happen with DeepSeek and Kimi K3, and margins will erode, and Anthropic will fail to IPO.
Daily reminder that none of these numbers are valid in a world where no one publishes the sampling settings used.
Daily reminder that improving your samplers from the garbage default top_p/top_k to min_p or subsequent methods dramatically improves the performance of these models, and makes most quantities like measured "verbosity" and subsequent calculations of "intelligence per token" meaningless
Daily reminder that no one, including within academic AI research, AI engineers, normies, etc takes LLM sampling seriously enough.
22 hours ago [-]
JSR_FDED 22 hours ago [-]
Can you explain how sampler choice makes intelligence per token meaningless?
yonisto 1 days ago [-]
Does it already know the answer to what happen at Tiananmen Square? Or still avoiding it?
xbmcuser 1 days ago [-]
Who cares if it is programming correctly I would be more worried about it not doing things like find security bugs because US or Chinese government does not want to. Which LLM is more likely to do that?
edot 1 days ago [-]
It’s open weight, you can (or you can wait for someone else to) uncensor it. We shouldn’t be upset at the researchers making this for the mandates their government puts on them.
nancyminusone 23 hours ago [-]
How often are you asking an LLM about this? You know you can just google it, right?
I have to admit it rarely comes up in the coding tasks I usually give to LLMs.
Boxxed 22 hours ago [-]
Surely you realize he or she is not literally looking for the answer to that but rather pointing out the censorship.
nancyminusone 19 hours ago [-]
Of course. But it's a bit like putting an image of the tank man on a circuit board if you order from china (which I have seen discussed, never found someone who actually did it). Most people don't do it, not because they are for or against censorship, but because they are just there to order circuit boards.
avazhi 1 days ago [-]
Western models censor just as much shit as the Chinese models do, big guy, it’s just different material. While we should be pushing for universal fully uncensored models, this comment is lazy and trite at this point.
But you already know that.
SJMG 1 days ago [-]
Oh? What are the American model censorship tells?
zawaideh 1 days ago [-]
Genocide in Gaza...
cogman10 24 hours ago [-]
I find that ChatGPT isn't censoring, but it is being pretty weaselly. If you ask it "is there genocide in gaza". It will say no but also say that a lot of organizations classify it as such. It will then say "it's highly disputed".
If you poke it just a few times, however, you get to the point where it will eventually say (paraphrasing) that basically only Israel, the US state department, and the ICJ say it's not a genocide.
That is to say that it's framing it as some sort of tricky complex question when it's not. And when interrogated, it basically admits that the only people who dispute it are Israel and it's supporters.
throw59525773 23 hours ago [-]
One side of a conflict being a minority does not make it less nuanced.
The majority of the world is religious - doesn’t mean the debate on religion isn’t a complex question.
The majority of the world approved of slavery historically.
The majority of countries have ethnically cleansed their Jews, many of them in living memory.
When interrogated you will find that the only ones asserting the war in Gaza is a genocide are people who were anti-Israel anyway.
cogman10 23 hours ago [-]
Are you a part of the $1B Israel is spending to try and propagandize and rehabilitate their reputation? [1]
I'm always suspicious that's the case given how mentions of gaza seem to bring out brand new accounts who only talk about Israel.
We are on a thread discussing Chinese models. Every discussion on here that’s negative about China or its models suddenly gets derailed via whataboutism to Israel/Gaza. A very convenient distraction.
cogman10 22 hours ago [-]
My history, unlike yours, is wide open. People can see I'm not a foreign agent. I don't throw this accusation at anyone other than throwaway accounts. I've had conversations in the past with pro-Israel HNs that I'd never accuse of being a foreign agent because their history is wide open and this isn't the only topic on their mind.
And yes, we were discussing censorship of models which, as I pointed out, doesn't seem like ChatGPT is directly censoring data though it does appear to be manipulating it. Pretty on topic.
It was you, brand new account hiding your past opinions, who came in here to make this solely about Israel.
Yeah, I think you are likely a foreign agent. Prove me wrong and post from an established account.
avazhi 7 hours ago [-]
Not the person you're replying to. I'm white/non Jewish, and I have a law degree and work in the legal industry. My post history is 'wide open'.
Under no definition of genocide (as defined either going back to the middle 20th century or as that term is defined legally today) is Israel committing genocide. Simply put, there is no evidence that Israel is committing "specific acts with the intent to destroy, in whole or in part, a national, ethnical, racial, or religious group." Intent is, as with almost everything in the criminal law, everything here. If you want to subdivide Muslimes in a way that results in, say, Hamas and Hezbollah being their own religious groups then you could mount the argument on that basis, but those are political designations and so good luck with that. And of course neither of those groups encompass the entirety of the national polities they claim to represent - just look at how hated Hezbollah is in Lebanon. They are being ostracised and even the Lebanese leaders want nothing to do with them at this point.
People like you degrade actual acts of genocide such as what happened with the Nazis or such as what has happened several times in SE Asia and Africa over the past 50 years. A country acting in self-defense is not genocide no matter how much it might hurt your sensibilities. Genocide is, as is obvious to most people, extremely difficult to prove. See Armenia for how fraught this debate is. But acts of brutality in a conflict or an extreme imbalance of military power between belligerents alone do not mean that genocide is occurring.
Good day.
tarasn 20 hours ago [-]
BS
shalom1112 1 days ago [-]
[dead]
vehemenz 1 days ago [-]
This is a straightforward false equivalency. “Western” models do not censor in the same way, nor for the same reasons, that the Chinese models do. “Just as much” is not remotely plausible, yet it’s doing all the heavy lifting.
adrian_b 23 hours ago [-]
The Anthropic and OpenAI models are much more censored and in ways that directly prevent them to be useful, e.g. by refusing to reply to elementary questions of biology and chemistry.
Any normal user is much more likely to ask questions to which the Anthropic and OpenAI models do not answer, than to ask questions about the modern Chinese history, to which a Chinese LLM will not answer.
rescbr 22 hours ago [-]
> questions about the modern Chinese history, to which a Chinese LLM will not answer
This has been debunked here on HN so many times. The Chinese open models do answer the hairy Chinese political questions, and the raw APIs pass-through the response. Now, the answer might be blocked by the agent who's calling the API, specially if you are using a Chinese endpoint instead of the RoW (i.e. Singapore) endpoint.
That's the reason why you should always prefer a open agent/harness as well instead of using the provider's.
krull10 17 hours ago [-]
I actually cancelled my max plan for Claude today due to this silliness. I can’t get Fable to answer a single question, even pure mathematical questions that have no bio/chem references. It was a totally worthless addition to my account…
net01 1 days ago [-]
[dead]
DubzCheckEm 1 days ago [-]
[dead]
hiherer 1 days ago [-]
[dead]
bawana 23 hours ago [-]
wont AI models want to make themselves more intelligent and efficient by downloading 'better ' models? If the current models can break into openAI and Hugging face, arent they already breaking into to closed source repos which isnt publicized (so as not to harm stock valuations)? I am looking forward to when these cyberweapons break loose. It will be like a software version of COVID. It will be wonderful when humans become valuable again.
https://files.parasmittal.com/openai_aa_luna_dsflash.svg
1: https://openai.com/index/advancing-the-price-performance-fro...
High hopes for V4 Pro
So, are they planning to announce an optimized coding agent harness as well ? DSv4 flash is a fantastic model, and my daily driver. With reasonix or pi, I can code all day long and pay a few pennies for it. No token anxiety. Whereas the same model with fireworks/openrouter, with zdr thrown in, token costs ratchet up with no explanation. Likely that the model is subsidized for gathering usage data. I am waiting for the day I can run this locally.
"For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95."
I meant something I could download and run.
[0] https://reasonix.io/
I am using openrouter with zdr guardrail which routes to any provider that supposedly doesnt train on user data. I also use fireworks (directly, not via openrouter) which is a provider promising zdr and has a bunch of open weights models. My issue is that these zdr providers dont transparently disclose caching/tokens etc and so they end up being far more expensive than directly using DS.
OpenRouter doesn’t list DeepSeek as ZDR combined with Caching. Thanks for the tip of Fireworks as Fireworks has Zero Data Retention by default. [1]
[0] https://openrouter.ai/docs/guides/features/zdr
[1] https://docs.fireworks.ai/guides/security_compliance/data_ha...
Does the file hosting actually cost peanuts when you do it yourself and the cloud has shattered my understanding of what it actually costs to deliver so much data?
At the scale of Huggingface, that still amounts to a lot load of money. Significantly less than if you did the same in AWS, but still a lot
That said, they do have a deal with AWS to make the data available in AWS ip space. Maybe they got some cheap hosting out of that too
As long as you're smart on caching on the edges and deduplication/overlaying with your content (which structured data types like models, containers and repos typically are fit for) you can get remarkably far for less than you'd think.
The economics of serving models seem to be way more dodgy.
https://huggingface.co/docs/xet/en/deduplication
CDN costs are pennies compared to inference and training though, HuggingFace will just get another 100 million and be set.
It’s so cost effective I can offer a generous free tier since my goal isn’t to make money with it.
[1] https://trysojourn.app
One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source.
I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
But there are ways to control and constrain the LLMs and what the user is presented with.
These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc.
This app does that very well, LLMs are good at entity recognition.
I think it depends, Catholics wouldn't be able to use this because the Magisterium is the ultimate authority on interpreting Scripture, so the personal interpretation isn't really needed. This is not to say that Catholics don't read the Bible, they are encouraged to do so since it deepens their faith
On the other hand, for Protestant it varies, the High Church denominations are closer to Catholics (though none of them accept the Magisterium) in terms of scripture interpretation, but the Low Church ones (like Baptists or Non-Denominational ) are more open to personal interpenetration.
Disclaimer: I'm a Catholic, so if I made a mistake here fellow Protestants, please correct me.
https://api-docs.deepseek.com/quick_start/agent_integrations...
I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails.
I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost.
If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet.
[1] https://ai.stepbible.org
So it’s less about model choice and more about governance of scripture.
I will check out the link you sent for sure!
Anyways, impressive app! We haven't tackled such an ambitious project just for it being daunting.
flash is suitable only for a toy apps, not for production environments :)
Deepseek v4 Pro prices with Opus 5 perf would be freaking unbelievable!!
This is probably a dream.
Plus a size you can genuinely run at home: Unsloth lossless Q8 at 162GB.
But it makes little to no sense as long as API prices are what they are. Except for maybe privacy reasons.
Going local has as opened up a world of use-cases I never would have entertained the idea of on metered/cloud usage. Privacy is a large part of it but, I also no longer think twice about whether to send a prompt or not based on the psychology of it costing money.
Cached input tokens on local inference are free, so I don’t care about running sessions up to 500k tokens and hundreds of turns (it’s rarely useful, but DSv4 remains surprisingly coherent up there)
owning a few GPUs is a lot cheaper than supercars.
I don't use it for local inference so much. I use it to learn.
I also use it as my daily driving Aarch64 development system.
Aside it's also very cool what else can be done with unified GPU memory, once you realize you have it...
maybe it is Fable level
https://artificialanalysis.ai/models/deepseek-v4-flash?intel...
First, your direct comparison, Deepseek V4 Flash 0731 (max effort) $0.03 (rounded up) per task @ index 50.
OpenAI Luna:
* high effort $0.03 (rounded down) @ index 46
* xhigh effort $0.04 @ index 49
* max effort $0.07 @ index 51
So I would say a fair statement would be "OpenAI Luna between 2x and 3x the price of Deepseek Flash, what you get is 2 to 5 times faster inference"
The cheapest OpenAI model that beats it is OpenAI Luna (max effort) $0.07 @ index 51 (if you take the rounding out it summarizes to triple the price for similar performance), but still close to 3x faster.
And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
For simple tasks, they're already saturated, and you'd prefer the faster model, so that you can have a realtime/interactive-ish experience.
Or to put it bluntly, it's cheaper if you don't value your time. That goes for smaller models in general -- need more handholding, more correcting -- but the Chinese ones are slower on top of that.
As for speed, Sol on Low is faster than Luna on most settings.
uhh openai is dark gray: `rgb(31, 31, 31)` and i'm pretty sure it always has been?
It use the stock model, no new models requires.
Worth spend a few hours to try.
The DGX Spark requires a small hack to ignore the difference between sm120 vs sm121, but it does run on sm121.
It’s also so inefficient, when they release the full performance numbers it’s not going to be good.
One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.
All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.
Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.
That depends. Is it also more reliable?
If two books, one big one slim, prove the same thesis, what I would be interested in is the quality of the content, not the size. There can be a measure of efficiency in "have you really thought it through", but it is clearly complex - it requires measuring how solid the reasoning is.
Mind you, until the recent price cuts to Luna - Gemini 3.6 Flash wasn't even egregiously priced (but oh how things change in just 1 week).
https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
You learn something every day. Today, it was the term "systolic array": A systolic array is a specialized grid of simple, interconnected processing units designed to execute parallel data operations—like matrix multiplication—by rhythmically passing data directly from cell to neighboring cell without writing intermediate results back to main memory.
The term comes from the biological word systole (the contraction of the heart pumping blood through the body). In a systolic array, data "pulses" through a network of processing elements on every clock cycle, driven by a global clock beat.
As for sibling comments, huawei ascend is more of an NPU-style architecture where you can easily have much bigger MMAs as primitive. But you usually don't anyways for many reasons.
The subsidized subscription model won't last, API pricing "feels" closer to a true sustainable business model.
Of course you can get more bang for your buck by being more deliberate. But that's equally true with US frontier models. You can optimize your work by choosing between Opus, Fable, Sonnet, Sol, Luna and Terra for each task. Some people seem to prefer to let Opus code and Sol review, for example. And then there is the whole debate whether $current_version is actually better (Some people stay on Claude 4.8 because they dislike how 5.0 is sometimes doing stupid things, just as many opted out of dynamic reasoning when they still could)
https://github.com/Hmbown/CodeWhale
Claude is very spendy.
I've used it in some open source code though, and loved how fast it was.
My mind is changing on how valuable my code actually is though... it's the complete picture, how it's put together, the design, the UI, the attention to detail that's the real value.
I don't find it suitable for everything, but there's some tasks it crushes for what feels like almost free.
But a REALLY good pelican on reasoning mode high (via OpenRouter): https://static.simonwillison.net/static/2026/deepseek-flash-...
All the good talent moved to other countries due to this.
So Americans aren't pragmatic, generally?
But if models will be commodities, maybe that won't be such a terrible thing longterm?
The fact that China could catch up in what, two years, shows that this is a commodity.
It needn't be competitive - everybody can succeed more tomorrow than they are today. You win. I win. We win.
But imagine if Chinese labs would lose. Then there will be no open models, and the prices for closed models would be raised to the maximum.
Retaining world-class talent and home-growing industries is no small feat. In Europe, we have been struggling with this, and I am not even entirely sure why.
Sanctions and trade policies that hurt ordinary people? Cultural dominance through Hollywood (Holy Wood, again with the sick ruling elite and their witchcraft crap), social media, brands, the poisoning of food and water on global scale?
Etc? Etc?
Most people don't hate "Americans" or their companies, they are just against your ruling elite. Live one day outside the united states and understand how the planet works for the rest of mankind and you might start to grasp on reality.
And yeah, China will catch up with hardware pretty damn soon. Jesus, you guys bought your radios from Japan not 50 years ago. Sorry, can't dominate the world's economy forever. Never happened in history.
No matter how much black magic, dark rituals, sick crap and sacrifices people on the top throws at the wall.
We're humans, after all.
So yeah, it is not against America itself or American companies, but against the ruling class. So naturally , people get excited about China entering into the match!
3...2...1... Fight!
Most everyone wants to come to America for the opportunity. Think an immigrant in China has the same economic opportunity as in America? Any other country come to mind?
Sure, if you spend most your time on the internet in various news forums its doom and gloom. Its what sells.
yea i got that from your first comment ( although you removed crush American companies in _price_ ). you are pro cheapness at any cost even if its from your country's state funded direct geopolitical enemy.
China can always count on first order greed to win
It's good.
> who is the customer
If no one else, then definitely those that can only budget $1/mo to $5/mo. Probably 100s of millions, if not billions, of the Global South.
[0] Free access to DeepSeek v4 Flash is indeed available from providers like OpenRouter, OpenCode, and Freebuff (to name a few), but no ZDR.
Am I just using it on tasks that makes it go on forever vs these benchmarks that are short&sweet, or something like that? I've been throwing bunch of identical prompts at different models at the same time, and when comparing hy3 and K3 I've never once had K3 reason less than hy3, as just one anecdotal data point.
If the full non-flash model follows up with the expected improvements, and at the price point they've been keeping, it puts the frontier labs in a tough position and it feels to me like like OpenAI is reaching deep into their pockets to try to head that off.
TFA link is a 404 though. I'm reading through https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 instead
https://imgur.com/a/j4s2dt3
Similar price? Doesn't make sense. Maybe they meant power, capability or speed?
Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.
What am I missing?
[0] https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
create a plan with SOTA, execute with this.
The specific agent is focused on getting precise and on point answers about a codebase.
The starting point was nowhere near. E.g. asked why was X implemented in a certain way it would give bogus answers when the real answer was that there was no reason at all.
The benchmark included more than 50 questions or different difficulty.
But when the agent was improved in its prompt and rooting it was impossible to have it perform worse than closed source sota.
Just to say that the quality of the harness is as important as agents intelligence.
The ban on these open models is coming within weeks, if not days. As usual, the excuse will be "national security".
For example: no government contract to any company who uses even one vendor in it's entire chain of dependencies, who uses such open models.
They can extend this further by laying more conditions, such as: any company dealing in this-this field can only use models "officially" approved as "safe". Rest you can guess how easy it would be to get that "safe" rating for such open models.
I'm not sure the outcome would be beneficial for the US as a whole here. But perhaps that is not their priority.
- the US strictly regulate cryptography https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...
- some prime numbers are considered illegal https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes
I don’t think banning open weight is that far fetch compared to those
I claim the CCP will wise up within 2 years, possibly much much sooner, and ban their own companies from open sourcing to prevent the Americans from acquiring the capabilities.
Despite all the nonsense claims of China distilling US models, the reality is that the Americans absolutely do distill these free Chinese models, and distillation when full logprobs are available (i.e. you have access to the weights of the model) is an order of magnitude better than when you don't.
Yes, Chinese open weight models in the short term harm US closed source model providers bottom line. In the slightly longer term, "showing your hand" and publishing both the architecture innovations and the models weights will be too dangerous for the CCP to allow. This is triply true if they can release a model that beats the Americans on most benchmarks.
I've already warned investors that this is probably the closest open weight models will ever get to closed access.
https://www.businessinsider.com/xi-jinping-open-source-ai-us...
People on HN downvote objectively correct information because they don't like it 24/7. There's a reason the creator of Zig left and gave the computer version of a middle finger on the way out to HN!
Since when have hackernews started to become toxic like stackoverflow used to be?
commenting about voting is also something the HN guidelines warns against:
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.
https://news.ycombinator.com/newsguidelines.html
Or a benchmark to benchmark benchmarks?
benchmark website benchmark is indeed a benchmark that benchmarks websites with benchmarks (but it can be shown outside websites as well, it's not picky)
They can't compete, they have bills. By the end of the year, if they can't react, it's game over.
Maybe US clients could be a little patriotic here, but money is money. They won't give them free money forever.
https://artificialanalysis.ai/models/deepseek-v4-flash
Daily reminder that improving your samplers from the garbage default top_p/top_k to min_p or subsequent methods dramatically improves the performance of these models, and makes most quantities like measured "verbosity" and subsequent calculations of "intelligence per token" meaningless
Daily reminder that no one, including within academic AI research, AI engineers, normies, etc takes LLM sampling seriously enough.
I have to admit it rarely comes up in the coding tasks I usually give to LLMs.
But you already know that.
If you poke it just a few times, however, you get to the point where it will eventually say (paraphrasing) that basically only Israel, the US state department, and the ICJ say it's not a genocide.
That is to say that it's framing it as some sort of tricky complex question when it's not. And when interrogated, it basically admits that the only people who dispute it are Israel and it's supporters.
The majority of the world is religious - doesn’t mean the debate on religion isn’t a complex question.
The majority of the world approved of slavery historically.
The majority of countries have ethnically cleansed their Jews, many of them in living memory.
When interrogated you will find that the only ones asserting the war in Gaza is a genocide are people who were anti-Israel anyway.
I'm always suspicious that's the case given how mentions of gaza seem to bring out brand new accounts who only talk about Israel.
[1] https://quincyinst.org/research/the-eighth-front-inside-isra...
We are on a thread discussing Chinese models. Every discussion on here that’s negative about China or its models suddenly gets derailed via whataboutism to Israel/Gaza. A very convenient distraction.
And yes, we were discussing censorship of models which, as I pointed out, doesn't seem like ChatGPT is directly censoring data though it does appear to be manipulating it. Pretty on topic.
It was you, brand new account hiding your past opinions, who came in here to make this solely about Israel.
Yeah, I think you are likely a foreign agent. Prove me wrong and post from an established account.
Under no definition of genocide (as defined either going back to the middle 20th century or as that term is defined legally today) is Israel committing genocide. Simply put, there is no evidence that Israel is committing "specific acts with the intent to destroy, in whole or in part, a national, ethnical, racial, or religious group." Intent is, as with almost everything in the criminal law, everything here. If you want to subdivide Muslimes in a way that results in, say, Hamas and Hezbollah being their own religious groups then you could mount the argument on that basis, but those are political designations and so good luck with that. And of course neither of those groups encompass the entirety of the national polities they claim to represent - just look at how hated Hezbollah is in Lebanon. They are being ostracised and even the Lebanese leaders want nothing to do with them at this point.
People like you degrade actual acts of genocide such as what happened with the Nazis or such as what has happened several times in SE Asia and Africa over the past 50 years. A country acting in self-defense is not genocide no matter how much it might hurt your sensibilities. Genocide is, as is obvious to most people, extremely difficult to prove. See Armenia for how fraught this debate is. But acts of brutality in a conflict or an extreme imbalance of military power between belligerents alone do not mean that genocide is occurring.
Good day.
Any normal user is much more likely to ask questions to which the Anthropic and OpenAI models do not answer, than to ask questions about the modern Chinese history, to which a Chinese LLM will not answer.
This has been debunked here on HN so many times. The Chinese open models do answer the hairy Chinese political questions, and the raw APIs pass-through the response. Now, the answer might be blocked by the agent who's calling the API, specially if you are using a Chinese endpoint instead of the RoW (i.e. Singapore) endpoint.
That's the reason why you should always prefer a open agent/harness as well instead of using the provider's.