CIO View: The Real Price Of A Token

DeepSeek just published its cost sheet

On October 20, DeepSeek open sourced DeepInfer, the software that runs its own AI service. Near the top of the release notes sat two numbers the big Western labs do not publish. One cluster serves 155 trillion tokens a day. Each GPU in it brings in about $38 an hour, and that is at DeepSeek's off peak prices (DeepSeek, Dealroom).

Yuchen Jin ran the math in public. Rent Nvidia's B300 chips at about $7 an hour and the gross margin still comes out near 82% (BigGo Finance). Alexey Fateev summed it up on X when he wrote that the disclosure should make you angry.

He has a point. For two years, CIOs have been told that AI is costly to run. Budgets, contracts and business cases were built on that story. Now a company serving a popular open model at huge scale has shown its cost sheet. Cheap AI turns out to be good business, not charity.

Anger is not a plan, though. The useful question is who captures the gap between what a token costs to make and what you pay for it. The answer runs through open models and open standards, and it belongs on every CIO's agenda.

What DeepSeek actually disclosed

Start with the basic unit. A token is a small piece of a word, roughly three quarters of an English word. It is the unit on your AI meter, the way a kilowatt hour is the unit on your power bill. Inference is the work of running a trained model to answer requests. If training is building the power plant, inference is running it every day.

DeepSeek's notes describe one cluster serving its V4.1 Flash model. That cluster handles 155 trillion tokens a day and earns about $38 per GPU hour (DeepSeek). A GPU hour is one AI chip used for one hour. Think of a delivery truck rented by the hour. What matters is how much freight it hauls and how much it earns while it is on the road.

Two details matter. First, the $38 is measured at valley pricing, DeepSeek's off peak rate. Off peak is half the peak price, like night rates on electricity. At peak, V4.1 Flash costs $0.30 per million new input tokens and $1.20 per million output tokens (BigGo Finance). So the revenue figure is closer to a floor than an average.

Second, most of those 155 trillion tokens are reused input. DeepSeek keeps recent conversation data for three to seven days on a storage pool of about 100 petabytes. About 96% of input hits that cache (Dealroom). A cache hit costs one fiftieth of a fresh input token (BigGo Finance). Picture a coffee shop that keeps its most popular orders ready to pour.

These are company reported numbers for one cluster, not audited averages (Dealroom). DeepSeek also did not say which GPU earned the $38 (BigGo Finance). Treat them as a strong signal, not the final word.

A token's price is a business choice

Gross margin is what is left of each sales dollar after paying for the thing you sold. Think of a restaurant's menu price minus the cost of the food. It ignores rent, staff and the chef's years of training.

The math here is simple. At $38 of revenue per GPU hour and about $7 to rent a B300, roughly 82 cents of every dollar is left over. One outside analyst put the range at 84% to 96% using cheaper chip costs (BigGo Finance). Separate reporting from The Information put DeepSeek's API gross margin at 82.9% through July (BigGo Finance).

On these numbers, cheap AI is neither charity nor dumping. DeepSeek actually raised its API prices in August, by 50% to 1,100% depending on the model and time of day (Reuters via The Star). It priced for profit, and the margin held.

Now the honest part, gross margin is not profit. DeepSeek still lost about $106 million in the first seven months of 2026, and its overall gross margin was 44.6% (BigGo Finance). Training the next model and paying researchers costs real money, and gross margin does not count it. Every lab's price has to fund the next model, not just the power bill for this one.

That is exactly why the number matters. Western labs do not publish their serving costs (Dealroom). Until now, no CIO could split an AI bill into real value and margin. Real value includes a stronger model, support and legal cover. Now there is a public reference point so use it.

The secret is the stack, and the stack is going open

DeepSeek's edge is not exotic hardware. Its notes say the system was built around ordinary 8 GPU servers and ordinary clusters of 64 to 128 GPUs (DeepSeek). The edge is in how the work is organized. Picture a busy restaurant kitchen and how optimization are made, we'll reuse that analogy.

The model is a mixture of experts. V4.1 Flash holds 552 billion parameters but uses only 8 to 16 billion of them for each token (DeepInfra). That is like a hospital sending each patient to one specialist instead of every doctor. Reading the prompt and writing the answer run on separate machines, like prep cooks and line cooks at their own stations. Finished work is kept for days so it can be reused, like sauces made in the morning and served all night. A small draft model guesses the next few words and the big model checks them, like a junior chef plating and the head chef approving (DeepSeek).

Here is what CIOs should notice. DeepSeek released this engine, DeepInfer, under the Apache 2.0 license, which allows commercial use. The team says it built mainly on vLLM, a widely used open source engine, and borrowed ideas from others (BigGo Finance). A day later, the vLLM team reported its own engine within 6% of DeepInfer's published output speed on the same model and Nvidia hardware. It also posted step by step recipes (AlphaSignal). That result came from the vLLM team itself, not a neutral lab, so treat it as strong but unaudited.

The playbook is public but running it well is still where the complexity lies. DeepSeek has not fully released its cache and load balancing code, which it says is tied to internal systems. Its smallest efficient setup is 64 GPUs for prompts plus 64 GPUs for answers (DeepSeek). A GPU only earns its keep when it is busy, the way a taxi only earns with a passenger aboard. Few agencies or enterprises will keep 128 GPUs that busy. They do not need to as many hosting companies can, and they compete to sell the same model.

Open weights plus open standards turn one price into a market

Open weights mean the trained model is published for anyone to run. Think of a drug whose formula is public. Once generic makers can produce it, the price drifts toward the cost of making it. V4.1 Flash ships under the MIT license, and 19 providers serve it on OpenRouter. Blended prices across hosts vary by as much as 4.9 times (DeepInfra). That is what competition looks like. A closed model has one maker and one price list.

Open weights are only half of the lever. The other half is the plug. Open standards are agreed shapes that let parts from different makers fit together, like the shipping container or the wall outlet. AI now has a few. DeepSeek also released the layer that turns requests into tokens. It reads the Chat Completions and Responses formats OpenAI made popular, plus the Messages format Anthropic uses. Any host that runs it behaves like DeepSeek's own API (DeepSeek). For tools and data, the Model Context Protocol, created by Anthropic, now sits with the Linux Foundation's Agentic AI Foundation. ChatGPT, Gemini, Copilot and others use it, and more than 10,000 MCP servers exist (eeNews Europe).

[embedded content: Locked in vs open · one vendor or many hosts]

With one vendor, the vendor sets the price. With open weights behind a standard gateway, hosts compete for your traffic and the price moves toward cost.

CIOs have seen the other picture. After Broadcom bought VMware, an analyst at Info-Tech Research Group put typical renewal increases at 2 to 12 times (CIO Dive). AT&T said it was quoted a 1,050% rise (The Register). The product did not get ten times better. What changed was how hard it was to leave. As Forrester's Naveen Chhabra told CIO Dive, "If migration was so easy, Broadcom wouldn't have invested $61 billion in VMware." AI is young enough to choose the open path before the lock sets.

Cost per token is not cost per task

Price per token is like price per gallon. What you actually pay for is cost per mile. A cheap model that rambles, retries or fails can cost more per finished job than a pricier one that gets it right the first time.

NIST's Center for AI Standards and Innovation has shown this in both directions. In 2025 it found that one U.S. model cost 35% less on average than the best DeepSeek model to reach similar results across 13 benchmarks (NIST). In May 2026 it found DeepSeek V4 Pro cheaper than GPT-5.4 mini on five of seven tests. Cost per task ranged from 53% lower to 41% higher. Neither model was cheaper on every kind of token. DeepSeek charged more for input and less for output (NIST). Same two models, very different answers, depending on the job.

Model habits matter too. On one standard test suite, V4.1 Flash produced about 250 million output tokens, against a median of 140 million for similar models (DeepInfra). A chatty model eats its own discount. Caching cuts the other way. Apps that reuse long instructions and documents pay the cache price, one fiftieth of the fresh price.

The lesson is simple. Judge every model by the cost of a finished, correct piece of work, measured on your own tasks.

What to do this quarter

Put a gateway between your apps and any single model. It works like a universal travel adapter. Apps speak one standard format, and the gateway sends each request to the right model and host. Switching becomes a setting, not a rewrite.

Build your own scoreboard. Pick 50 to 100 real tasks from your own work, each with a known good answer. Run every candidate model against them and track cost per finished task. Public benchmarks are a test drive on someone else's road.

Ask for the numbers, then negotiate. Ask vendors for prices by token type, cache rules and discount hours. Write price drop clauses, model swap rights and clean exit terms into every AI contract. Make sure your prompts, data and tuned models can leave with you. Epoch AI found the price of a fixed level of capability falling between 9 and 900 times a year, depending on the task (Epoch AI). Locking in today's price for years is a bad bet.

Design for caching and timing. Reuse long instructions so they hit the cache. Move batch work to discount hours. It is the same logic as running the dishwasher when power is cheap.

Run a portfolio. Send the hardest reasoning to the strongest model you can justify, closed or open. Send high volume routine work to a cheaper open model. If most of your volume is routine, most of your savings live there.

Be honest about owning hardware. Self hosting pays only with steady, heavy demand. Rent first, and own only what you can keep busy.

A note for public sector leaders

None of this is an argument to send government data to DeepSeek. On October 20, DHS published an order requiring federal civilian agencies to remove DeepSeek products and stop buying them. It also covers third party products developed using DeepSeek implementations, and it reaches federal contractors. The notice does not define what counts as a DeepSeek implementation. By February 2025, Congress, NASA, the Navy, the Pentagon and Texas had already barred DeepSeek from government devices (The Hacker News).

That undefined term matters, because open source engines borrow from each other. vLLM, for example, now includes several DeepSeek libraries (AlphaSignal). Ask counsel how the order applies to your stack before you build on it.

Security needs its own test. NIST found that agents built on DeepSeek's most secure 2025 model were 12 times more likely than U.S. frontier models to follow malicious instructions planted to hijack them (NIST). Newer models may differ, so test them yourself.

The lesson for government is leverage, not loyalty to any one vendor. Use the public cost reference in every negotiation. Favor open models whose origin passes your supply chain review, served from environments you can audit. Keep the gateway and the scoreboard so you can switch when policy or prices change. Taxpayers deserve the same price discipline we expect from any utility.

Turn the anger into leverage

DeepSeek did CIOs a favor. it put a public number on what it costs to serve a strong model at scale. Some of the gap between that number and your bill is fair payment for better models, support and research. Some of it is margin you pay because leaving is hard. Open weights and open standards are how you tell the two apart, and how you keep the difference. Be angry if you like. Then build the gateway, run the scoreboard and rewrite the contracts.

References

1. DeepSeek V4.1 Flash Inference Deployment Q&A, DeepSeek AI, https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md
2. DeepSeek open-sources its serving engine and reveals $38 of revenue per GPU-hour, Dealroom News, https://dealroom.co/news/164841-deepseek-open-sources-its-serving-engine-and-reveals-38-of-revenue-per-g/
3. DeepSeek Open-Sources Inference Framework DeepInfer, Discloses $38 Hourly Revenue per GPU, BigGo Finance, https://finance.biggo.com/news/538ddce1-7259-43f0-9763-fa6d8949e475
4. Post on DeepSeek token economics, Alexey Fateev on X, https://x.com/superalesha/status/2112653788574339075
5. DeepSeek Revenue Surges Nearly 10-Fold in First Seven Months, Still Posts Loss Exceeding CNY 700 Million, BigGo Finance citing The Information, https://finance.biggo.com/news/707aa309-2ed3-49b7-b919-ff89dfd461f5
6. DeepSeek raises API pricing for its V4 models, Xiuhao Chen and Ryan Woo for Reuters, https://www.thestar.com.my/tech/tech-news/2026/08/13/deepseek-raises-api-pricing-for-its-v4-models
7. DeepSeek V4.1 Flash API: Speed, Latency & Cost, DeepInfra, https://deepinfra.com/blog/deepseek-v4-1-flash-api-benchmarks
8. vLLM Matches DeepSeek's DeepInfer Within 6% on DeepSeek-V4.1-Flash, AlphaSignal Newsroom, https://alphasignal.ai/news/vllm-matches-deepseek-s-deepinfer-within-6-on-deepseek-v4-1-flash
9. Linux Foundation launches Agentic AI Foundation to drive open agent standards, eeNews Europe, https://www.eenewseurope.com/en/linux-foundation-launches-agentic-ai-foundation-to-drive-open-agent-standards/
10. VMware's first contentious year under Broadcom drives customers to weigh other options, Matt Ashare for CIO Dive, https://www.ciodive.com/news/broadcom-vmware-acquistion-vcf-private-cloud/733800/
11. AT&T claims VMware by Broadcom offered it a 1,050 percent price rise, Simon Sharwood for The Register, https://www.theregister.com/software/2024/10/01/att-claims-vmware-offered-it-a-1050-percent-price-rise/511969
12. CAISI Evaluation of DeepSeek AI Models Finds Shortcomings and Risks, NIST Center for AI Standards and Innovation, https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks
13. CAISI Evaluation of DeepSeek V4 Pro, NIST Center for AI Standards and Innovation, https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro
14. LLM inference prices have fallen rapidly but unequally across tasks, Ben Cottier, Ben Snodin, David Owen and Tom Adamczewski for Epoch AI, https://epoch.ai/data-insights/llm-inference-price-trends
15. U.S. Orders DeepSeek Removed From Federal Civilian Systems, Bans New Purchases, Swati Khandelwal for The Hacker News, https://thehackernews.com/2026/10/us-orders-deepseek-removed-from-federal.html

© 2026 Symvoulos LLP. You may quote or excerpt this article with attribution and link to the original URL as well as referenced material. Republication in full, adaptation, translation, or use as the basis of derivative or secondary works requires attribution. Removal or alteration of attribution is prohibited.

Stathy Touloumis is President of Symvoulos (Symv), a boutique technology consultancy serving all sectors of business and government entities including regulated enterprises. His experience spans over 30 years with most recent to state, local, and public entities across infrastructure modernization, security, service delivery strategy, and the governance questions raised by AI adoption. Symvoulos’ expertise starts where most stall: multi-year cost models, risk vectors and staffing plans an agency can defend in a budget hearing. Be sure to follow us at symv.ai.