General Compute and Positron AI announced a partnership this week to split AI inference workloads between GPUs and so-called memory-first silicon. That is the entire factual payload. No architecture diagrams, no tape-out status, no dollar figures, no delivery timeline, no named customers. Two companies, one sentence, and a press template that reads "redefining AI infrastructure."
I have audited hardware pitches since 2017, when I was reading token whitepapers for cryptographic soundness while the rest of the market was reading Telegram announcements. The pattern has not changed. The announcement is the product. The silicon is an afterthought.
So let us do the work the press release did not.
What "Splitting Inference" Actually Means
Large language model inference has two phases with fundamentally different physics, and this is not a marketing distinction โ it is a roofline constraint.
Prefill processes the entire input sequence in parallel GEMM operations. It is compute-bound. It wants FLOPS, and it wants tensor cores at saturation. A GPU is the correct tool.
Decode generates one token at a time. Every single step requires reading the full model weights plus the entire KV cache out of memory and through the compute units. Arithmetic intensity at batch size one typically lands under 10 FLOPs per byte โ far below the inflection point where a GPU's tensor cores stop being the bottleneck. In practice, decode leaves GPU compute utilization somewhere between 5% and 20%.
You are paying for silicon that sits idle most of the time. That waste is the entire business case for a memory-first chip.
The hardware choice that matters here is almost certainly LPDDR5X rather than HBM3E. I have followed Positron's public positioning, and the argument runs like this: HBM costs roughly $30 to $40 per gigabyte and is constrained by CoWoS advanced packaging capacity, which has been the industry's hard bottleneck since 2023. LPDDR5X costs $3 to $5 per gigabyte, uses standard packaging, and sidesteps the CoWoS queue entirely. Decode does not need peak bandwidth โ it needs capacity multiplied by bandwidth divided by dollars. HBM delivers 3.35 TB/s but only 80 GB. LPDDR5X can stack past 256 GB with a wide bus and multiple chips in parallel.
Physically, that reasoning holds. What the announcement does not mention is the trade-off on the other side of the ledger: LPDDR5X latency runs roughly double HBM's. In long-context workloads where the KV cache is being randomly accessed across a 128K-token window, that latency penalty eats throughput. Nobody in this announcement has published a number.
The Part Nobody Wants to Say Out Loud
Here is the uncomfortable fact: disaggregated prefill and decode is not new. NVIDIA's Dynamo framework shipped exactly this in 2025. vLLM and SGLang have supported separate prefill and decode deployment for longer than most people realize.
So the innovation is not the split. The innovation, if there is one, is the silicon on one end of the split.
That reframes the entire value proposition. If PD disaggregation is already standard software practice, then heterogeneous hardware only earns its place through cost per token. Not through capability. Not through novelty. Cost.
And this announcement provides zero cost data. Not a target, not a range, not a comparison. An inference chip partnership that publishes no cost figures has published nothing.
The Cost Structure Nobody Modeled
Follow the memory supply chain, because this is where the thesis either survives or dies.
If memory-first architectures work, the largest financial beneficiary is not Positron. It is Samsung, SK Hynix, Micron, and CXMT. Every memory-first accelerator consumes two to four times the DRAM capacity of a comparable GPU deployment.
That creates a self-negating loop. The route's success drives demand for its own primary input, which raises the price of that input, which erodes the cost advantage that justified the route in the first place. We watched this exact dynamic play out in 2024 and 2025, when HBM production crowded out general-purpose DRAM capacity and sent commodity memory prices vertical.
If memory-first chips ship at scale, LPDDR5X pricing goes up. Positron's gross margin compresses. The $/token advantage narrows. And meanwhile, H100 rental prices keep falling.
Positron's real competitor is not NVIDIA's roadmap. It is the secondary market price of a used H100.
That is a timing problem, and it is brutal. Every month that the memory-first route spends building a software stack is a month the incumbent's rental economics improve against it.
Where the NVIDIA Disruption Narrative Breaks
Let me dismantle the headline claim directly, because "challenges GPU dominance" is a claim I have watched fail for five straight years.
In a disaggregated deployment, prefill still runs on GPUs โ and prefill is the compute-dense half of the workload. The typical shape of a heterogeneous rack is one GPU paired with several specialized accelerators. Fewer GPU units, but higher-end ones, because the prefill node wants the best silicon available. NVIDIA's revenue per unit of compute can go up in that configuration. The company is selling the disaggregation story itself through Dynamo.
This is the largest logical hole in the coverage. The challenger's architecture may increase the incumbent's margin per rack.
Now look at the graveyard. Graphcore, functionally exited. SambaNova, pivoted to systems integration. Habana, absorbed and marginalized inside Intel. Groq, carved out a niche but never scaled to threat level. Cerebras, still loss-making after going public. Dozens of others simply gone. The base rate for this category is close to zero, and it is not because the technical arguments were wrong. It is because CUDA is not a performance advantage. It is a language, and everyone already speaks it.
Positron's public funding sits somewhere in the $50 to $60 million range. A single 5nm or 4nm AI chip, from design through mass production โ IP licensing, tape-out, packaging, test, plus a software team โ costs $50 million to $150 million. NVIDIA spends over $3 billion per quarter on R&D alone.
That math means the runway covers one, maybe two tape-outs. The tolerance for error is effectively zero. One failed spin, one lost anchor customer, and the company is in liquidation.
And the software question, unaddressed in the announcement, is the one that decides everything. Can a PyTorch model run without a line of code changed, and how fast? That requires custom kernels, a FlashAttention equivalent, quantization support across FP8 and FP4, and deep vLLM integration. Plus KV cache transfer between the prefill and decode nodes, which consumes serious interconnect bandwidth โ PCIe Gen5, NVLink, or Ethernet, with overhead as a percentage of end-to-end latency that nobody has published.
The Regulatory Dimension Nobody Is Pricing
There is a second risk vector entirely absent from the discussion. If memory-first silicon scales, its critical input is DRAM, and the DRAM supply chain runs straight through China. CXMT is the fastest-growing player in that market, while US semiconductor controls tighten every quarter.
If the target market for cheap inference includes China โ and it does, because Chinese inference demand is enormous and cost-sensitive โ then export controls may wall off the largest addressable market before the first rack ships.
Memory-first inference also changes data center power density and cooling requirements in ways the announcement does not quantify. And multi-tenant inference clusters sharing memory across devices raise a side-channel question that no vendor has answered.
What to Watch
Ignore the partnership language. It is a supply-side agreement dressed as a paradigm shift, and it carries the signature of a non-binding announcement timed for a fundraising window rather than a technical milestone.
Watch four things instead. First, whether Positron publishes its software stack as open source โ a closed stack means the ecosystem never forms, and an open one means the path to revenue never forms, which is the classic specialty-silicon dilemma. Second, whether any hyperscaler makes a strategic investment, because that is the only variable that changes the competitive math. Third, the actual delivery schedule and whether the silicon is in volume production or still on a roadmap slide. Fourth, and most importantly, the cost per million tokens with a third-party benchmark attached.
Bets are cheap; exits are expensive. This one has no exit priced in yet, and the memory vendors are the only party certain to collect.