Yes, for a specific part of some local AI workloads. On November 21, 2026, the llama.cpp project published prerelease b12602 with an Intel SYCL optimization for mixture-of-experts models. In the contributor’s Intel Arc Pro B60 measurements, processing a long input prompt became markedly faster. Generating the answer one token at a time did not. That difference is the practical reason to examine this release before changing a local inference setup.
This article compares the actual measurements, explains the GPU operation that changed, and identifies the hardware and configuration limits. INGOAMPT reviewed the published release, implementation and test methodology; we have not independently run the GPU benchmark. The figures below are the project contributor’s results, not promises for every Intel card or model.
What changed in llama.cpp b12602?
The release incorporates pull request #31428, which changes how certain Intel GPU matrix calculations handle quantized mixture-of-experts (MoE) model weights. In a MoE model, a routing mechanism sends each token to a subset of specialized components called experts. That can make the work of multiplying input activations by each selected expert’s weights a bottleneck, depending on the model and hardware.
The previous Intel XMX grouped matrix path unpacked some compressed weights into shared local memory (SLM), and then loaded them into the GPU’s matrix operation. The new ESIMD path can unpack the weights directly into the registers used by Intel’s DPAS/XMX matrix instruction, avoiding that intermediate staging and the associated synchronization barriers. This is a change to a particular GPU kernel—not a new foundation model, not a new API price, and not a general improvement to every inference operation.
The important distinction: reading a prompt versus writing the answer
Local inference usually has two visible phases. Prompt processing (often called prefill) reads the supplied instructions, document and conversation history and produces the state needed to start the reply. Token generation (decode) then emits the response incrementally. Optimizing matrix work during prompt processing can make the start of a response quicker without noticeably increasing the number of generated answer tokens per second.
| Operation | ESIMD off | ESIMD on | Reported difference |
|---|---|---|---|
| Prompt processing, 512 tokens (pp512) | 1,338.3 / 1,338.4 tokens/s | 2,317.0 / 2,311.7 tokens/s | +72.9% |
| Prompt processing, 2,048 tokens (pp2048) | 1,327.8 / 1,327.4 tokens/s | 2,256.4 / 2,255.8 tokens/s | +69.9% |
| Answer generation, 128 tokens (tg128) | 86.9 / 87.5 tokens/s | 87.7 / 86.9 tokens/s | +0.1% |
These values were reported by the developer of the change. A second test with Qwen3.8-Flash-Next on three Arc Pro B60 cards showed smaller prompt-processing gains of roughly 21% and nearly unchanged decoding. The implementation author explicitly notes that cache behavior, weight reuse and ESIMD availability determine whether the optimization helps. There is no independent end-to-end latency, accuracy or battery-life result here.
Who can use the change?
The updated SYCL backend documentation names compatible Intel XMX families, including supported Arc A-series and B-series GPUs, Arc Pro and some Intel data-center devices. It is limited to the grouped XMX path and supported quantized weight types, including Q4_K and several IQ formats. A CPU-only installation, an Apple Metal build or an unsupported GPU does not gain this Intel-specific kernel simply by updating llama.cpp.
The new documented setting GGML_SYCL_MMID_ESIMD defaults to 1, which selects the optimized path when its format, device and shape checks pass. Setting it to 0 chooses the earlier joint_matrix path; 2 requests a slower computation ordering for a bit-identical comparison in the documented case. Builds without GGML_SYCL_SUPPORT_XMX do not compile the new grouped kernel. These controls offer a meaningful way to compare the new path with the old one on supported systems, not a shortcut to compatibility on every Intel GPU.
The release provides project-built binaries for several platforms, including Linux SYCL variants, but obtaining a usable build and the correct Intel runtime still depends on the operating system and GPU. Anyone evaluating an installation should check the official SYCL backend documentation and match the precise model, weight quantization and GPU to the release. The project marks b12602 as a prerelease, so production deployments should also assess stability and rollback requirements.
What does this mean for a solo developer?
For an offline document assistant that repeatedly ingests long prompts on a supported Intel Arc card, the prompt-processing result may be worth testing. For a chat app whose main delay comes from generating long replies, the same tests offer no evidence of a meaningful gain in that phase. Those are different user experiences: the first determines how quickly generation can begin, while the second determines how quickly the answer appears after it starts.
A useful evaluation compares the same model, quantization, prompt length, hardware and build with the new ESIMD path enabled and disabled. Record both prefill and decode speed, end-to-end time, peak memory and output correctness; do not use an isolated kernel number as an application-wide benchmark. The source author ran the published tests, but INGOAMPT has not run this comparison on a physical Intel device.
Because llama.cpp can run models locally, it may help developers keep selected workloads on their own machine. But local execution alone does not prove that an entire product is private or offline: the app, model license, downloaded weights, update checks and any external services also matter. The optimization does not change those data-handling questions.
Availability, costs and limits
llama.cpp is open-source software and the b12602 release is available from the official project repository. This does not make compatible Intel hardware, licensed model weights, electricity or deployment costs free. No new model inference pricing or free marketing distribution program is announced in this release. The gain is narrower and measurable: improved prompt-processing throughput in the contributor’s supported MoE tests.
For a broader comparison before choosing hardware, see INGOAMPT’s MLX, CUDA and ROCm hardware guide. Our earlier llama.cpp Apple GPU convolution report covers a different kernel and device family; its measurements should not be combined with this Intel test.
Sources and methods
Original release: llama.cpp b12602, first published November 21, 2026 at 04:28:53 UTC (05:28 Vienna time; 23:28 November 20 in New York). Implementation and test evidence: contributor’s pull request #31428 and its documented SYCL controls. The PR merge and code commit timestamps are not being substituted for the release’s first-publication time. Illustrations and phase comparison are INGOAMPT’s original explanatory presentation; performance figures are attributed to the project’s public tests.
