freenode
AI & ML

llama.cpp adds SYCL graph record and replay for Intel GPUs

Opt-in graphs on the oneAPI backend show modest decode gains in early Arc tests, with timeouts still under review.

llama.cpp has gained SYCL graph record and replay support, so the Intel GPU backend can capture and reuse command sequences the way the project's CUDA path already does. The aim is lower submission overhead when running local large language models through oneAPI.

The feature is opt-in via an environment flag. Early Arc results are modest. On an Arc 140T with a quantized Gemma 4 E4B model, one run saw token generation rise from about 13.6 to 14.3 tokens per second with graphs on, while prompt processing stayed essentially flat. Other models and machines show smaller or less consistent swings, and testers are still aligning driver stacks and baselines.

Hang-wait intervals were taken straight from the CUDA side. Contributors note those values may need retuning for SYCL. The approach mirrors llama.cpp's CUDA graph logic rather than wrapping Intel's native SYCL Graph API directly.