llama.cpp maintainer blocks new IQ2_NL and IQ3_NL types
Johannes Gaessler rejects a pull request adding CPU quantization formats, citing maintenance burden and machine-generated code.
A llama.cpp maintainer has come out strongly against a pull request that would add two new low-bit quantization formats, IQ2_NL and IQ3_NL, for CPU inference.
Johannes Gaessler wrote that the project should not take on more data types until it has better infrastructure to judge whether they are actually worthwhile. Even with that bar met, he argued, large volumes of machine-generated code create a lasting maintenance burden and should only land with an explicit commitment from maintainers. "An outsider should not be doing this at all," he said.
In a follow-up, Gaessler stressed that the real cost falls on backend maintainers, not on whoever generates the initial code. Someone unfamiliar with a given backend, he wrote, cannot reliably decide what counts as a sound implementation, and the work ought to be overseen by the people who already own those backends.
The objection leaves the proposed IQ2_NL and IQ3_NL CPU types on hold and underscores a wider tension in the project: new quantization schemes are easy to propose, but supporting them across backends is expensive and hard to reverse.