Git eyes faster connectivity checks for large repos
An opt-in incremental verifier aims to make fetch and push checks scale with incoming changes instead of full boundary size.
Git may gain a much cheaper connectivity check for fetches and pushes on large repositories, under a design from Kristofer Karlsson of Spotify now under review on the Git development list.
After receiving objects, Git verifies that new tips do not point at missing content. The existing full check walks the entire object closure of the boundary commits. On repositories whose boundary trees are huge, that cost barely shrinkes even when only a handful of new commits arrive, so small updates pay for the whole reachable graph in time and memory.
The proposed alternative keeps a growing set of trusted objects and walks each incoming commit against its already-trusted parents. Unchanged subtrees are skipped by object id; only changed paths and new trees are examined. Work then tracks what actually changed rather than the full reachable closure.
A new transfer.connectivityCheck setting would select the algorithm. The default remains full. Values incremental and auto opt in; auto uses the incremental path for modest incoming histories (default cutoff 1000 commits) and falls back to full for larger batches, clones into empty repositories, and deepening fetches that lack a normal trusted boundary. Unknown settings warn and fall back to full so newer configs do not break older Git.
Benchmarks show the payoff on large trees. Against a 3.4 GB boundary closure, verifying one commit dropped from about 1.9s and 3.4 GB RSS under the full walk to roughly 0.01s and 14 MB incremental, with large gains still visible for tens of commits. Synthetic tests report speedups approaching 20x in the happy case. With thousands of incoming commits the fixed cost of the full walk can win again, which is why auto exists.
Partial-clone rules are preserved: promised objects count as connected, and the check must not trigger on-demand fetches. Maintainer Junio C Hamano has been reviewing edge cases around that boundary and how the mode is selected inside rev-list. The change remains opt-in and is still under active review.