HAMZA KAREEM / CONCEPTS hamzawi.dev

Lesson 1022 min read

Shallow Clone vs Partial Clone

Two Ways to Not Download Your Repo

Developer toolingdeveloper tooling / git & CI

The setup

Monorepo: 2.1 GB, 180k commits. CI spends 95s cloning before a single test runs. You reach for --depth=1, your "git diff against main" breaks, you set fetch-depth: 0, and you're back to 95s.

The fix is knowing that a repo has TWO sizes, and these flags shrink different ones.

Two axes

  • History = the commit/tree graph (how many revisions)
  • Content = blobs (the actual file bytes, every version ever)

Content is usually 80-95% of the bytes. History is what your tooling needs.

                    history   blobs    clone
full clone            all      all     2.1 GB / 95s
--depth=1             1        1 rev   180 MB / 12s
--filter=blob:none    all      lazy    310 MB / 18s

Shallow cuts history (and gets blobs for free, since there's one revision). Partial keeps ALL history and skips the blobs, fetching them on demand.

What each one breaks

--depth=1 (shallow):

  • git merge-base origin/main HEAD → fatal, no common ancestor. So git diff origin/main...HEAD dies. This is the #1 CI mystery.
  • git describe --tags → no tags in range, your version string becomes "0.0.0-dev"
  • git blame / log on anything older than the tip → empty

--filter=blob:none (partial, needs Git ≥2.19 + server support):

  • Nothing breaks logically — but blob access becomes a network call. git blame on one file can fire hundreds of lazy fetches and be SLOWER than a full clone. Same for git log -p.
  • Checking out an old tag downloads that whole revision anyway.

The rule

Need the graph (diffs vs base, tags, changed-file detection)? → partial clone.

Need one snapshot to build and throw away? → shallow.

In GitHub Actions, the common mistake is this pair:

# breaks changed-file detection
- uses: actions/checkout@v4          # fetch-depth: 1 default
# fixes it, but pays full 95s
- uses: actions/checkout@v4
  with: { fetch-depth: 0 }

Better — full history, no blob payload:

- uses: actions/checkout@v4
  with:
    fetch-depth: 0
    filter: blob:none

Or raw git:

git clone --filter=blob:none --no-checkout https://... repo
cd repo
git sparse-checkout set services/api packages/shared
git checkout main

Now you have 180k commits of graph, and only the files two directories actually need.

The gotcha that bites teams

Shallow clones are not cheap for the SERVER. The remote has to compute a custom packfile instead of serving its existing ones, so 50 parallel shallow CI jobs can be more load than 50 full clones. Partial clone reuses the server's packs. If your git host is getting hammered at 9am, this is often why.

Also: git fetch --unshallow after a shallow clone downloads everything you skipped, plus you paid for the custom pack. Deepening is not free.

Why it works

Partial clone leaves "promisor" placeholders where blobs should be. Git knows the object exists and which remote promises it. git cat-file -p <sha> on a missing blob silently fetches it — that's why things never break, they just get mysteriously slow in the wrong spots.

git rev-list --objects --missing=print HEAD | grep '^?' | wc -l

...tells you how many objects your checkout is still only promising.

Your exercise

Clone any mid-size repo twice: once with --filter=blob:none, once plain. Compare du -sh .git. Then run git blame on a file with long history in each and time both. The partial one will probably lose — explain in one sentence why, in terms of round trips.