Your Own Git Server, and What GitHub Adds

40.0 What this chapter gives you#

1. You will be able to say, in one sentence, what a git server is, and prove it by building one in an empty folder in under a second.
2. You will be able to explain what --bare means, what a bare repository contains, what it deliberately lacks, and why pushing into a normal repository’s checked-out branch is refused by default.
3. You will be able to run the whole loop yourself: create a server repository, push to it, clone it somewhere else, commit there, push back, and pull the change into the first copy, with real output at every step.
4. You will be able to do the same thing over SSH and see that a remote URL is nothing but a path plus a way of getting to it.
5. You will be able to read and write a refspec, and say exactly what +refs/heads/*:refs/remotes/origin/* does, character by character.
6. You will be able to write a server-side hook that deploys a website on every push, and say why a client-side hook is a helper and never a rule.
7. You will be able to state precisely what GitHub is: hosting for bare repositories, plus a web interface, plus identity and permissions, plus automation, founded in 2008 and bought by Microsoft in 2018 for 7.5 billion United States dollars.
8. You will be able to sort every feature you use into “this is git” or “this is GitHub”, and defend each answer.
9. You will be able to explain the reader’s own outage exactly: why git kept working while the GitHub API did not, using a three-column table of operations.
10. You will be able to choose between GitHub, a self-hosted Forgejo or GitLab, and a plain SSH remote with no web interface at all, and say what you gain and what you take on.
11. You will be able to mirror, migrate and back up a repository properly, and say what a backup of git does not cover.

40.1 A git server is just another repository#

PLAIN40.1.1 in simple words#

1. Here is the whole idea of this chapter in one line.
2. A git “server” is just another git repository that other machines can reach.
3. That is all. There is no server program that must be bought. There is no special file format. There is no central authority.
4. Every copy of a git repository holds the full history. Every copy can send history to another copy and receive history from another copy.
5. So a “server” is only a copy that everybody agrees to treat as the meeting point.
6. We call it the server because of where it sits and who can reach it, not because of what it is made of.
7. The one thing a meeting-point copy usually does differently is that it has no working files. Only the history. We will see why in the next section.
8. Everything else in this chapter is proof of that single claim.

PLAIN40.1.2 a picture in your head#

1. Think of a study group of four students who each keep a notebook of the same lecture course.
2. Each notebook is complete. Any student can copy pages from any other student.
3. The group decides to leave one spare notebook in the department office, because the office is open to all four and nobody has to chase anybody.
4. The office notebook is not magic. It is the same kind of notebook. It has the same pages. It is special only because everyone can walk to it.
5. If the office burns down, the course is not lost. Four complete notebooks still exist in four bags.
6. That is exactly the relationship between your laptop’s repository and the one you call the server.

Where this comparison breaks:

7. Copying pages by hand is slow and error-prone. Git copies exactly, and it copies only the pages the other side is missing.
8. Two students writing on the same page at the same time makes a mess. Git refuses that case rather than making a mess: it rejects the push and makes you combine the work first.
9. And a notebook has one order of pages. Git history is a graph, so two students can genuinely have two different valid orders at once.

PLAIN40.1.3 a worked example#

1. We will use three folders on one machine for the whole chapter. One machine is enough to prove the point, because git does not know or care whether the other repository is across the room or across the world.
2. The layout is this.

/tmp/lab/proj          your working copy, folder number one
/tmp/lab/srv/proj.git  the "server", a repository with no files
/tmp/lab/copy          a second working copy, folder number three

3. Here is a real repository being created and given two commits. Every block in this chapter is real output from a real run, not an illustration.

$ git init proj
Initialized empty Git repository in /tmp/lab/proj/.git/

$ git add README.md && git commit -m "First commit: add README"
[main (root-commit) 041c4ad] First commit: add README
 1 file changed, 1 insertion(+)
 create mode 100644 README.md

$ git add app.py && git commit -q -m 'Add app.py' && git log --oneline
08391eb Add app.py
041c4ad First commit: add README

4. Two commits exist. Nothing has touched a network. Nothing has touched a server, because there is not one yet.
5. In the next section we will make the server, and it will take one command and about ten milliseconds.

PLAIN40.1.4 what is really happening inside#

1. When two git repositories talk, one of them runs a small program that reads history out, and the other runs a small program that writes history in.
2. The reading program is called git-upload-pack. It says “here are all the names I have and what they point to”, and then sends whatever objects the other side asks for.
3. The writing program is called git-receive-pack. It accepts a bundle of objects and a list of name changes, and applies them.
4. When you type git fetch, your git starts git-upload-pack on the other side and reads from it.
5. When you type git push, your git starts git-receive-pack on the other side and writes to it.
6. That is the entire client and server story. Two small programs and a way to connect their inputs and outputs.
7. If the other repository is a folder on the same disk, git simply runs the program directly. No network at all.
8. If it is on another machine, git needs some way to run that program over there and pipe bytes back and forth. That is what SSH does. That is what HTTPS does, with a small wrapper. That is what the git protocol does.
9. So the phrase “git server” really means “a machine willing to run git-upload-pack and git-receive-pack on a repository for you”.

TECHNICAL40.1.5 the engineer’s version#

1. Git’s wire protocol is a framed byte stream called pkt-line. Each frame begins with four hexadecimal digits giving the frame’s total length, including those four digits. The value 0000 is a flush packet and marks the end of a section.
2. We can see the raw advertisement by running the server program by hand over SSH. This is the real output, trimmed on the right to fit the page.

$ ssh gitbox 'git upload-pack --advertise-refs \
    /tmp/lab/srv/proj.git' | cat -A | head -3
010b64952410dab49907fec630138e55e7504834b082 HEAD^@multi_ack
 thin-pack side-band side-band-64k ofs-delta shallow ...
 symref=HEAD:refs/heads/main object-format=sha1
 agent=git/2.43.0$
003d64952410dab49907fec630138e55e7504834b082 refs/heads/main$
0000

3. Read the first line. 010b is hexadecimal for 267, the length of that frame. Then a 40-character SHA-1, a space, the ref name HEAD, a NUL byte (shown by cat -A as ^@), and then the capability list.
4. The second frame is 003d, hexadecimal for 61: four length digits, 40 hash characters, one space, refs/heads/main at 15 characters, and one newline. That is 4 + 40 + 1 + 15 + 1 = 61. The framing is exact.
5. 0000 ends the advertisement.
6. Protocol versions: version 0 is the original, version 1 adds a version string, and protocol v2 was introduced in Git 2.18 (June 2018) and made the default for fetch in Git 2.26 (March 2020). Version 2 lets the client ask for a filtered ref list instead of receiving every ref up front, which matters on repositories with tens of thousands of refs.
7. The four transports and how the server program is started:

8. Port 9418 is registered with IANA for the git protocol. The git protocol has no authentication and no encryption, which is why GitHub permanently disabled unencrypted git:// access on 15 March 2022.
9. Commands that observe this layer: git ls-remote, GIT_TRACE=1 git fetch, GIT_TRACE_PACKET=1 git fetch, and git daemon --verbose.

WORDS40.1.6 remember these#

1. Repository — a project’s full history plus its settings — a directory containing an object database, a ref namespace and a configuration file.
2. Server — the copy everybody can reach — any repository exposed through a transport that can start upload-pack or receive-pack.
3. upload-pack — the program that sends history out — the server side of fetch and clone, which advertises refs and streams a packfile.
4. receive-pack — the program that takes history in — the server side of push, which validates ref updates and runs the receive hooks.
5. pkt-line — git’s way of cutting a stream into labelled chunks — a framing format with a four-hex-digit length prefix and 0000 as flush.
6. Transport — the road the bytes travel on — the mechanism (file, ssh, http, git) that connects the two programs’ standard input and output.

40.2 Bare repositories#

PLAIN40.2.1 in simple words#

1. A normal repository has two parts: the files you edit, and a hidden folder called .git that holds the history.
2. A bare repository is the second part on its own. The history, with no files to edit.
3. You make one with git init --bare. By convention its folder name ends in .git, for example proj.git.
4. A bare repository is what you use for a server, because a server has nobody sitting at it editing files.
5. It is missing two things on purpose: a working tree (the actual editable files) and an index (git’s staging list of what will go into the next commit).
6. Those two things only make sense when a human is working. A server has no human working in it.
7. You cannot run git status usefully in a bare repository, because there is no working tree to compare against.
8. You can run git log, git show, git cat-file and everything else that only reads history, because all of the history is right there.

PLAIN40.2.2 a picture in your head#

1. Think of a normal repository as a desk. On the desk are the pages you are writing. In a drawer under the desk is the complete archive of every version ever saved.
2. A bare repository is the drawer, on its own, in a storeroom. No desk.
3. You can pull any past version out of the drawer. You just cannot scribble on anything, because there is no desk surface.
4. When somebody pushes, they are handing new folders into the drawer and updating the labels on the front.

Where this comparison breaks:

5. A drawer takes more space as you add papers. Git’s drawer stores each distinct version once and stores similar versions as small differences, so it grows far more slowly than the total of all versions.
6. And a real drawer does not check what you put in it. Git verifies every object against its hash on the way in and on the way out.

PLAIN40.2.3 a worked example#

1. Here are the two kinds side by side, from a real run.

$ git init --bare srv/proj.git
Initialized empty Git repository in /tmp/lab/srv/proj.git/

$ ls -1 /tmp/lab/proj/.git      $ ls -1 /tmp/lab/srv/proj.git
COMMIT_EDITMSG                  HEAD
HEAD                            branches
branches                        config
config                          description
description                     hooks
hooks                           info
index                           objects
info                            refs
logs
objects
refs

2. Look at what matches. HEAD, branches, config, description, hooks, info, objects, refs are in both. That is the whole history machinery.
3. Look at what only the normal repository has: index, logs and COMMIT_EDITMSG. Those are all about a person doing work here.
4. And the normal repository has one more thing that the bare one does not: the actual files, one level up.

$ ls -1 /tmp/lab/proj
README.md
app.py

5. Git records the difference in one configuration setting.

$ git -C /tmp/lab/proj config --get core.bare
false

$ git -C /tmp/lab/srv/proj.git config --get core.bare
true

$ cat /tmp/lab/srv/proj.git/config
[core]
    repositoryformatversion = 0
    filemode = true
    bare = true

6. That is the entire difference at the configuration level. One boolean.

PLAIN40.2.4 what is really happening inside#

1. Now the important part: why you cannot push into a normal repository’s checked-out branch.
2. A checked-out branch means three things are supposed to agree: the branch pointer, the index, and the files on disk.
3. If somebody pushes new commits into that branch from outside, git would move the branch pointer but would not touch the index or the files.
4. The person sitting at that repository would then be told they have deleted everything that arrived, because their files no longer match their branch.
5. So git refuses. Here is the real refusal, with the long lines wrapped to fit this page and nothing else changed.

$ git push wt main
remote: error: refusing to update checked out branch:
  refs/heads/main
remote: error: By default, updating the current branch in a
  non-bare repository is denied, because it will make the
  index and work tree inconsistent with what you pushed, and
  will require 'git reset --hard' to match the work tree to
  HEAD.
remote:
remote: You can set the 'receive.denyCurrentBranch'
  configuration variable to 'ignore' or 'warn' in the remote
  repository to allow pushing into its current branch;
  however, this is not recommended unless you arranged to
  update its work tree to match what you pushed in some
  other way.
To /tmp/lab/wt
 ! [remote rejected] main -> main (branch is currently
   checked out)
error: failed to push some refs to '/tmp/lab/wt'

6. Read the message closely. It does not say “you may not do this”. It says “this would make the index and work tree inconsistent”.
7. A bare repository has no index and no work tree, so there is nothing to make inconsistent, so the refusal never applies.
8. That is the real reason servers use bare repositories. Not policy. Arithmetic.

TECHNICAL40.2.5 the engineer’s version#

1. git init --bare sets core.bare = true and lays the repository contents directly in the given directory rather than in a .git subdirectory.
2. Contents of a bare repository, and what each part is:

3. Absent from a bare repository: index (the staging area, a binary file in the format documented as the git index format, version 2, 3 or 4), the working tree, logs/ unless core.logAllRefUpdates is turned on, and COMMIT_EDITMSG.
4. The refusal above comes from receive.denyCurrentBranch, whose default is refuse. Related settings are receive.denyDeleteCurrent, default refuse, and receive.denyNonFastForwards, default false.
5. Three ways to make a bare repository, all equivalent in result:

git init --bare proj.git
git clone --bare /path/to/proj proj.git
git clone --mirror /path/to/proj proj.git

6. They differ in what refs come along. --bare from init starts empty. clone --bare copies branches and tags but sets no fetch refspec. clone --mirror copies every ref under refs/* and configures the repository to keep tracking them. We measure that difference in 40.10.
7. There is a fourth arrangement worth knowing: a non-bare repository with receive.denyCurrentBranch = updateInstead. Then a push into the checked out branch also updates the working tree, provided the working tree is clean. This is useful for a small deploy target and is handled by the push-to-checkout hook.
8. Observation commands: git rev-parse --is-bare-repository, git rev-parse --git-dir, git count-objects -vH.

WORDS40.2.6 remember these#

1. Bare repository — the history with no editable files — a repository with core.bare = true, containing the object database and refs at top level, with no working tree and no index.
2. Working tree — the files you actually edit — the checked-out state of one commit, materialized on the filesystem.
3. Index — git’s list of what goes in the next commit — a binary file mapping paths to blob hashes and stat data, also used during merges.
4. Checked-out branch — the branch you are currently on — the branch named by the symbolic ref HEAD.
5. denyCurrentBranch — the rule that stops pushes into a busy branch — the receive.denyCurrentBranch configuration key, default refuse.

40.3 Doing it end to end#

PLAIN40.3.1 in simple words#

1. We will now do the whole loop, with nothing hidden.
2. Make a server repository. Tell your project about it. Send your history.
3. Copy the server repository into a fresh folder as if you were a second person on a second machine.
4. Make a commit in that fresh folder. Send it to the server.
5. Go back to the first folder. Bring the change down. Now both folders and the server agree.
6. Then we do the same thing again over SSH, to show that the only thing that changed is the address.
7. Here is the shape of the whole thing.

  folder one              server              folder three
  /tmp/lab/proj      /tmp/lab/srv/proj.git   /tmp/lab/copy
       |                    |                     |
       |---- push main ---->|                     |
       |                    |<---- clone ---------|
       |                    |                     |
       |                    |<--- push main ------|
       |<--- fetch/pull ----|                     |

PLAIN40.3.2 a picture in your head#

1. Think of a shared drop box in a corridor between two rooms.
2. You put a copy of your work in the box. The person in the other room takes a copy out.
3. They add a page and put their copy back in. You take the new page.
4. Nobody ever has to be in the corridor at the same time. The box holds the agreed state.
5. Git’s box is smarter than a real one, because it never loses the older versions when a new one arrives. It stacks them all, in order, with each one pointing at the one before.

Where this comparison breaks:

6. A drop box has no opinion. Git’s box refuses anything that would erase history it already holds, unless you force it explicitly.
7. And a drop box holds one thing at a time. Git’s box holds many named lines of work at once, one name per branch.

PLAIN40.3.3 a worked example#

1. Step one. Point the project at the server and push.

$ git remote add origin /tmp/lab/srv/proj.git

$ git remote -v
origin  /tmp/lab/srv/proj.git (fetch)
origin  /tmp/lab/srv/proj.git (push)

$ git push -u origin main
To /tmp/lab/srv/proj.git
 * [new branch]      main -> main
branch 'main' set up to track 'origin/main'.

2. The server now really has the history. Look inside it directly.

$ ls -1 /tmp/lab/srv/proj.git/refs/heads
main

$ cat /tmp/lab/srv/proj.git/refs/heads/main
fa776d1882d36bdc0af68926f71ff710cdea281a

$ git -C /tmp/lab/srv/proj.git log --oneline
fa776d1 Change app.py
08391eb Add app.py
041c4ad First commit: add README

3. A branch on the server is one file containing one 40-character hash. That is all a branch has ever been.
4. Step two. Clone it into a third folder, as a second person would.

$ git clone /tmp/lab/srv/proj.git /tmp/lab/copy
Cloning into '/tmp/lab/copy'...
done.

$ git -C /tmp/lab/copy log --oneline
fa776d1 Change app.py
08391eb Add app.py
041c4ad First commit: add README

$ git -C /tmp/lab/copy branch -a
* main
  remotes/origin/HEAD -> origin/main
  remotes/origin/main

5. Step three. Commit in the copy and push it back. This is the terminal view, where the percentage counters overwrite themselves in place.

$ git add NOTES.md && git commit -q -m 'Add NOTES.md'

$ git push --progress origin main
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 2 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 328 bytes | 328.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To /tmp/lab/srv/proj.git
   fa776d1..249b3ed  main -> main

6. Three objects and 328 bytes went across: one commit, one tree, one blob. Nothing else was sent, because the server already had everything else.
7. Step four. Back in folder one, fetch first so you can see the two halves of a pull separately.

$ git status -sb
## main...origin/main

$ git fetch origin
From /tmp/lab/srv/proj
   fa776d1..249b3ed  main       -> origin/main

$ git status -sb
## main...origin/main [behind 1]

$ ls
README.md
app.py

8. Read that carefully. After fetch, git knows about the new commit and says you are behind by one. But ls shows no new file. Fetch changed nothing in your working tree. It only downloaded history.
9. Now pull, which is fetch plus merge.

$ git pull
Updating fa776d1..249b3ed
Fast-forward
 NOTES.md | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 NOTES.md

$ ls
NOTES.md
README.md
app.py

10. The loop is closed. Three repositories, one history, no service, no account, no web page.

PLAIN40.3.4 what is really happening inside#

1. Now the same thing over SSH, to prove the point of this chapter.
2. A remote URL has two parts: how to get there, and where it is once you arrive. Change the first part and nothing else changes.

$ git remote add ssh-origin \
    ssh://root@127.0.0.1:2222/tmp/lab/srv/proj.git

$ git ls-remote ssh-origin
249b3edc818ac4a38fc39b982e19a880675326de    HEAD
249b3edc818ac4a38fc39b982e19a880675326de    refs/heads/main

3. Exactly the same repository, reached a different way. Now push to it.

$ git commit -q -am 'Document how to run app'

$ git push ssh-origin main
To ssh://127.0.0.1:2222/tmp/lab/srv/proj.git
   249b3ed..6495241  main -> main

$ git -C /tmp/lab/srv/proj.git log --oneline -1
6495241 Document how to run app

4. And clone over SSH using the short form, which needs an entry in your SSH configuration file because the short form cannot carry a port number.

$ cat /root/.ssh/config
Host gitbox
  HostName 127.0.0.1
  Port 2222
  User root
  IdentityFile /root/.ssh/id_ed25519

$ git clone gitbox:/tmp/lab/srv/proj.git /tmp/lab/viassh
Cloning into '/tmp/lab/viassh'...

$ git -C /tmp/lab/viassh log --oneline -2
6495241 Document how to run app
249b3ed Add NOTES.md

5. What SSH actually did was log in and run one command. Here is git’s own verbose trace of that decision, reproduced by running the command by hand.

$ ssh gitbox 'git upload-pack --advertise-refs \
    /tmp/lab/srv/proj.git' | head -c 60
010b64952410dab49907fec630138e55e7504834b082 HEAD

6. So SSH is not doing anything clever. It is a remote login that runs git-upload-pack or git-receive-pack and connects the pipes.
7. That is why an SSH remote needs no git-specific server software at all. If the account can log in and git is installed, you have a git server.

TECHNICAL40.3.5 the engineer’s version#

1. The git protocol also works, which shows the third transport. Here is a real git daemon serving the same bare repository.

$ git daemon --reuseaddr --base-path=/tmp/lab/srv \
    --export-all --verbose

$ git ls-remote git://127.0.0.1/proj.git
a2648f3588da557802930bc984745c8a574b7a38    HEAD
8a65572b3a58cded170998be063e26b8cbea76e6    refs/heads/demo
0327736b4327edde443c3dfa9456555015d0b3f4    refs/heads/feature/login
a2648f3588da557802930bc984745c8a574b7a38    refs/heads/main
c541c6934d9bd45d02e86d902253422366134e87    refs/notes/commits
9ee3b0bda37deeffc6317b5dcda3c483895f490c    refs/tags/v1.0

2. The daemon log confirms protocol v2 and which program was requested.

[17254] Ready to rumble
[17260] Connection from 127.0.0.1:42896
[17260] Extended attribute "host": 127.0.0.1
[17260] Extended attribute "protocol": version=2
[17260] Request upload-pack for '/proj.git'
[17254] [17260] Disconnected

3. And the git protocol is read-only unless you go out of your way, because it has no authentication:

$ git push origin main
fatal: remote error: access denied or repository not exported:
  /proj.git

4. Timings and sizes from this run, on a small repository:

5. Server-side authorization for SSH remotes is normally done by the account, not by git. The standard pattern on a self-hosted box is a single git user whose ~/.ssh/authorized_keys has one line per developer key, each line prefixed with command="git-shell -c \"$SSH_ORIGINAL_COMMAND\"" plus no-port-forwarding,no-agent-forwarding,no-pty.
6. git-shell is a restricted login shell shipped with git that only permits git-upload-pack, git-upload-archive and git-receive-pack. It has existed since the early git releases and is the simplest real access control you can deploy.
7. Useful environment variables for debugging transports: GIT_TRACE=1, GIT_TRACE_PACKET=1, GIT_SSH_COMMAND, GIT_CURL_VERBOSE=1.

WORDS40.3.6 remember these#

1. Remote — a saved nickname for another repository — a named set of configuration keys holding one or more URLs and one or more refspecs.
2. origin — the usual nickname for where you cloned from — a convention, set automatically by git clone, changeable with git clone -o name.
3. Fetch — download history without touching your files — updating remote-tracking refs and the object database only.
4. Pull — fetch and then combine — git fetch followed by git merge or git rebase, depending on pull.rebase.
5. Fast-forward — moving a branch pointer along a straight line — an update where the old commit is an ancestor of the new one, so no merge is needed.
6. git-shell — a login shell that only allows git — a restricted shell that rejects every command except the three git transport programs.

40.4 Remotes in depth#

PLAIN40.4.1 in simple words#

1. A remote is a saved nickname for another repository.
2. Instead of typing a long address every time, you say origin and git looks up the address.
3. You can have as many remotes as you like. They are just entries in a text file inside your repository.
4. The address can take four shapes: a plain folder path, an SSH address, an HTTPS address, or a git:// address.
5. You can add a remote, rename it, change its address and delete it, and none of that touches your history at all. It is only a phone book entry.
6. A remote also carries a rule that says which of the other side’s names get copied into which of your names. That rule is called a refspec.
7. The refspec is the part that most people never read, and it is the part that explains almost every confusing thing about push and fetch.

PLAIN40.4.2 a picture in your head#

1. Think of the contacts list in a phone.
2. origin is a contact name. Behind it sits a number.
3. Changing the number does not change any of your past conversations. It only changes where the next call goes.
4. Deleting the contact does not delete the conversations either.
5. Now imagine each contact also has a filing rule attached: “anything this person sends me goes in the folder named after them”. That is the refspec.
6. So messages from Anita land in a folder called Anita, and messages from Bala land in a folder called Bala, and they never overwrite each other even if both send a file with the same name.

Where this comparison breaks:

7. A phone contact has one number. A git remote can have one address for reading and several different addresses for writing.
8. And a filing rule in a phone is your private business. A refspec is negotiated with the other side, because the other side must agree that the names you ask for exist.

PLAIN40.4.3 a worked example#

1. Everything about a remote lives in the repository’s configuration file. Here is the real file after adding two remotes.

$ cat .git/config
[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = /tmp/lab/srv/proj.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
    remote = origin
    merge = refs/heads/main
[remote "ssh-origin"]
    url = ssh://root@127.0.0.1:2222/tmp/lab/srv/proj.git
    fetch = +refs/heads/*:refs/remotes/ssh-origin/*

2. Now the refspec, character by character. The value is:

+refs/heads/*:refs/remotes/origin/*

3. It splits at the colon into a source and a destination.

4. In plain words: “take every branch on the other side, and store it under a name of mine that begins with refs/remotes/origin/, and let it move backwards or sideways if it has to.”
5. So the other side’s main becomes your refs/remotes/origin/main, which you normally type as origin/main.
6. This is why origin/main is not a branch you can commit on. It is a read-only copy of a name that lives somewhere else.
7. Watch the destination folder appear on disk after fetching from two different remotes.

$ ls -R .git/refs
.git/refs:
heads
remotes
tags

.git/refs/heads:
main

.git/refs/remotes:
origin
ssh-origin

.git/refs/remotes/origin:
main

.git/refs/remotes/ssh-origin:
main

8. Now push with an explicit refspec, which lets you send a local branch to a differently named remote branch.

$ git push origin HEAD:refs/heads/wip-login
To /tmp/lab/srv/proj.git
 * [new branch]      HEAD -> wip-login

$ git ls-remote origin
64952410dab49907fec630138e55e7504834b082    HEAD
64952410dab49907fec630138e55e7504834b082    refs/heads/main
0327736b4327edde443c3dfa9456555015d0b3f4    refs/heads/wip-login

9. And an empty source means “delete the destination”. That is the whole trick behind deleting a remote branch.

$ git push origin :refs/heads/wip-login
To /tmp/lab/srv/proj.git
 - [deleted]         wip-login

$ git ls-remote origin
64952410dab49907fec630138e55e7504834b082    HEAD
64952410dab49907fec630138e55e7504834b082    refs/heads/main

10. git push origin --delete wip-login is a friendlier spelling of exactly the same refspec.

PLAIN40.4.4 what is really happening inside#

1. The leading + deserves its own demonstration, because it is the difference between “I refuse to lose history” and “overwrite it”.
2. We put a commit on a branch, copy it with a refspec that has no +, then rewrite the commit on the server, then fetch again.

$ git fetch -q origin \
    '+refs/heads/demo:refs/remotes/nf/demo'
$ git rev-parse --short refs/remotes/nf/demo
4f4b9de

3. Now the branch is rewritten on the server, with git commit --amend followed by a force push. The old commit 4f4b9de becomes 8a65572, and the two are not ancestors of each other.

$ git push --force origin demo
To /tmp/lab/srv/proj.git
 + 4f4b9de...8a65572 demo -> demo (forced update)

4. Fetch with a refspec that has no +, and git refuses. Fetch with +, and git obeys. The same command, run twice, differing by one character.

$ git fetch origin refs/heads/demo:refs/remotes/nf/demo
From /tmp/lab/srv/proj
 ! [rejected]        demo -> nf/demo  (non-fast-forward)
 + 4f4b9de...8a65572 demo -> origin/demo  (forced update)

$ git fetch origin '+refs/heads/demo:refs/remotes/nf/demo'
From /tmp/lab/srv/proj
 + 4f4b9de...8a65572 demo -> nf/demo  (forced update)

$ git rev-parse --short refs/remotes/nf/demo
8a65572

5. Notice something in the middle of the first result. nf/demo was rejected, and in the very same command origin/demo was force-updated.
6. That is because the default refspec for origin has the +, and the one we typed by hand did not. Two rules, one fetch, two different outcomes.
7. That single output block explains the whole meaning of the plus sign better than any paragraph could.

TECHNICAL40.4.5 the engineer’s version#

1. URL forms, all four, with real examples from this chapter’s run:

2. The short SSH form, user@host:path, is called the scp-like syntax. It cannot express a port number. If you need a non-default port you must use the ssh:// form or a Host block in ~/.ssh/config. That is a real limitation, not a style preference.
3. In the ssh:// form the path is absolute. In the scp-like form the path is relative to the login user’s home directory unless it starts with a slash. This trips people up constantly.
4. Remote management commands:

git remote add <name> <url>
git remote rename <old> <new>
git remote remove <name>
git remote set-url <name> <newurl>
git remote set-url --push <name> <pushurl>
git remote set-url --add --push <name> <another>
git remote show <name>
git remote prune <name>

5. Different fetch and push URLs are genuinely useful. Here we set one remote that reads from one place and writes to two, so a single push updates both a primary and a backup.

$ git remote -v
both    /tmp/lab/srv/proj.git (fetch)
both    /tmp/lab/srv/proj.git (push)
both    /tmp/lab/srv/mirror.git (push)

$ git push both main
Everything up-to-date
To /tmp/lab/srv/mirror.git
 * [new branch]      main -> main

6. Common reasons to have several remotes:

7. Refspec grammar, stated properly. A refspec is [+]<src>[:<dst>]. For fetch, src is a ref on the remote and dst is a ref in your repository. For push, src is a local ref or any expression that resolves to a commit, and dst is a ref on the remote. The + sets the force flag for that one mapping. An empty src on push means delete dst. An omitted dst on fetch means “download it but only record it in FETCH_HEAD”.
8. A remote may carry several fetch refspecs. Adding one is how you fetch GitHub pull-request refs:

git config --add remote.origin.fetch \
  '+refs/pull/*/head:refs/remotes/origin/pr/*'

9. URL rewriting is a useful production trick. insteadOf rewrites a prefix for all commands:

$ git config --global \
    url.'ssh://root@127.0.0.1:2222/tmp/lab/srv/'.insteadOf 'kb:'

$ git ls-remote kb:proj.git | head -1
a2648f3588da557802930bc984745c8a574b7a38    HEAD

10. pushInsteadOf does the same but only for pushes, which is the standard way to clone over HTTPS in continuous integration while pushing over SSH from a developer machine using the same recorded URL.

WORDS40.4.6 remember these#

1. Refspec — the rule that maps their names onto your names — a string of the form [+]<src>:<dst> used by fetch and push.
2. Remote-tracking ref — your read-only copy of their branch — a ref under refs/remotes/<remote>/, updated only by fetch, push and remote prune.
3. Force flag — permission to lose history on one ref — the + prefix in a refspec, or --force on the command line.
4. Scp-like syntax — the short SSH address form — user@host:path, which cannot carry a port number.
5. insteadOf — a find-and-replace for remote addresses — configuration keys url.<base>.insteadOf and url.<base>.pushInsteadOf.
6. FETCH_HEAD — a scratch note of what the last fetch brought — a file listing the refs and hashes downloaded by the most recent fetch.

40.5 Hooks: server-side and client-side#

PLAIN40.5.1 in simple words#

1. A hook is a small program that git runs automatically at a certain moment.
2. Hooks live in the hooks directory of a repository. They are ordinary executable files. Shell, Python, anything the machine can run.
3. Git ships them as .sample files, which are inert. Rename one to remove the .sample and it becomes live.
4. There are hooks that run on your own machine, and hooks that run on the server when a push arrives.
5. The three important server hooks are pre-receive, update and post-receive.
6. pre-receive runs once for the whole push, before anything is saved. If it exits with a non-zero status, the entire push is rejected.
7. update runs once per branch or tag being changed. It can reject one branch and let the others through.
8. post-receive runs once after everything is saved. It cannot reject anything. It is for doing things afterwards: deploying, notifying, triggering a build.
9. Client hooks such as pre-commit and commit-msg run on your own machine. They are helpful, and they are not enforcement, because anybody can skip them with one flag.

PLAIN40.5.2 a picture in your head#

1. Think of a building with a security desk at the entrance and a noticeboard inside.
2. pre-receive is the guard who checks the whole delivery at the door. If any item on the trolley is wrong, the whole trolley goes back.
3. update is a second guard who checks each box on the trolley separately. Some boxes may pass while others are turned away.
4. post-receive is the person who, once the delivery is inside, updates the noticeboard and phones the department that ordered it. That person cannot send the delivery back. It is already in.
5. Client hooks are the checklist taped to your own desk before you leave the house. Useful. Easy to ignore.

Where this comparison breaks:

6. A guard can be persuaded. A hook is a program and does exactly what it says, every time, with no judgement.
7. And a real guard sees the whole delivery. A hook sees only three pieces of text per ref: the old hash, the new hash and the ref name. Everything else it must look up for itself.

PLAIN40.5.3 a worked example#

1. Here is a real, working post-receive hook that deploys a website. It is in the server repository at hooks/post-receive and is executable.

#!/bin/sh
# Deploy the site when main is updated.
DEPLOY_DIR=/tmp/lab/www
BRANCH=refs/heads/main

while read old new ref
do
  if [ "$ref" = "$BRANCH" ]; then
    echo "post-receive: deploying $ref"
    echo "post-receive: old=$(git rev-parse --short $old)"
    echo "post-receive: new=$(git rev-parse --short $new)"
    git --work-tree="$DEPLOY_DIR" --git-dir=. checkout -f main
    echo "post-receive: deployed to $DEPLOY_DIR"
  else
    echo "post-receive: ignoring $ref"
  fi
done

2. Line by line, because every line earns its place.
3. #!/bin/sh is the shebang. It tells the kernel which interpreter to run. Without it, git will try to execute the file directly and fail.
4. DEPLOY_DIR is where the website files should end up. In production this would be the directory your web server serves.
5. BRANCH is the only branch we deploy. Pushing a feature branch must not change the live site.
6. while read old new ref is the key line. Git feeds post-receive one line per updated ref on standard input, and each line has exactly three fields: the hash before, the hash after, and the full ref name.
7. The if compares the ref name to refs/heads/main. Note it is the full ref name, not the short name. A hook never sees main on its own.
8. git rev-parse --short turns a 40-character hash into a 7-character one for the log line. For a brand new branch the old hash is forty zeros.
9. The deploy line is the interesting one. --work-tree tells git to put the files somewhere other than the repository, and --git-dir=. says the history is right here. checkout -f main then writes every file of main into the deploy directory, overwriting whatever is there.
10. echo output from a server hook is sent back to the person pushing, prefixed with remote:. That is how you talk to your users.
11. Now the real push, with the hook live. The deploy directory was empty before this command.

$ git add index.html && git commit -q -m 'Add home page'

$ git push origin main
remote: post-receive: deploying refs/heads/main
remote: post-receive: old=6495241
remote: post-receive: new=adb4a16
remote: Already on 'main'
remote: post-receive: deployed to /tmp/lab/www
To /tmp/lab/srv/proj.git
   6495241..adb4a16  main -> main

$ ls -la /tmp/lab/www
-rw-r--r-- 1 root root   21 Aug 13 03:12 NOTES.md
-rw-r--r-- 1 root root   47 Aug 13 03:12 README.md
-rw-r--r-- 1 root root   18 Aug 13 03:12 app.py
-rw-r--r-- 1 root root   17 Aug 13 03:12 index.html

$ cat /tmp/lab/www/index.html
<h1>KedByte</h1>

12. That is a complete deployment system in fourteen lines of shell. No service, no agent, no vendor.

PLAIN40.5.4 what is really happening inside#

1. Now the two hooks that can say no. First pre-receive, enforcing a branch naming policy.

#!/bin/sh
fail=0
while read old new ref
do
  case "$ref" in
    refs/heads/main)      ok=yes ;;
    refs/heads/feature/*) ok=yes ;;
    refs/tags/*)          ok=yes ;;
    refs/notes/*)         ok=yes ;;
    *)                    ok=no  ;;
  esac
  if [ "$ok" = "no" ]; then
    echo "policy: bad branch name: $ref"
    fail=1
  fi
done
if [ "$fail" = "1" ]; then
  echo "policy: whole push rejected"
  exit 1
fi
exit 0

2. Watch it reject a whole push because one of two branches broke the rule. Both branches had real new commits. Both were refused.

$ git push origin main tmp/scratch
remote: policy: bad branch name: refs/heads/tmp/scratch
remote: policy: whole push rejected
To /tmp/lab/srv/proj.git
 ! [remote rejected] main -> main (pre-receive hook declined)
 ! [remote rejected] tmp/scratch -> tmp/scratch (pre-receive
   hook declined)
error: failed to push some refs to '/tmp/lab/srv/proj.git'

$ git ls-remote origin refs/heads/main
adb4a16ce58824aa2079ee092be20959ecf7cb92    refs/heads/main

3. The server’s main did not move. That is the all-or-nothing behaviour of pre-receive, and it is why it is the right place for policy that must never be half-applied.
4. Now update, which is per-ref. This one makes main fast-forward only.

#!/bin/sh
ref="$1"; old="$2"; new="$3"
zero=0000000000000000000000000000000000000000

if [ "$ref" = "refs/heads/main" ]; then
  if [ "$new" = "$zero" ]; then
    echo "policy: main may not be deleted"
    exit 1
  fi
  if [ "$old" != "$zero" ]; then
    base=$(git merge-base "$old" "$new")
    if [ "$base" != "$old" ]; then
      echo "policy: main is fast-forward only"
      exit 1
    fi
  fi
fi
exit 0

5. update receives its three values as command-line arguments, not on standard input. That is a genuine difference from pre-receive and it is easy to get wrong.
6. The forty zeros mean “nothing”. A new hash of all zeros means the ref is being deleted. An old hash of all zeros means the ref is being created.
7. git merge-base "$old" "$new" finds the youngest common ancestor. If that ancestor is the old commit itself, then the new commit is a descendant and the update is a fast-forward. If not, history is being rewritten.
8. Here is a real force push being stopped by that rule.

$ git commit -q --amend -m 'Add about page (message fixed)'

$ git push --force origin main
remote: policy: main is fast-forward only
remote: error: hook declined to update refs/heads/main
To /tmp/lab/srv/proj.git
 ! [remote rejected] main -> main (hook declined)
error: failed to push some refs to
  '/tmp/lab/srv/proj.git'

$ git ls-remote origin refs/heads/main
a07dcbed1fbb04efa6dbb8f6272a2aed8c361556    refs/heads/main

9. That is branch protection, built by hand, in twenty lines. GitHub’s branch protection is the same idea with a web form in front of it.
10. One honest note from the same run. When we tried git push origin :main to delete the branch, git’s own built-in receive.denyDeleteCurrent check fired before our hook did, because main is the branch that the bare repository’s HEAD points at. Our hook line is still worth having, for every branch that is not the current one.

TECHNICAL40.5.5 the engineer’s version#

1. Server-side hooks, in the exact order git runs them on a push:

2. post-update is the oldest of these and its main modern use is running git update-server-info, which is only needed for the “dumb” HTTP transport, where a plain web server serves the repository files with no git-aware program behind it.
3. Client-side hooks that matter day to day:

4. Here are two real client hooks and the real refusals they produce. First pre-commit, refusing a marker string:

#!/bin/sh
if git diff --cached -U0 | grep -q 'DO-NOT-COMMIT'; then
  echo "pre-commit: found DO-NOT-COMMIT marker"
  exit 1
fi
exit 0

$ git add secrets.py && git commit -m 'KB-42: add config'
pre-commit: found DO-NOT-COMMIT marker

5. Then commit-msg, enforcing a ticket prefix. Git passes the path of the file holding the message as the first argument.

#!/bin/sh
first=$(head -n 1 "$1")
if ! echo "$first" | grep -Eq '^KB-[0-9]+: .+'; then
  echo "commit-msg: need a KB-<number>: prefix"
  echo "commit-msg: you wrote: $first"
  exit 1
fi
exit 0

$ git commit -m 'add contact page'
commit-msg: need a KB-<number>: prefix
commit-msg: you wrote: add contact page

$ git commit -m 'KB-43: add contact page'
[main 40048a8] KB-43: add contact page
 1 file changed, 1 insertion(+)

6. And now the honest part, demonstrated rather than asserted. One flag skips both hooks, and the resulting commit pushes perfectly happily.

$ git commit --no-verify -m 'whatever, no ticket, no checks'
[main a2648f3] whatever, no ticket, no checks
 1 file changed, 1 insertion(+)
 create mode 100644 leak.py

$ git push -q origin main && git ls-remote origin main
a2648f3588da557802930bc984745c8a574b7a38    refs/heads/main

7. The secret marker went in. The ticket prefix was skipped. Both hooks were bypassed by a single documented flag.
8. Client hooks are not enforcement. They are a convenience that catches honest mistakes. Three independent reasons: --no-verify exists, hooks are not committed to the repository so a fresh clone has none, and a hook file that is not executable is silently ignored.
9. Because hooks are not versioned, teams use a manager that installs them from a committed configuration. The common ones are pre-commit (a Python tool, first released in 2014), Husky for Node.js projects, and setting core.hooksPath to a directory that is committed. core.hooksPath arrived in Git 2.9 (June 2016) and is the plainest option.
10. Real enforcement lives on the server: pre-receive and update hooks on a self-hosted server, or branch protection plus required status checks on a hosted platform. Everything else is advice.

WORDS40.5.6 remember these#

1. Hook — a program git runs at a set moment — an executable in the hooks directory or in core.hooksPath, invoked with a defined argument or stdin contract and judged by its exit status.
2. pre-receive — the all-or-nothing gate on a push — a server hook reading old new ref lines on stdin, whose non-zero exit rejects every ref.
3. update — the per-branch gate — a server hook called once per ref with three arguments, whose non-zero exit rejects that ref only.
4. post-receive — the after-the-fact action — a server hook that runs once the refs are updated and cannot reject anything.
5. no-verify — the flag that skips client hooks — --no-verify on commit and push, which is why client hooks are advice and not policy.
6. core.hooksPath — a way to share hooks with a team — a configuration key pointing at a directory of hooks, which can be committed to the repository.

40.6 What GitHub actually is#

PLAIN40.6.1 in simple words#

1. GitHub is four things stacked on top of each other.
2. One: hosting for bare repositories. Machines that hold your repositories and run git-upload-pack and git-receive-pack for you.
3. Two: a web interface. Pages that let you read code, history and differences in a browser, and comment on them.
4. Three: an identity and permission system. Accounts, organizations, teams, tokens, and rules about who may do what to which repository.
5. Four: automation. Servers that run your tests and builds when something happens, and store the results.
6. GitHub was founded in 2008 by Tom Preston-Werner, Chris Wanstrath and P. J. Hyett. The site opened to the public in April 2008.
7. Microsoft announced it would buy GitHub on 4 June 2018 for 7.5 billion United States dollars in Microsoft stock. The purchase completed on 26 October 2018.
8. Here is the sentence that matters most in this chapter: the git part of GitHub is not the valuable part.
9. Anybody can host bare repositories. It took us one command and ten milliseconds. What is hard to copy is the other three layers, and above all the fact that millions of developers already have an account there.

PLAIN40.6.2 a picture in your head#

1. Think of a public library that also happens to have meeting rooms, a membership desk and a printing service.
2. Holding books is the easy part. Any building with shelves can hold books.
3. What makes the library useful is that you know where it is, everyone else uses it too, there is a catalogue, there is a librarian who checks membership cards, and there is a room you can book.
4. GitHub is that. The shelves are cheap. The catalogue, the membership system and the fact that everyone is already a member are the product.
5. That is also why leaving is harder than it looks. Your books come with you. Your reading room bookings, your notes in the margins and your library card do not.

Where this comparison breaks:

6. A library lends you one copy and keeps the original. Git gives you the whole collection every time, so the “original” has no special status.
7. And a library card is issued by the library. Your git history is not issued by GitHub. It is yours, complete, on your own disk.

PLAIN40.6.3 a worked example#

1. Here is proof that GitHub’s git layer is ordinary git, taken live during the writing of this chapter.

$ git ls-remote https://github.com/git/git | head -5
745601a9a94110d74769ab605ccd4f61339758d2    HEAD
165e5ad3169d0fd26637da3383a4514f1a9d1e72    refs/heads/bisect
426ec014a6630fb78c5a7334461bc33568f07743    refs/heads/jch
e9019fcafe0040228b8631c30f97ae1adb61bcdc    refs/heads/maint
745601a9a94110d74769ab605ccd4f61339758d2    refs/heads/master

$ git ls-remote https://github.com/git/git | wc -l
5297

2. That is exactly the same command, producing exactly the same shape of output, as the one we ran against a folder in /tmp.
3. The git source repository on GitHub advertises 5,297 refs. Our little bare repository advertised seven. Same protocol, same program, different scale.
4. Now here is proof of the second layer, the part that is not git. GitHub stores pull requests as extra refs that plain git can see but that plain git never creates.

$ git ls-remote https://github.com/cli/cli 'refs/pull/*' \
    | head -4
e9a3253762e768badaa1d4a5b3d267416d1e42f4    refs/pull/1/head
9966ca8bdb06ff86febbb49ff08480ef3fb4f790    refs/pull/10/head
24b83202679eeddb72bf9ebeb35752bf3f5ea737    refs/pull/100/head
b4b8e8a13bb99bae479929048ea5f64808d0bc32    refs/pull/10002/head

$ git ls-remote https://github.com/cli/cli 'refs/pull/*' \
    | wc -l
5147

5. Five thousand one hundred and forty-seven pull-request refs. Git can list them, because they are refs. Git cannot create one, cannot comment on one, cannot approve one and does not know what “pull request” means.
6. That single distinction is the subject of the next two sections.

PLAIN40.6.4 what is really happening inside#

1. When you git push to GitHub, this is the sequence.
2. Your git resolves github.com to an address. In the reader’s own session it resolved to 20.207.73.82, which is in a Microsoft-owned range, because GitHub is a Microsoft company and fronts traffic through Microsoft’s network edge.
3. Your git opens a connection, either TCP port 22 for SSH or TCP port 443 for HTTPS.
4. It authenticates. Over SSH that is your key pair. Over HTTPS that is a personal access token or a credential helper.
5. GitHub’s front end checks who you are and whether you may write to that repository. This check is GitHub’s, not git’s.
6. If allowed, GitHub runs git-receive-pack against the bare repository it holds for you. From this point the bytes are ordinary git.
7. GitHub’s own pre-receive hooks then run. Branch protection, required signatures, push rules and secret scanning all live here.
8. If nothing objects, the refs update and GitHub’s equivalent of post-receive fires. That is what creates the timeline entry, sends the notification, updates the pull request, and starts an Actions workflow.
9. Steps 4, 5, 7 and 8 are GitHub. Steps 3 and 6 are git. That is the whole boundary.

TECHNICAL40.6.5 the engineer’s version#

1. Dates and figures worth committing to memory:

2. GitHub removed password authentication for git operations over HTTPS on 13 August 2021. Since then HTTPS pushes require a personal access token, an OAuth token, a GitHub App installation token, or SSH keys instead.
3. On 24 March 2023 GitHub replaced its RSA SSH host key after the private key was briefly exposed in a public repository. Every user with the old key in known_hosts saw a host key verification failure. This is a good reminder that host key pinning is real security with real operational cost.
4. GitHub’s git hosting uses a custom system for replication and routing. Publicly described pieces include Spokes, formerly called DGit, for three-way replication of repositories across servers, and a proxy layer in front of the git transports. The important engineering point is that nothing about this changes the wire protocol. Your git client cannot tell.
5. The scale of GitHub’s non-git surface is what makes it expensive to run: the REST API, the GraphQL API, webhooks, Actions runners, package registries, Pages hosting and code search. None of those are git.
6. Estimated user counts published by GitHub have grown from roughly 100 million developers reported in early 2023 to figures above that since. Treat any single number as approximate and dated, because the company publishes it as a marketing figure rather than an audited one.

WORDS40.6.6 remember these#

1. Forge — a website that hosts repositories and adds collaboration — the general term covering GitHub, GitLab, Gitea, Forgejo and Bitbucket.
2. Hosting — somebody else runs the bare repository — a service that exposes upload-pack and receive-pack over authenticated transports.
3. Personal access token — a password made for programs — a revocable credential with a fixed set of scopes, used in place of a password.
4. Scope — the list of things a token is allowed to do — a named permission such as repo, workflow or read:org attached to a token.
5. refs/pull — GitHub’s storage for pull request heads — read-only refs created by GitHub, visible to git ls-remote, not creatable by git.

40.7 What GitHub adds that plain git does not have#

PLAIN40.7.1 in simple words#

1. Git has exactly one job: move history between repositories, exactly.
2. It has no idea what a pull request is. No idea what a review is. No idea what an issue, a label, a release, a package or a build is.
3. Every one of those is added by the website you happen to be using.
4. Here is the whole list, in plain words, one item per line.
5. Pull requests. A written request saying “please take my branch into yours”, with a conversation attached to it.
6. Code review. Named people reading the change and marking it approved, or asking for changes, or just commenting.
7. Branch protection. A rule saying “nobody may push straight into main”.
8. Required checks. A rule saying “the tests must pass before the merge button is allowed to work”.
9. CODEOWNERS. A file listing which people must review which folders.
10. Issues and projects. A shared to-do list, and a board of cards.
11. Releases. A named point in history with files attached, such as a compiled program somebody can download.
12. Packages. A place to publish libraries that other projects install.
13. Actions. Computers that run your tests and builds when something happens, and keep the logs.
14. Two programming interfaces, REST and GraphQL. Ways for programs to ask GitHub questions and give it orders.
15. Permissions. Accounts, organizations, teams, roles and tokens.
16. Not one of those is part of git. Git ships none of them, and would work exactly the same if GitHub had never existed.

PLAIN40.7.2 a picture in your head#

1. Think of a shared workshop where people build furniture.
2. Git is one machine in the corner. It copies a design exactly, down to the last measurement, and hands the copy to somebody else.
3. That machine is superb at that job and does nothing else. It has no opinion about who you are or whether the design is any good.
4. The workshop around the machine is everything else. A pass reader on the door. A noticeboard of jobs to do. A sign-off sheet outside the paint room saying two people must approve the colour. An overnight test rig. A shelf of finished pieces with labels on them.
5. You can take the copying machine home and still build furniture.
6. You cannot take the noticeboard, the sign-off sheets, the pass reader or the test rig. They belong to the building.
7. GitHub is the building. Git is the machine.

Where this comparison breaks:

8. The building also stores the designs, so day to day the two feel like one thing. The split is in the protocol, not in the floor plan.
9. And the sign-off sheet is not decoration. On GitHub, branch protection really does stop the push at the door. It is a lock, not a note.

PLAIN40.7.3 a worked example#

1. A pull request has two halves. One half is refs, which git can see. The other half is a database row, which git cannot see.
2. Here is the visible half, read live with plain git, no account, no token.

$ git ls-remote https://github.com/cli/cli 'refs/pull/10002/*'
b4b8e8a13bb99bae479929048ea5f64808d0bc32 refs/pull/10002/head

$ git ls-remote https://github.com/cli/cli 'refs/pull/*/head' \
    | wc -l
4981

$ git ls-remote https://github.com/cli/cli 'refs/pull/*/merge' \
    | wc -l
166

3. Read those two numbers. 4,981 pull requests have a head ref, which is the commit the author is proposing.
4. Only 166 have a merge ref. That is GitHub’s pre-computed trial merge, kept only while a pull request is open and can be merged cleanly.
5. Now the invisible half. Git cannot answer any of these, at any price: who opened it, what the title says, who approved it, which comment sits on which line, whether the tests passed, whether the merge button works.
6. Proof by absence. Ask git for every command it has that mentions requests or reviews.

$ git help -a | tr ' ' '\n' | grep -iE 'request|review|issue'
request-pull

7. One result, and it is not what you think. git request-pull has existed since 2005 and prints an email for you to send to a maintainer by hand.

$ git request-pull main /tmp/mg/base feature/login
The following changes since commit cd18e975:

  Base commit (2026-08-13 07:43:39 +0000)

are available in the Git repository at:

  /tmp/mg/base feature/login

for you to fetch changes up to 00ee7d7c:

  Fix login typo (2026-08-13 07:43:39 +0000)

Reader (2):
      Start login
      Fix login typo

 login.py | 2 ++
 1 file changed, 2 insertions(+)

8. That is the true ancestor of the pull request. It produces text for a mailing list. It has no state, no approval, no button and no thread.
9. GitHub’s version added the state, the approval, the button and the thread. That addition is the product.

PLAIN40.7.4 what is really happening inside#

1. A pull request is three things glued together.
2. One: two ref names. A base, meaning where you want the work to land, and a head, meaning the work. Both are ordinary git refs.
3. Two: a row in a database, holding a number, a title, a body, a state (open, closed or merged) and an author.
4. Three: a conversation. Comments, review states, requested reviewers, labels, and links to issues.
5. When you press Merge, GitHub runs an ordinary git merge on its own server, into the base branch, using whichever strategy you picked.
6. Review states are values on that database row: APPROVED, CHANGES_REQUESTED, COMMENTED, DISMISSED, PENDING.
7. Branch protection is enforced by GitHub’s own pre-receive hook. Earlier in this chapter we wrote a pre-receive hook by hand that rejected a push. GitHub runs a much larger version of the same thing.
8. A required status check is a record attached to a commit SHA, not to a branch. Something posts “the tests passed for this SHA” through the API.
9. The protection rule then asks one question: does the head SHA of this pull request carry a passing record with the required name?
10. That is exactly why, in the reader’s own session, checking CI by run ID against the exact head SHA was stronger than trusting a green tick beside a branch name. The tick is drawn next to a name, and names move. The record is attached to a SHA, and a SHA never moves.
11. CODEOWNERS is the one hybrid. The file is a real blob inside your repository, so git stores and versions it. The rule “these owners must approve” is GitHub’s, and git ignores the file completely.
12. Actions is a scheduler plus a fleet of machines. A push fires an internal event, a workflow file in the repository says what to run, and a runner picks up the job.
13. Releases and packages are file storage with a database index in front. The attached files are not git objects and are not in the history.
14. So the rule is short: refs and objects are git; anything with an opinion is GitHub.

TECHNICAL40.7.5 the engineer’s version#

1. The classification. “Truth lives in” means where the authoritative copy is.

2. Dates, so you can see that every GitHub feature has a birthday and git has none of them.

3. Rate limits, from GitHub’s published documentation, current as of 2026. REST, unauthenticated: 60 requests per hour per IP address. REST, with a personal access token: 5,000 per hour. GitHub Apps and OAuth apps owned by a GitHub Enterprise Cloud organization: 15,000 per hour. GraphQL is budgeted in points, 5,000 points per hour for a user token.
4. The API keeps its own books, and will tell you so:

$ curl -s https://api.github.com/rate_limit
{ "resources": {
    "core":    { "limit": 15000, "remaining": 15000 },
    "search":  { "limit": 30,    "remaining": 30 },
    "graphql": { "limit": 5000,  "remaining": 5000 } } }

5. Honest note on that output: the request above ran through a proxy that attached an application credential, which is why core shows 15,000 and not 60. The number you see depends on how you authenticated.
6. Git has no equivalent of any of that, because git has no accounts. There is nothing in the git protocol to rate-limit against.
7. The permission model, as of 2026. Repository roles: Read, Triage, Write, Maintain, Admin. Organization roles: Member, Moderator, Billing manager, Owner, plus custom roles on paid plans. Teams can nest, and effective permission is the union of every grant a user receives.
8. Token types. Classic personal access tokens carry coarse scopes such as repo, workflow, read:org and admin:repo_hook. Fine-grained personal access tokens, generally available since 2022, are limited to chosen repositories, carry per-resource read or write permissions, and must have an expiry date.
9. The reader’s own session shows why the scope list is not decoration. One push was refused because the token lacked the workflow scope and the push modified a CI workflow file. Git had already packed and sent the objects. GitHub’s server-side rule refused the ref update.
10. GraphQL exposes fields with no git equivalent at all. The reader saw mergeStateStatus values CLEAN, BLOCKED and UNKNOWN; the full set also includes BEHIND, DIRTY, DRAFT, HAS_HOOKS and UNSTABLE.
11. Git cannot compute any of those values, because each depends on a rule git does not know exists.
12. Scale figures published by GitHub are marketing figures, not audited ones. Roughly 100 million developers were reported in early 2023, about 150 million for May 2025, and the repository count was reported to pass one billion in June 2025. Treat all three as approximate and dated.

WORDS40.7.6 remember these#

1. Pull request — a request to merge one branch into another, with a discussion — a GitHub database object referencing a base ref and a head ref, exposed to git only as refs/pull/N/head.
2. Review state — the verdict a reviewer left — one of APPROVED, CHANGES_REQUESTED, COMMENTED, DISMISSED, PENDING.
3. Branch protection — a rule stopping direct pushes to a branch — a server-side policy implemented in GitHub’s pre-receive path.
4. Required status check — the tests must pass before merging — a named check record that must exist and be successful for the head SHA.
5. Check run — one reported result for one commit — an API object attached to a SHA with a name, status, conclusion and output.
6. CODEOWNERS — a file saying who owns which folders — a versioned path pattern file whose enforcement is entirely the forge’s, not git’s.
7. Fork — your own server-side copy of somebody’s repository — a GitHub database relationship plus a repository that may share object storage.
8. GraphQL API — one endpoint you send a shaped query to — GitHub’s typed schema at a single endpoint, budgeted in points rather than requests.
9. mergeStateStatus — GitHub’s summary of whether merging is allowed — a GraphQL enum with values such as CLEAN, BLOCKED and BEHIND.
10. Merge queue — a line that tests changes in order before merging — a GitHub service that builds candidate merges and merges only what passes.

40.8 The exact boundary, using the reader’s own outage as evidence#

PLAIN40.8.1 in simple words#

1. During the reader’s outage one sentence describes everything: git kept working, and the GitHub API did not.
2. Committing worked. Branching worked. Merging worked. Rebasing worked. Reading old history worked. Tagging worked.
3. Pushing failed. Fetching failed. Looking at a pull request failed. Checking a CI run failed.
4. That is not luck and not a coincidence. It follows from two facts the book has already given you.
5. Fact one: a git command needs the network if and only if it must read from, or write to, another repository. Everything else is reading and writing files inside the .git folder, and files need no network.
6. Fact two: every GitHub-specific feature is a network service. There is no local copy of a pull request, an issue, a review or a build result.
7. Put the two together and the boundary is exact. The local half of git kept working. The remote half of git, and the whole of GitHub, did not.
8. Nothing here is special to GitHub. Point your remote at a self-hosted Forgejo and the same line falls in the same place.

PLAIN40.8.2 a picture in your head#

1. Imagine you keep a paper diary at home, and your office keeps a shared wall planner that everyone updates.
2. The phone line to the office is cut.
3. You can still write in your diary. Every page, any amount. The cut line does not touch your pen.
4. You can still read every old page, because they are in your hands.
5. You cannot see today’s wall planner, and you cannot change it.
6. Here is the sharp part. You can still look at the photo of the wall planner you took last Tuesday. It will show you Tuesday’s planner, confidently, and it will be out of date.
7. That photo is origin/main. During the outage the reader’s machine reported origin/main as 0a95cc8. That was a cached value from the last successful fetch, not a live fact about the server.

Where this comparison breaks:

8. A photo obviously looks like a photo. A remote-tracking ref does not. git log origin/main prints normally, with no warning and no note saying how old the information is.
9. And a wall planner suggests one true copy. Git has no true copy. During the outage origin/main was stale, but the reader’s own main was not somehow less real.

PLAIN40.8.3 a worked example#

1. Here is the boundary as a table. Read the third column as the reason, not as an excuse.

2. Every row above was reproduced in a sandbox for this chapter. The remote was pointed at 203.0.113.9, an address reserved by RFC 5737 for documentation, so nothing could answer. The outage was real, not faked with a flag.

git log --oneline -1        rc=0  f657694 First commit
git branch -a               rc=0  * main
git switch -c feature/x     rc=0
git commit --allow-empty    rc=0
git tag v0.9                rc=0
git diff HEAD~1 --stat      rc=0
git rev-parse origin/main   rc=0  f6576945ec22d250a17a6fe3...
git status -sb              rc=0  ## feature/x

3. Note line seven. git rev-parse origin/main succeeded and printed a full SHA while the server was unreachable. That is the cached photo.
4. In the same second, on the same repository, the network commands:

$ git ls-remote origin
ssh: connect to host 203.0.113.9 port 22: Connection timed out
fatal: Could not read from remote repository.

$ git fetch origin
ssh: connect to host 203.0.113.9 port 22: Connection timed out
fatal: Could not read from remote repository.

$ git push origin main
ssh: connect to host 203.0.113.9 port 22: Connection timed out
fatal: Could not read from remote repository.

5. Three failures, all identical, all at the same layer: the connection never opened. Git never got as far as speaking git.
6. That is the same shape as the reader’s own evidence. curl -v https://github.com printed Trying 20.207.73.82:443... and then timed out after 15 seconds with no response of any kind. No RST, no ICMP unreachable, silence.
7. What that proves: nothing on that path answered. What it does not prove: who dropped it. The same request succeeded instantly over mobile data, which is consistent with a silent drop somewhere on the fixed-line path, and not consistent with GitHub being down.

PLAIN40.8.4 what is really happening inside#

1. Take the three cases apart, one at a time.
2. Case one, a local git command. git commit reads the index file, writes blob, tree and commit objects under .git/objects, and rewrites one 41-byte ref file. Zero sockets are opened. There is nothing to fail.
3. Case two, a remote git command. git push must change a ref on another machine. It resolves a name, opens TCP port 22 or 443, authenticates, runs the remote helper, and negotiates. Any broken step ends it.
4. Case three, a GitHub feature. Opening a pull request is an HTTPS request to api.github.com. There is no offline path, because there is no local copy of the data and no local code that understands it.
5. Now the important consequence for the reader. Because case one never touches a network, work does not stop. The reader committed to branches all through the outage and pushed them later, when the path came back.
6. Nothing had to be redone. The commits made during the outage were already complete, already hashed, already in the object database. Pushing later just copied objects that already existed.
7. That is the entire practical payoff of the distributed design, and the reader lived it.
8. One more piece from the same session. Twice, a push failed with send-pack: unexpected disconnect while reading sideband packet.
9. That message is different from the ones above. It means the connection was established, authentication passed, objects were sent, and the link died while git was reading the server’s reply.
10. So the error tells you the answer never arrived. It tells you nothing about whether the server acted before the link died.
11. Both times the operation had not been applied. But that could not be known from the message. It had to be re-checked with git ls-remote origin, which asks the server what its refs are now.
12. The rule to carry away: a failure while reading a response is not evidence that the write did not happen. Only a fresh query is evidence.

TECHNICAL40.8.5 the engineer’s version#

1. What each operation actually needs on the wire.

2. Note what that table means. Everything except plain SSH shares one port and often one path. When that path is silently dropped, git over HTTPS and the whole API fail together, which is what the reader observed.
3. Plain SSH on port 22 is a different destination and can survive when HTTPS does not. In the reader’s session one transport worked while the other failed, in the same minute, on the same machine.
4. That asymmetry is a diagnostic, not a fix. If SSH works and HTTPS does not, the fault is path-specific or port-specific, not a dead server.
5. Failure taxonomy, with the exact signal for each:

6. Only the last row is ambiguous, and it is the one that bites. The git protocol sends the report-status response inside sideband channel 1 during receive-pack. If the stream dies there, the client has no report.
7. The correct recovery is a re-query, never a retry decided by guesswork:

$ git ls-remote origin refs/heads/main
0a95cc8a5f0b...   refs/heads/main

8. Compare that SHA with your local main. If they match, the push landed. If not, it did not. That is a fact from the server, not an inference.
9. There is a second reason the re-query matters. A push is not atomic across refs by default. Pushing several refs at once can leave some applied and some not, unless you use --atomic, added in Git 2.4 (April 2015), which makes the whole push succeed or fail as one unit on servers that support it.
10. Chapter 37 gave the local versus remote split. This chapter adds the second axis. Together they explain every row of the table in 40.8.3:

                    | needs no network | needs the network
  --------------------------------------------------------
   pure git          | commit, log,     | fetch, push,
                     | merge, rebase,   | clone, ls-remote
                     | tag, bisect      |
  --------------------------------------------------------
   forge feature     | (nothing)        | PR, review, CI,
                     |                  | issues, releases

11. The bottom-left cell is empty, and that emptiness is the point. There is no such thing as an offline pull request.
12. Tools that appear to break this rule cache data locally, which is the same stale-photo problem as origin/main and should be treated the same way.

WORDS40.8.6 remember these#

1. Local operation — anything git can do with your own files — a command whose entire working set is the object database, index and refs on this disk.
2. Remote-tracking ref — your last photo of the server — a ref under refs/remotes/ updated only by fetch, never live.
3. Silent drop — a packet vanishes with no reply — no RST and no ICMP time-exceeded, observed as a connection timeout.
4. Sideband — the channel carrying the server’s messages — multiplexed stream inside the pack protocol, where the report-status reply travels.
5. Re-query — ask again instead of guessing — using git ls-remote to learn the server’s current refs after an ambiguous failure.
6. Atomic push — all refs or none — git push --atomic, added in Git 2.4 (April 2015), requiring server support.

40.9 The self-hosted alternatives#

PLAIN40.9.1 in simple words#

1. If you do not want to use GitHub, you have four honest choices. Here they are from heaviest to lightest.
2. GitLab Community Edition. Close to GitHub in features, and heavy to run.
3. Gitea, or its community fork Forgejo. Most of what a small team needs, in a program small enough for a Raspberry Pi.
4. Bitbucket Data Center from Atlassian. Bought mainly by companies that already run Jira and want one supplier.
5. Plain SSH remotes with no web interface at all. A machine, an account, and git. Nothing else.
6. All four store bare repositories the same way and speak the same protocol. Every difference is in the layers above git.
7. What you gain by self-hosting: control over the rules, privacy for the code, and no lock-in to one company’s features or prices.
8. What you lose is simpler than people admit. You become the operator. Backups are yours. Uptime is yours. Upgrades and security patches are yours. At three in the morning, it is you.

PLAIN40.9.2 a picture in your head#

1. Compare eating in a canteen with cooking at home.
2. In the canteen, somebody else buys the food, cooks it, washes up and keeps the doors open. You accept their menu and pay their price.
3. Cooking at home, you choose everything. You know exactly what is in the food. Nobody can change the price or close the kitchen.
4. But you also shop, cook, wash up and mend the cooker. Every day. Including days you are ill and days you are away.
5. Most people cook at home for four and hire a caterer for four hundred.
6. Self-hosting scales the same way. It is genuinely easy for a small trusted group and genuinely hard for a large public project full of strangers.

Where this comparison breaks:

7. A broken cooker ruins one dinner. A broken git server does not lose your history at all, because every clone is a complete copy. The stakes are lower than the picture suggests.
8. What an outage really costs you is the meeting point, plus the metadata: issues, reviews and build results. Those are the parts with no second copy on anybody’s laptop.

PLAIN40.9.3 a worked example#

1. Here is the smallest useful server anyone can build, and it needs no software beyond git and an SSH daemon.
2. Make one account on a machine, called git. Set its login shell to git-shell, a program that ships with git and refuses everything except git’s own transport commands.
3. Put each developer’s public key in that account’s authorized_keys file.
4. Proof that git-shell really is a lock and not a suggestion:

$ git-shell -c "ls -la"
fatal: unrecognized command 'ls -la'

5. And proof that the git commands still work through it. This is the real first bytes of a ref advertisement, wrapped to fit the page:

$ git-shell -c "git-upload-pack '/srv/git/proj.git'"
01032304fd03...16d6 HEAD multi_ack thin-pack side-band
side-band-64k ofs-delta shallow deepen-since deepen-not
no-progress include-tag multi_ack_detailed
symref=HEAD:refs/heads/main object-format=sha1
agent=git/2.43.0
0046a3bfdce1...3a refs/heads/feature/login

6. That is the whole server. Somebody who logs in with that key can clone, fetch and push, and cannot get a shell, list files or read other accounts.
7. Cost of running it: git itself. Measured in the sandbox for this chapter, peak memory of the git process during real work:

git clone (11,772 commits)   0.58 s   peak RSS 42.1 MB
git gc    (blobless clone)   0.93 s   peak RSS 39.2 MB
git status (1,356 files)     0.29 s   peak RSS 16.5 MB
git log --oneline (all)      0.17 s   peak RSS 14.8 MB

8. Tens of megabytes, for a few seconds, per operation. That is why a plain SSH git server runs happily on the smallest machine you can rent.
9. Now compare that with what each option needs before it will start at all.

10. And what you actually get for that memory.

PLAIN40.9.4 what is really happening inside#

1. The four options differ in exactly one way: how many processes must be running before a developer can do anything.
2. Plain SSH: one process, and only while somebody is connected. sshd accepts the connection, git-shell checks the command, and git-upload-pack or git-receive-pack does the work and exits.
3. Nothing runs between pushes. There is no database, no queue, no web server and no cache. That is why it costs almost nothing.
4. Gitolite adds a single Perl program in front. It reads one configuration repository that lists users, repositories and per-branch rules, and it installs an update hook to enforce them.
5. Gitea and Forgejo are one compiled program written in Go. It contains the web server, the issue tracker, the review system and its own SSH server. Storage is SQLite for small installations, or PostgreSQL or MySQL.
6. GitLab is not one program. A standard installation runs a Ruby on Rails application, background job workers, PostgreSQL, Redis, a reverse proxy, a metrics stack, and Gitaly, which is GitLab’s own service that every other part must call in order to touch git at all.
7. That list is the entire reason for the memory difference. It is not that GitLab is badly built. It is that it is many services, and each one has a floor.
8. In every case the bytes on the wire are identical. A developer cloning from plain SSH and a developer cloning from GitLab run the same client, speak the same protocol and get the same objects.

TECHNICAL40.9.5 the engineer’s version#

1. Provenance and licensing, which matter more than feature lists when you are choosing something to depend on for ten years.

2. The Gitea and Forgejo split is a governance story, not a technical one. Gitea’s trademarks and operations were transferred to a company in 2022, contributors asked for community control, the request was refused, and Forgejo was forked in December 2022. Codeberg e.V., a German non-profit association, leads it.
3. Forgejo stopped tracking Gitea after Gitea 1.21, in February 2024, and became a hard fork. It relicensed from MIT to the GNU General Public Licence in August 2024. Code written before that remains MIT.
4. Bitbucket was founded in 2008 by Jesper Nohr, acquired by Atlassian on 29 September 2010, and dropped Mercurial support in July 2020. Support for the self-hosted Bitbucket Server product ended on 15 February 2024. Self-hosting now means Bitbucket Data Center, which is licensed and priced for larger organizations.
5. That last date is a lesson in its own right. A self-hosted product can be discontinued by its vendor. Self-hosting removes the hosting dependency; it does not by itself remove the vendor dependency. Only an open licence with a living community does that.
6. Hardware, from the projects’ own documentation as of 2026. Gitea states that “2 CPU cores and 1GB RAM is typically sufficient for small teams/projects” and that a Raspberry Pi 3 is enough for small workloads. GitLab’s single-node baseline is 8 vCPU and 16 GB of RAM, with 8 GB possible only with extra configuration. Older GitLab documentation quoted 4 GB, so check the version-specific page rather than trusting a number you remember.
7. That is a factor of roughly sixteen in memory between Gitea and GitLab for the same core job. It is the single most useful figure in this section.
8. Access control on a plain SSH server, in increasing order of precision: UNIX file permissions on the repository directory; one shared git account with git-shell; Gitolite for per-repository and per-branch rules; and server-side pre-receive or update hooks for anything else.
9. Read-only web browsing without a full forge: cgit, a C program, or gitweb, which ships with git itself. Both render a repository as HTML and neither has accounts, issues or reviews.
10. Honest trade-off summary. Self-hosting is a good decision when the group is small and trusted, when the code must not leave your control for legal reasons, when your costs at a hosted provider have become large, or when you need rules a hosted provider will not give you.
11. It is a poor decision when your project depends on drive-by contributions from strangers, when nobody on the team wants to be on call, or when the real cost of an engineer’s time doing operations exceeds the subscription you were avoiding.
12. Experts disagree about the middle ground. One camp argues that a Forgejo instance is so cheap to run that any team should own its own. The other argues that discovery, and the fact that contributors already have GitHub accounts, outweighs everything else for public projects. Both are right about different projects, and the deciding question is whether strangers need to contribute.

WORDS40.9.6 remember these#

1. Self-hosting — running the server yourself — operating the repository host, its storage, its backups and its upgrades on infrastructure you control.
2. git-shell — a login that can only do git — a restricted shell shipped with git that accepts only git-upload-pack, git-receive-pack and git-upload-archive.
3. Gitolite — a rules file in front of a git server — a Perl layer using one configuration repository and an update hook for per-branch access control.
4. Forge — a website around repositories — the general term for GitHub, GitLab, Gitea, Forgejo and Bitbucket.
5. Gitaly — GitLab’s git service — the component every other GitLab process must call to read or write a repository.
6. Vendor lock-in — hard to leave — dependence on features whose data has no portable export, as opposed to the commit history, which always exports.
7. Operator — the person who is responsible when it breaks — the role you take on the moment you self-host, covering backups, uptime and patching.

40.10 Migrating and mirroring#

PLAIN40.10.1 in simple words#

1. Two jobs get confused with each other, so separate them first.
2. Mirroring means keeping a second copy that stays in step with the first. Both live. You did not leave anywhere.
3. Migrating means moving to a new home and shutting the old one, or leaving it as an archive.
4. For the git part, both jobs are easy and take one or two commands.
5. For everything else, both jobs are hard, and some of it simply cannot be moved at all.
6. Here is the sentence to remember, and it is the honest warning of this whole section: your commit history is portable, and almost nothing else is.
7. Commits, branches, tags and file contents move perfectly, byte for byte, with identical hashes.
8. Issues, pull requests, review approvals, build history, release downloads, stars, webhooks and protection rules do not move by themselves, because none of them is in git.

PLAIN40.10.2 a picture in your head#

1. Think about moving house.
2. Your books, clothes and furniture go in boxes. They arrive exactly as they left. Nothing about them changes.
3. The fitted kitchen does not come. The neighbours do not come. Your children’s school place does not come. The postal redirection has to be arranged, lasts a while, and then stops.
4. Git history is the boxes. Everything the website added is the fitted kitchen and the school place.
5. And notice which one people actually worry about on moving day. Not the boxes.

Where this comparison breaks:

6. Moving house removes things from the old place. Cloning does not. After you migrate, the old repository still exists, complete, and can still serve.
7. That is why a git migration is unusually safe. You can run both for weeks and switch back with one git remote set-url.

PLAIN40.10.3 a worked example#

1. First, what --mirror actually gives you. This is a real run.

$ git clone --mirror proj mirror.git

$ git -C mirror.git config --get remote.origin.mirror
true

$ git -C mirror.git config --get remote.origin.fetch
+refs/*:refs/*

$ git -C mirror.git show-ref
a3bfdce1... refs/heads/feature/login
2304fd03... refs/heads/main
fb32611e... refs/tags/v1.0

2. Compare that with an ordinary clone of the same repository:

$ git clone proj plain
$ git -C plain show-ref
2304fd03... refs/heads/main
2304fd03... refs/remotes/origin/HEAD
a3bfdce1... refs/remotes/origin/feature/login
2304fd03... refs/remotes/origin/main
fb32611e... refs/tags/v1.0

3. Read the difference carefully. The ordinary clone has one local branch and puts the rest under refs/remotes/origin/. The mirror keeps every ref at its original name.
4. That is the whole point of a mirror: it is a copy of the ref namespace, not a copy for working in.
5. Second, keeping a mirror up to date. One command, and --prune matters:

$ git -C mirror.git remote update --prune
From /tmp/mir/up
 - [deleted]   (none) -> hotfix/urgent

6. Third, pushing one commit to two servers with one command. Add a second push URL to the same remote:

$ git remote set-url --add --push origin /tmp/lab3/srv1.git
$ git remote set-url --add --push origin /tmp/lab3/srv2.git
$ git remote -v
origin  /tmp/lab3/srv1.git (fetch)
origin  /tmp/lab3/srv1.git (push)
origin  /tmp/lab3/srv2.git (push)

$ git push origin main
To /tmp/lab3/srv1.git
   99b5afc..a98de55  main -> main
To /tmp/lab3/srv2.git
 * [new branch]      main -> main

7. One git push, two servers, and both now hold the identical SHA a98de5517c0e80180ed2a1ccd6795d47f155beb5.
8. Fourth, the trap. A mirror push is a replica, not a backup. Watch what happens when a branch is deleted upstream:

$ git -C up branch -D hotfix/urgent
Deleted branch hotfix/urgent (was d9a902a).

$ git -C mir.git remote update --prune
 - [deleted]  (none) -> hotfix/urgent

$ git -C mir.git push --mirror /tmp/mir/dest.git
To /tmp/mir/dest.git
 - [deleted]  hotfix/urgent

9. The deletion was faithfully copied to the second server. A mirror reproduces mistakes exactly as well as it reproduces work. Keep that in mind for the backup section.

PLAIN40.10.4 what is really happening inside#

1. git clone --mirror does three things and nothing magical.
2. It makes the new repository bare, so there is no working tree.
3. It sets remote.origin.mirror to true.
4. It sets the fetch refspec to +refs/*:refs/*, meaning “take every ref, of every kind, and store it under the same name, forcing where needed”.
5. Compare that with the ordinary clone refspec, +refs/heads/*:refs/remotes/origin/*, which takes only branches and files them under a separate prefix.
6. That single line of configuration is the entire difference between a copy you work in and a copy that stands in for the original.
7. On the push side, git push --mirror sends every local ref and deletes every destination ref that you do not have. That is why the deletion propagated above.
8. One consequence surprises everybody who mirrors a GitHub repository. +refs/*:refs/* includes GitHub’s pull request refs. Here is a real mirror clone of a public repository:

$ git clone --mirror https://github.com/octocat/Hello-World hw.git
$ git -C hw.git show-ref | wc -l
3439
$ git -C hw.git show-ref | grep -c refs/pull/
3436

9. Three branches, and three thousand four hundred and thirty-six pull request refs. If you then push that mirror to a new host, you are trying to create three thousand refs the new host neither wants nor allows.
10. The fix is to fetch or push only what you mean, using an explicit refspec such as +refs/heads/*:refs/heads/* and +refs/tags/*:refs/tags/*.

TECHNICAL40.10.5 the engineer’s version#

1. What actually travels, and what does not.

2. The wiki row is worth knowing precisely: GitHub stores a repository wiki as a separate git repository, reachable at the repository address with .wiki.git on the end. So the wiki does migrate, but only if you deliberately clone it as a second repository.
3. A correct migration of the git part, in full:

git clone --mirror https://github.com/you/proj.git
cd proj.git
git remote set-url --push origin git@newhost:you/proj.git
git push --prune origin \
    '+refs/heads/*:refs/heads/*' \
    '+refs/tags/*:refs/tags/*'

4. That is deliberate rather than clever. It moves branches and tags, prunes what is gone, and does not attempt to move refs/pull/*.
5. Verify the move by comparing ref lists rather than by looking at a web page:

OLD=https://old/proj.git
NEW=git@newhost:you/proj.git
git ls-remote --heads --tags $OLD | sort > old.txt
git ls-remote --heads --tags $NEW | sort > new.txt
diff old.txt new.txt && echo "identical"

6. Metadata is a separate project with separate tools, and no standard format exists. As of 2026 the realistic options are:

7. For a do-it-yourself export the practical route is the API. The GitHub CLI can page through issues and write JSON, which you then store in the repository itself so it is versioned with the code:

gh api --paginate repos/OWNER/REPO/issues?state=all \
    > issues.json
gh api --paginate repos/OWNER/REPO/pulls?state=all \
    > pulls.json

8. GitHub also offers migration archives through its migrations API for organizations, producing a downloadable archive of repositories and associated metadata. Availability and contents depend on your plan, so check before you rely on it.
9. Now the honest warnings, which matter more than the commands.
10. Issue numbers will not survive. Every forge assigns its own numbering, so “fixes #42” in a commit message will point at the wrong thing after migration. The commit message is immutable, so this cannot be corrected without rewriting history.
11. Cross-references, mentions of users, and links between issues and pull requests are stored as identifiers that mean nothing on the new host.
12. Timestamps and authorship on imported issues usually become “imported by the migration account on the day of import” unless the importer has special support, because most APIs will not let you post as someone else.
13. Attachments in comments are files on the old host, served from the old host’s storage, and vanish when the repository does.
14. Build history and logs are effectively unmovable. Treat them as data with a retention period, not as a permanent record.
15. So the practical rule for anything you truly cannot lose: put it in the repository. A decision written in a Markdown file in docs/ migrates perfectly, forever, because it is a blob. The same decision written in an issue comment is the platform’s, not yours.

WORDS40.10.6 remember these#

1. Mirror clone — a copy of every ref under its own name — git clone --mirror, which sets remote.origin.mirror=true and the refspec +refs/*:refs/* on a bare repository.
2. Mirror push — make the far side match me exactly — git push --mirror, which creates, updates and deletes refs to match the local set.
3. Prune — remove refs the other side no longer has — --prune on fetch, remote update or push.
4. Second push URL — one remote, two destinations — extra push URLs added with git remote set-url --add --push.
5. Migration — moving to another host — copying refs and objects with git, then copying metadata with each platform’s own tools, imperfectly.
6. Platform metadata — everything not in git — issues, reviews, runs, releases and rules, which have no interchange standard.

40.11 Working with others, practically#

PLAIN40.11.1 in simple words#

1. There are only two ways to arrange a team around a repository.
2. The shared repository model. Everyone can push to the same repository, and they push branches, not straight to main.
3. The fork model. Nobody outside the core team can push at all. Outsiders make their own server-side copy, push there, and ask for their work to be taken.
4. The shared model is for people you trust: a company team, a class group, a startup.
5. The fork model is for strangers: any public project where anyone in the world may send a change.
6. Both end at the same place, a pull request, so they look identical in the web interface. The difference is only in who may write where.
7. On top of that choice sits a second one: how branches are organized over time. There are three well-known answers, and they have real trade-offs.
8. And on top of that sits one rule everybody agrees on: protect main.

PLAIN40.11.2 a picture in your head#

1. Think of a shared document at a college.
2. The shared repository model is a group project. Everyone has edit rights, but the agreement is that you work on your own page and ask before merging your page into the final report.
3. The fork model is a published book. You cannot write in the book. You make a photocopy, mark it up, and post your marked copy to the editor, who decides whether to take your change.
4. A publisher does not give edit rights to every reader. That would be absurd, and it is exactly why open-source projects use forks.
5. A group of four friends does not photocopy the report for each other. That would be pointless ceremony, and it is why company teams use the shared model.

Where this comparison breaks:

6. A photocopy goes out of date the moment the book changes. A fork does not have to. You can add the original as a second remote and pull from it, so your copy stays current.
7. And the editor in this picture merges by hand. A pull request merge is performed by the server, exactly, using git.

PLAIN40.11.3 a worked example#

1. Here is the fork-and-pull-request path drawn out. Only two steps in it are git; the rest are the website.

   upstream/proj        you cannot write here
        |
        |  "Fork"  (GitHub action, not a git command)
        v
   you/proj             your copy on the server, you own it
        |
        |  git clone            (git)
        v
   laptop: main -> feature/login
        |
        |  git push origin feature/login   (git)
        v
   you/proj  feature/login
        |
        |  "Open pull request"  (GitHub action)
        v
   upstream: refs/pull/N/head -> review -> merge

2. In the shared repository model, the first two boxes disappear. You clone upstream/proj directly, push feature/login to it, and open the pull request from there.
3. Now branch names. Git’s rules are stricter than people expect, and this is real output from git check-ref-format:

OK   feature/login
OK   feature/ABC-123-add-login
OK   release/1.0
OK   Feature/Login
BAD  feature/add login
BAD  feature//login
BAD  feature/login.lock
BAD  feature/.hidden
BAD  feature/login~1
BAD  feature/login^

4. So a space is illegal, a double slash is illegal, a component starting with a dot is illegal, an ending of .lock is illegal, and the characters ~ ^ : ? * [ \ are illegal.
5. Feature/Login is legal but a bad idea, because on macOS and Windows the filesystem usually ignores case and two branches differing only in case will collide.
6. And here is the trap that catches every team once:

$ git branch feature/login
$ git branch feature
fatal: cannot lock ref 'refs/heads/feature':
 'refs/heads/feature/login' exists;
 cannot create 'refs/heads/feature'

7. A loose ref is a file on disk, so feature cannot be both a file and a directory. Pick a prefix scheme and never use a bare prefix as a branch.
8. Finally, the three ways a pull request can land. All three of these are real graphs from the same starting point:

1. merge commit (--no-ff)
*   0685fe9 Merge pull request: login
|\
| * 00ee7d7 Fix login typo
| * 046ed60 Start login
* | ec53eed Unrelated work on main
|/
* cd18e97 Base commit

2. squash merge
* 23a1f88 Add login (#42)
* ec53eed Unrelated work on main
* cd18e97 Base commit

3. rebase then fast-forward
* 12a1676 Fix login typo
* 0ae1582 Start login
* ec53eed Unrelated work on main
* cd18e97 Base commit

9. Look at what each one costs. The merge commit keeps every step and the shape. The squash keeps one tidy commit and throws away the steps. The rebase keeps the steps and throws away the shape, and it made new SHAs: 046ed60 became 0ae1582.
10. That last point is the reader’s own experience. Rebasing a local branch onto a moved main changed the branch’s SHA, because rebase does not move commits. It creates new commit objects with new parents, and therefore new hashes.

PLAIN40.11.4 what is really happening inside#

1. A fork, mechanically, is a repository on the server plus a database row saying “this came from that”. Git has no idea it happened.
2. When you fork on GitHub, the new repository may share object storage with the original, which is why forking a huge repository is instant. That is an implementation detail of one product, not part of git.
3. A pull request from a fork works because both repositories are on the same server, so the server can read your head commit directly.
4. Now the three branching models, described by what they actually do to the graph.
5. Git flow, published by Vincent Driessen on 5 January 2010, keeps two permanent branches, main and develop, plus feature/*, release/* and hotfix/*. Work flows feature into develop, develop into a release branch, release into main and back into develop.
6. GitHub flow, published by Scott Chacon on 31 August 2011, keeps one permanent branch. Branch off main, open a pull request, get review, merge, deploy. There is no develop branch and no release branch.
7. Trunk-based development keeps one permanent branch and adds a rule about time: branches live hours, not days, and unfinished work is hidden behind feature flags rather than hidden on a branch.
8. Notice what the three actually disagree about. Not tooling. They disagree about how long a change may live away from the mainline.
9. Long-lived branches are the cost. Every day a branch lives, the mainline moves under it, and the merge gets harder. This is not an opinion; it is what a graph does when two lines diverge.
10. Protecting main is the mechanism that makes any of the three safe. A protected branch cannot be pushed to directly, cannot be force-pushed and cannot be deleted, and merges into it require whatever checks you set.
11. Without protection, all three models are just polite suggestions, exactly like a client-side hook.

TECHNICAL40.11.5 the engineer’s version#

1. The two collaboration models compared on the things that actually differ.

2. The last row is a real security point. A workflow triggered by a pull request from a fork must not be given repository secrets, or any stranger could exfiltrate them. GitHub’s pull_request event withholds secrets from fork pull requests for exactly this reason, while pull_request_target runs with them and is dangerous if misused.
3. The three branching models, with honest trade-offs.

4. Driessen added a note to his own 2010 article in March 2020 saying that if your team does continuous delivery you should adopt a much simpler workflow such as GitHub flow, and that git flow still fits software that is explicitly versioned or must support several versions in the wild.
5. That is the fairest summary available, and it comes from the author of the model people argue about.
6. The research position: the 2018 book Accelerate, by Nicole Forsgren, Jez Humble and Gene Kim, reports that high-performing teams have fewer than three active branches, branch lifetimes under a day, and rarely or never have code freezes.
7. Read that as a correlation found in survey data, not as a law. The critics’ point is fair too: trunk-based development requires strong automated tests and feature flags, and a team without them will simply break main faster.
8. Recommended day-one setup for a small team, concretely. This is a working configuration, not a philosophy.
9. Set the default branch name once per machine, so nobody argues:

git config --global init.defaultBranch main

10. That option arrived in Git 2.28, released 27 July 2020. GitHub changed the default branch for newly created repositories to main in October 2020. Older repositories were not renamed, which is why both names are still everywhere.
11. On the repository, turn on branch protection for main with these settings: require a pull request before merging, require at least one approving review, require status checks to pass, require branches to be up to date before merging, and block force pushes and deletions.
12. Choose one merge method and disable the others, so history has one shape. Squash merge is the usual choice for small teams because it makes main one commit per change and keeps git log --oneline readable.
13. Agree a branch prefix scheme and write it in the README. A workable one is feature/, fix/, chore/, docs/, plus the ticket identifier, giving names like feature/ABC-123-add-login.
14. Add a CODEOWNERS file even with three people. It costs two lines and it means nobody has to remember who reviews what.
15. Add one CI workflow that runs the tests on every pull request, and make it a required check. An unenforced test suite decays within weeks.
16. Turn on tag protection or use signed tags for anything you release, so a version number cannot be quietly repointed.
17. Verify CI by run identifier against the exact head SHA rather than by trusting a green tick beside a branch name. In the reader’s own session that distinction was the difference between believing a result and knowing one.
18. Do not add: a develop branch, a release branch per version, or a rule that two people must approve, until you have a concrete reason. Each of those is a real cost paid every day.

WORDS40.11.6 remember these#

1. Fork — your own server-side copy of a project — a forge-level repository copy plus a recorded parent relationship, invisible to git.
2. Upstream — the original project you forked from — conventionally a second remote named upstream alongside origin.
3. Shared repository model — everyone pushes branches to one repository — collaboration requiring write access, with policy enforced by protection rules.
4. Git flow — the two-permanent-branch model from 2010 — main plus develop with feature/*, release/* and hotfix/* branches.
5. GitHub flow — branch, pull request, merge, deploy — a single permanent branch model published in 2011.
6. Trunk-based development — branches measured in hours — mainline development with short-lived branches and feature flags instead of long branches.
7. Feature flag — a switch that hides unfinished work — a runtime condition allowing incomplete code to be merged but not executed.
8. Protected branch — main cannot be pushed to directly — a server-side rule blocking direct pushes, force pushes and deletion.
9. Squash merge — many commits become one — a merge that applies the combined change as a single new commit with one parent.
10. check-ref-format — the rules for legal ref names — git check-ref-format, which rejects spaces, .., ~, ^, :, ?, *, [, trailing .lock and leading dots.

40.12 Monorepos and large repositories#

PLAIN40.12.1 in simple words#

1. A monorepo is one repository holding many projects that could have been separate repositories.
2. Big companies use them for one reason above all others: one commit can change a library and every user of that library at the same time.
3. That removes the hardest problem in large systems, which is not writing code but keeping many versions of many pieces compatible.
4. You also get one place to search, one build configuration, and no arguing about which repository a change belongs in.
5. The price is size, and size hurts git in specific, measurable ways.
6. Clones get slow, because a clone copies the whole history by default.
7. Everyday commands get slow, because git checks every file it tracks.
8. Tools get slow or give up, because editors and search programs were not written with a million files in mind.
9. Git has four answers, and you can combine them: shallow clone, partial clone, sparse checkout, and Git LFS for large files.
10. And one weakness has no good answer: binary files. Git stores them, but it stores them badly, and this section will show exactly how badly.

PLAIN40.12.2 a picture in your head#

1. Think of a library that keeps every edition of every book it has ever held.
2. If you want to borrow one novel, the librarian hands you the entire archive on a trolley, including every draft of every book, and says “it is all yours now, take it home”.
3. That is git clone. It is wonderful once you have it and painful the first time.
4. A shallow clone is asking for only the newest edition of everything.
5. A partial clone is asking for the catalogue now and the actual pages later, only when you open a book.
6. A sparse checkout is taking the whole archive home but only unpacking the one shelf you work on.
7. Git LFS is the library keeping the heavy art books in a warehouse and giving you a slip of paper saying where each one is.

Where this comparison breaks:

8. Fetching a page later feels free in a library. In a partial clone it is a network round trip, and if you ask for thousands of pages one at a time it is dramatically slower than having taken the trolley. There are real numbers for this in the worked example, and they are worse than most people guess.

PLAIN40.12.3 a worked example#

1. Every number here was measured for this chapter against one real public repository, the GitHub CLI source, which has 11,772 commits and 1,356 tracked files. It is a medium repository, not a giant one.
2. First, the three ways to clone it.

3. Read the middle row carefully. The blobless clone still has every commit and every tree. It has 11,772 commits, same as the full clone. What it does not have is old file contents.
4. That is usually the right trade, because history browsing needs commits, and commits are small. File contents are what is big.
5. Object counts make the same point:

full clone     in-pack 90,539   size-pack 79.75 MiB
blobless       in-pack 64,891   size-pack 25.66 MiB
shallow  1     in-pack  1,776   size-pack 14.58 MiB

6. Second, sparse checkout, which is about the working tree rather than the history. This checks out one directory only:

$ git sparse-checkout set --cone pkg/cmd/pr
$ git ls-files | wc -l
1356
$ find . -path ./.git -prune -o -type f -print | wc -l
115

7. Git still tracks all 1,356 files. Only 115 exist on disk. The working tree shrank from 27 MB to 1.4 MB.
8. The index shows how it is done. 1,241 entries are marked skip-worktree and 115 are present.
9. Third, the honest cost of a partial clone. The same command in both:

10. That is not a typo. Reading the patch text of the last 300 commits took a fifth of a second with all objects present, and one minute forty-six seconds when each missing blob had to be fetched over the network.
11. The .git directory grew from 27 MB to 29 MB during that command, and 51 extra pack files appeared, each marked .promisor, meaning “fetched lazily from the promising remote”.
12. So partial clone is not free speed. It moves cost from clone time to the first command that needs old content.
13. Fourth, Git LFS. Here is what git actually stores when a 1 MiB binary file is tracked by LFS:

$ git cat-file -p HEAD:blob.bin
version https://git-lfs.github.com/spec/v1
oid sha256:5e20a633da26ff9bceb11f08d4671c234
38c49362cefce5042e8c77bb58bbdf9
size 1048576

$ git cat-file -s HEAD:blob.bin
132

14. The object in git is 132 bytes of text. The real megabyte lives outside the object database, under .git/lfs/objects, and is fetched by a separate program over a separate protocol.
15. Note the oid line. It is a SHA-256 of the file contents, and it matches what sha256sum reports for the real file. The pointer is content-addressed, just like git itself.
16. Fifth, why binaries are git’s weak point. Two repositories, ten commits each, both starting from nothing:

17. Ten revisions of a one-megabyte binary cost ten megabytes. Ten revisions of a two-megabyte text file cost 148 kilobytes.
18. The text repository is fourteen times smaller while holding twice as much content per revision. That gap is the whole story of binaries in git.

PLAIN40.12.4 what is really happening inside#

1. Git compresses history in two stages: zlib on each object, then delta compression, where one object is stored as a small set of instructions for turning another object into it.
2. Delta compression is why a text repository stays tiny. Change one line in a 60,000-line file and the delta is a few dozen bytes.
3. A compressed or random binary file has no useful similarity to the previous version. Change one pixel in a PNG and the compressed bytes change everywhere. The delta is as big as the file, so git stores a whole new copy.
4. That is not a bug and no setting fixes it. It is what content-addressed storage plus general-purpose compression can do.
5. Worse, git never forgets. Delete the 500 MB file today and it is still in every clone forever, because it is still reachable from an old commit.
6. Git LFS works around this by keeping a tiny text pointer in the repository and the real bytes elsewhere. A clean filter converts the file to a pointer on git add, and a smudge filter converts it back on checkout.
7. So the history contains only 132-byte pointers, which delta beautifully, and the big files are transferred by a separate service that fetches only the versions you actually check out.
8. The four scaling techniques attack four different costs, and it is worth being exact about which is which.
9. Shallow clone cuts the number of commits you download. It is the right tool for a build machine that only needs the newest source.
10. Partial clone cuts the number of objects you download, keeping the full commit graph. It is the right tool for a developer who needs history but not every old file.
11. Sparse checkout cuts the number of files written to disk. It does not reduce download size at all, but it makes git status, editors and search tools fast again.
12. Git LFS cuts what enters the history in the first place. It is the only one of the four that changes what is committed.
13. And the index is the quiet cost nobody expects. It is one file listing every tracked path with its metadata. For 1,356 files it was 156,135 bytes. Scale that to a million files and the index alone is over one hundred megabytes, read and written by ordinary commands.

TECHNICAL40.12.5 the engineer’s version#

1. Real monorepos, with their published numbers.

2. The Google figures come from the paper “Why Google Stores Billions of Lines of Code in a Single Repository” by Rachel Potvin and Josh Levenberg, published in Communications of the ACM in July 2016 and describing a January 2015 snapshot. It also reports roughly 25,000 developers, about 16,000 human commits and about 24,000 automated commits on a typical workday.
3. Important honesty point that most retellings omit: Google’s monorepo does not run on git. It runs on Piper, an in-house system, with a virtual filesystem client. So it is evidence that monorepos work at scale, not evidence that git works at that scale.
4. The Windows figures come from Brian Harry’s post of 24 May 2017, “The largest Git repo on the planet”, describing about 4,000 engineers, 1,760 official builds per day across 440 branches, and the Git Virtual File System that Microsoft had to build to make it possible.
5. GVFS was announced in February 2017, later renamed VFS for Git, and the ideas were pushed upstream over the following years as partial clone, sparse checkout in cone mode, the commit-graph file and the built-in filesystem monitor.
6. Version history of the relevant features, so you know what your git can do.

7. Partial clone filters worth knowing: --filter=blob:none omits all blobs; --filter=blob:limit=1m omits blobs above a size; --filter=tree:0 omits trees as well and is meant for tooling, not for humans.
8. A partial clone records the remote as a promisor. Objects arrive lazily, and lazily fetched packs are marked with a .promisor file. In the measurement above, 51 such packs appeared during one git log -p.
9. The server must permit it. uploadpack.allowFilter must be true on the serving side, or --filter is silently ignored and you get a full clone.
10. Sparse checkout in cone mode restricts patterns to whole directories, which lets git match paths with prefix comparisons instead of running every pattern against every path. Non-cone mode accepts full gitignore-style patterns and is much slower on large trees.
11. Practical settings for a large repository, all real configuration keys:

git config core.fsmonitor true
git config core.untrackedCache true
git config feature.manyFiles true
git maintenance start

12. feature.manyFiles is an umbrella that sets index version 4 and core.untrackedCache. git maintenance start schedules background repacking and commit-graph writing rather than waiting for git gc to interrupt you.
13. Git LFS, released by GitHub in April 2015, is not part of git. It is a separate program plus a separate HTTP API. git lfs track "*.psd" writes a line to .gitattributes naming the clean, smudge and merge filters.
14. Consequences people discover late: anyone cloning your repository needs git-lfs installed or they get pointer files instead of content; LFS storage and bandwidth are billed separately by most hosts; and converting an existing repository to LFS rewrites history, which changes every SHA.
15. Alternatives worth naming: git-annex, which predates LFS and is more flexible; and simply keeping large assets out of the repository entirely, in object storage referenced by a versioned manifest file.
16. Removing a large file that is already in history requires rewriting it out. The current recommended tool is git filter-repo; the older git filter-branch is deprecated for being slow and easy to misuse. Both change every commit hash after the touched point, so every collaborator must re-clone.

WORDS40.12.6 remember these#

1. Monorepo — one repository for many projects — a single version-controlled tree containing multiple deliverables with one commit history.
2. Shallow clone — only the newest commits — --depth N, producing a truncated history with grafted boundary commits.
3. Partial clone — download objects on demand — --filter=blob:none or similar, with the origin recorded as a promisor remote.
4. Promisor remote — the server that owes you objects — the remote a partial clone will lazily fetch missing objects from.
5. Sparse checkout — only some files on disk — index entries marked skip-worktree so git neither writes nor scans them.
6. Cone mode — directory-shaped sparse patterns — the fast form of sparse checkout, matching whole directories by prefix.
7. Delta compression — store the difference, not the file — packfile encoding of one object as instructions against a similar base object.
8. Git LFS — big files kept outside history — a filter that commits a small pointer file and stores real content on a separate service.
9. Pointer file — the 130-odd bytes git actually stores — a three-line text blob giving spec version, a SHA-256 oid and a byte size.
10. filter-repo — the supported way to rewrite history — the recommended replacement for git filter-branch, which rewrites every affected commit and therefore every hash.

40.13 Backing up a git repository properly#

PLAIN40.13.1 in simple words#

1. People say “every clone is a backup”. That is half true, and the missing half is the half that hurts.
2. A clone gives you every commit reachable from the branches you cloned, and the tags. That really is a lot, and it really does survive the server burning down.
3. A clone does not give you other refs, the reflog, stashes, the repository’s own configuration, its hooks, or anybody’s uncommitted work.
4. A mirror gives you every ref, of every kind, under its original name. That is more than a clone.
5. A mirror still does not give you reflogs, hooks or configuration, and it has a specific danger: it copies deletions too.
6. Neither gives you a single thing that lives on the website: issues, pull requests, reviews, build history, releases, secrets or settings.
7. And neither, on its own, protects you from the most common real disaster, which is not hardware failure. It is somebody force-pushing over a branch, or deleting one, and the automatic mirror faithfully copying that within the hour.
8. A real backup therefore has three properties a mirror does not have: it is dated, it is read-only once written, and it is kept for a while.

PLAIN40.13.2 a picture in your head#

1. Imagine three ways of protecting a handwritten manuscript.
2. A friend borrowed a copy last month. That is a clone. It is genuinely useful, it is complete up to when they took it, and it is out of date.
3. A machine in the next room copies your desk every hour, exactly. That is a mirror. It is always current, and if you spill ink on page 40, the machine copies the ink stain within the hour.
4. A sealed, dated envelope goes into a fire safe every night and stays there for ninety days. That is a backup. It cannot be edited, it can be opened at any date you choose, and the ink stain does not reach it.
5. Most teams have the first two and call it done.

Where this comparison breaks:

6. A copy of a manuscript can be a slightly bad copy. A git clone cannot. Every object is verified against its hash, so a corrupted copy is detectable and usually refuses to load rather than lying to you.
7. And the friend with the month-old copy is genuinely enough to save the project, which is not true of a month-old photocopy of an edited book. Git history is append-only in normal use, so old copies stay valid.

PLAIN40.13.3 a worked example#

1. First, what a clone leaves behind. A real source repository was set up with a stash, extra reflog entries, untracked files, a hook and a local setting, then cloned normally:

source repository        clone
  stashes: 1               stashes: 0
  reflog entries: 3        reflog entries: 1
  untracked files: 2       untracked files: 0
  custom hooks: 1          custom hooks: 0
  local user.signingkey    (absent)
  commits: 2               commits: 2

2. The commits came across perfectly. Everything else did not.
3. Second, what a mirror adds. The same source, cloned with --mirror:

$ git -C mtest.git show-ref
dbcd961d... refs/heads/main
f4caa812... refs/stash
141f4579... refs/tags/backup-2026-08-13

$ ls mtest.git/hooks | grep -v sample | wc -l
0
$ git -C mtest.git reflog | wc -l
0

4. Look at the second line of show-ref. The mirror picked up refs/stash, because its refspec is +refs/*:refs/* and a stash is a ref. A plain clone never does that.
5. The hooks did not come. The reflog did not come. The source’s local configuration did not come.
6. Third, a real backup run, start to finish. This is the whole procedure and every line below is genuine output.

$ git -C repo.git remote update --prune

$ git -C repo.git fsck --full
dangling commit de306ffa47620fa7e0d5ef7146...

$ git -C repo.git bundle create vault/proj-2026-08-13.bundle --all

$ ls -l vault/
1045 proj-2026-08-13.bundle

$ git -C repo.git bundle verify vault/proj-2026-08-13.bundle
vault/proj-2026-08-13.bundle is okay
The bundle contains these 4 refs:
a3bfdce1... refs/heads/feature/login
2304fd03... refs/heads/main
fb32611e... refs/tags/v1.0
2304fd03... HEAD
The bundle records a complete history.
The bundle uses this hash algorithm: sha1

7. That bundle is one ordinary file. You can copy it anywhere, put it on any storage, and it needs no git server to be useful.
8. dangling commit in the fsck output is normal, not an error. It means an object exists that no ref points at any more, which happens after a rebase or an amended commit.
9. Fourth, and this is the step almost everyone skips, the restore drill:

$ git clone vault/proj-2026-08-13.bundle restore
$ git -C restore log --oneline | head -3
2304fd0 Add app.py
20b8927 First commit: add README
$ git -C restore rev-list --count --all
3

10. It restored. Now you know. A backup you have never restored is not a backup; it is a hope.

PLAIN40.13.4 what is really happening inside#

1. Git keeps an object only while something can reach it. Refs are the roots. Anything reachable from a ref is kept; anything not is eventually removed by garbage collection.
2. That single rule explains the whole backup problem.
3. When you clone, git asks for the refs it wants and receives everything reachable from them. Objects reachable only from refs you did not ask for simply do not come.
4. A plain clone asks for refs/heads/* and tags. So refs/stash, refs/notes/* and any custom ref namespace are left behind.
5. A mirror asks for refs/*, so it gets all of them.
6. Neither asks for the reflog, because the reflog is not a ref. It is a local log file under .git/logs, private to one repository, recording where each ref used to point.
7. That matters because the reflog is what saves you from your own mistakes. It is the thing that lets you recover a commit after a bad reset. It is also the thing no backup of the server can give you.
8. Now the deletion problem. Suppose somebody force-pushes over main, discarding fifty commits. On the server the old commits become unreachable.
9. Your hourly mirror runs, sees main moved, moves its own main, and now the mirror is unreachable from those commits too. Run garbage collection on the mirror and they are gone for good.
10. A dated bundle written before the force-push is unaffected, because it is a frozen file, not a live replica. This is the reason bundles exist in a backup plan alongside mirrors.
11. What a bundle actually is: a packfile with a small header listing ref names and, for an incremental bundle, the commits the receiver must already have. git bundle verify checks both the pack and those prerequisites.
12. And a bundle is a first-class remote. git clone file.bundle, git fetch file.bundle, and git ls-remote file.bundle all work, which is why the restore drill above is one ordinary clone.

TECHNICAL40.13.5 the engineer’s version#

1. Coverage table. This is the section in one picture.

2. Incremental bundles work and are small. A real example: the full bundle was 1,497 bytes and the bundle of everything since a tag was 703 bytes.

$ git bundle create incr.bundle main --not backup-2026-08-13
$ git bundle verify incr.bundle
incr.bundle is okay
The bundle contains this ref:
dbcd961d... refs/heads/main
The bundle requires this ref:
141f4579...

3. “Requires this ref” is the prerequisite. Applying it to a repository that already has that commit works; applying it to an empty one fails cleanly rather than silently producing a broken repository.

$ git fetch /tmp/bk/incr.bundle main:restored
$ git log --oneline restored
dbcd961 Fourth
9cbb377 Third
141f457 Second
69199b0 First

4. A backup procedure that actually holds up. Run it from a machine that is not the git server.

# once
git clone --mirror git@host:org/proj.git /srv/bk/proj.git

# every hour: refresh the replica
git -C /srv/bk/proj.git remote update --prune

# every night: dated, immutable snapshot
D=$(date +%F)
git -C /srv/bk/proj.git fsck --full --strict
git -C /srv/bk/proj.git bundle create \
    /srv/vault/proj-$D.bundle --all
git -C /srv/bk/proj.git show-ref > /srv/vault/proj-$D.refs
git -C /srv/bk/proj.git bundle verify \
    /srv/vault/proj-$D.bundle

5. Then copy /srv/vault off-site, keep snapshots for a stated retention period, and delete older ones on a schedule you wrote down.
6. Why each step is there:

7. The show-ref file is cheap and disproportionately useful. When somebody asks “what was main pointing at on the ninth?”, it answers in one line without unpacking anything.
8. The 3-2-1 convention is worth applying here: three copies, on two different kinds of storage, one of them off-site. It is a convention from the photography world, not a standard, but it is sound.
9. Verification is not optional and is cheap in git, because every object is named by the hash of its own contents. git fsck --full --strict recomputes those hashes and reports any mismatch. No other backup system gets this for free.
10. Restore drills should be scheduled, not improvised. Once a quarter, clone the newest bundle into a scratch directory, compare git rev-parse main with the recorded .refs file, and delete it.
11. Now the part git cannot help with. Back up the platform metadata separately, because none of it is in any bundle:

gh api --paginate repos/OWNER/REPO/issues?state=all \
    > vault/issues-$D.json
gh api --paginate repos/OWNER/REPO/pulls?state=all \
    > vault/pulls-$D.json
gh api repos/OWNER/REPO/branches/main/protection \
    > vault/protection-$D.json
gh release list --repo OWNER/REPO \
    > vault/releases-$D.txt

12. Secrets cannot be exported at all by design. Keep the authoritative copy in a password manager or a secrets service, and treat the platform’s copy as a deployment detail, not as storage.
13. Settings are best kept as code. Branch protection, teams and repository options can be declared in a configuration tool and re-applied to a fresh repository in minutes, which is a better recovery story than a JSON dump nobody knows how to replay.
14. Encryption note: a bundle is not encrypted and not signed. If your history is confidential, encrypt the file before it leaves your control, and record the checksum separately so you can prove the file is intact.
15. Retention worth defaulting to, in the absence of a policy: hourly mirror, nightly bundle kept 30 days, weekly bundle kept 12 weeks, monthly bundle kept 12 months. Adjust for how much you would pay to recover a given day.

WORDS40.13.6 remember these#

1. Reachability — an object is kept if a ref can reach it — the rule that decides what a clone transfers and what garbage collection removes.
2. Dangling object — nothing points at it any more — an object with no ref path to it, reported by git fsck, normal after rebase or amend.
3. Bundle — a whole repository in one file — a packfile plus a ref header, usable directly as a remote for clone, fetch and ls-remote.
4. Prerequisite — what an incremental bundle assumes you have — commit identifiers listed in the bundle header and checked by bundle verify.
5. Replica — always current, always obedient — a mirror that reproduces deletions and force-pushes as faithfully as it reproduces work.
6. Snapshot — frozen and dated — an immutable copy taken at a known time and kept for a stated retention period.
7. Restore drill — proving the backup works — a scheduled rehearsal that clones the newest snapshot and compares refs against a recorded list.
8. fsck — check every object against its own hash — git fsck --full --strict, which detects corruption that ordinary commands might not notice.

40.98 Common wrong ideas#

1. Wrong: GitHub stores my code differently from my laptop. Right: it stores the identical objects with the identical hashes. A commit is 2304fd03... in both places, or one of them is corrupt. GitHub’s replication and routing are invisible to the protocol.
2. Wrong: a pull request is a git feature. Right: git has no such concept. git help -a lists nothing called pull request. A pull request is a row in a forge’s database plus two ref names, exposed to git only as read-only refs/pull/N/head.
3. Wrong: forking is a git operation. Right: forking is a forge action. Git’s word for taking a copy is clone. A fork is a server-side repository plus a recorded parent relationship that git never sees.
4. Wrong: if GitHub goes away my history is lost. Right: every clone holds the complete commit history. What would be lost is the meeting point and the metadata: issues, reviews, build results and releases. Back those up separately, because git cannot.
5. Wrong: a git server needs special server software. Right: it needs a bare repository and a way to reach it. git init --bare plus an SSH account with git-shell is a complete, secure server.
6. Wrong: a mirror is a backup. Right: a mirror copies deletions and force-pushes too. It protects against the server dying and not against a person making a mistake. Dated bundles protect against the person.
7. Wrong: a clone copies everything in the repository. Right: it copies objects reachable from branches and tags. It leaves behind the reflog, stashes, hooks, local configuration and all uncommitted work.
8. Wrong: git push failing means the push did not happen. Right: a failure while reading the reply, such as send-pack: unexpected disconnect while reading sideband packet, tells you the answer was lost, not what the server did. Re-query with git ls-remote to find out.
9. Wrong: LFS makes big binary files cheap in git. Right: it moves them out of git. The repository holds a 132-byte pointer, and the bytes live on a separate service with separate storage, separate billing and a separate program that everyone must install.
10. Wrong: a partial or shallow clone is strictly better because it is smaller. Right: it moves cost, it does not remove it. The same git log -p -300 took 0.197 seconds in a full clone and 106 seconds in a blobless clone, because every missing blob became a network round trip.

40.99 Chapter summary in 20 lines#

1. A git server is just another git repository that other machines can reach; there is no special server format and no central authority.
2. The meeting-point copy is normally bare, meaning it has the contents of .git at its top level and no working files.
3. git init --bare proj.git builds one in about ten milliseconds, and pushing into a non-bare repository’s checked-out branch is refused by default.
4. Running the whole loop by hand proves the model: create the bare repository, push, clone elsewhere, commit there, push back, pull into the first copy.
5. A remote is a name for a URL plus a refspec, and a URL is only a path plus a way of reaching it, so /tmp/lab/srv/proj.git and git@host:proj.git are the same idea.
6. The refspec +refs/heads/*:refs/remotes/origin/* says: take every branch, store it under a separate prefix, and allow non-fast-forward updates.
7. Server hooks are real enforcement: pre-receive gates the whole push, update gates one ref, post-receive acts afterwards and cannot refuse.
8. Client hooks are advice, because --no-verify skips them, they are not versioned, and a non-executable hook file is silently ignored.
9. GitHub is four layers: hosting for bare repositories, a web interface, identity and permissions, and automation. Founded 2008, bought by Microsoft in 2018 for 7.5 billion United States dollars.
10. The git layer of GitHub is ordinary git; git ls-remote against it prints exactly the same shape of output as against a folder in /tmp.
11. Pull requests, reviews, branch protection, required checks, CODEOWNERS behaviour, issues, projects, releases, packages, Actions, both APIs and the permission model are all GitHub, not git.
12. The reader’s outage proves the boundary: local git kept working, remote git and the whole GitHub API did not, exactly as the local/remote split predicts.
13. origin/main reading 0a95cc8 during the outage was a cached photograph, not a live fact, and a failure while reading a reply says nothing about whether the write landed.
14. Self-hosting is real and cheap: SSH plus git-shell costs tens of megabytes, Gitea or Forgejo runs in 1 GB, GitLab CE wants 8 to 16 GB.
15. Self-hosting buys control, privacy and no lock-in, and it makes you the operator, responsible for backups, uptime and patches at three in the morning.
16. Migration moves commits perfectly and almost nothing else; issue numbers, cross-references, timestamps, attachments and build history do not survive.
17. Choose the collaboration model by trust: shared repository for a team you trust, forks for strangers; then protect main, or every rule is a suggestion.
18. Git flow, GitHub flow and trunk-based development disagree about exactly one thing: how long a change may live away from the mainline.
19. Large repositories are managed with shallow clones, partial clones, sparse checkout and LFS, and binaries remain git’s weak point because they do not delta-compress.
20. A clone is not a backup, a mirror is not a backup, and a backup you have never restored is not a backup either.