Wednesday, September 23, 2009

Linux kernel workflow with Git

You worked on some part of Linux kernel. It works great. Now, how to generate the patch series and send it out for review? For this, I always used to generate diffs, create a set of draft mails (one for each patch) in KMail or Thunderbird, and send all these mails one-by-one. This workflow quickly became a big headache. Then I learned Git (and some related tools) to do all this from command line and wow! what a relief!
This is just a quick dump of my notes which I made while struggling to make it all work. There are probably many other (most probably, better) workflows but this is what works for me. I hope you will find it useful. Any comments/suggestions welcome!

Why workflow with KMail/Thunderbird was bad?

Firstly, for any Linux kernel patch series, the convention is to create an introductory email a.k.a. PATCH [0/n] where 'n' is number of patches in series. Then, all the following patches should be sent as reply to this introductory patch (so, if someone is not interested in your patches, they can simply collapse the thread). For example:
[PATCH 0/3] my super kernel feature
        |-> [PATCH 1/3] this is part1
        |-> [PATCH 2/3] this is part2
        |-> [PATCH 3/3] this is part3
With KMail or Thunderbird, I could not find any clean way to compose draft mails with this threaded structure. If when you compose drafts with this threading, it screws-up patch ordering while sending.
Secondly, almost all graphical email clients have the habit of tinkering with the text and corrupting the patch. BTW, this is not a problem with KMail atleast (Linux kernel includes documentation on how to avoid this problem with other email clients too, including Thunderbird).

Thirdly, whenever you have to send a patch series, you have to re-create entire To: and Cc: list. This can become quite cumbersome -- as more and more people review your code, this list can become quite long.


Workflow with Git

Creating patches:

Method 1:

Suppose you have Linux kernel tree with two branches: mywork and master. You make series of changes (commits) to mywork branch. If you have a clean work history -- i.e. last 'n' git commits reflect incremental development of your work then you can simply use following to create set of patches ready for submission:
git format-patch --cover-letter -o patch_dir master
Options:
    --cover-letter: also generate 'patch' that introduces your patch series.
    -o: specifies directory where to store patches created (patch_dir here).
    master: replace this with branch to diff against
(as usual, see man page for all other options and details).

This creates patch files:
out/0000-cover-letter.patch
out/0001-great-new-feature-something.patch
out/0002-great-new-feature-something-more.patch
... and so on.

The filename is based on first line of git commit message.
Each patch file begins with subject line:
[PATCH 0/n] <first line of git commit message> 
and rest of git commit message as the body, then followed by the actual patch.
If you want to omit automatic appending of '[PATCH 0/n]' part to the subject, you can use '--keep-subject' option for git format-patch.
Now, review subject and body for each of these patches -- especially cover letter which is, of course, without proper subject or body when initially created.

Method 2:

Sometimes you don't have a clean working history i.e. last 'n' commits do not represent incremental development of your work. This is usually the case with me. In my case, all development happened outside mainline. To prepare patches against mainline, I commit all files in one-go and make lot of small commits later as cleanups, bug fixes are done.
In such case, you can create another branch which has a 'clean' history by carefully committing changes in correct order to this new branch. Then, you git format-patch (as in Method 1) to generate patch series. However, I usually do following -- no good reason to prefer this but it just better suits my laziness.
Since my git commit history is full of uninteresting small commits, I create individual patches manually. For each patch, I hand-pick set of files to create patch like this:

git diff --patch-with-stat master path/to/file1 path/to/file2 patch/to/file3 > 0001-great_new_feature_something.patch
git diff --patch-with-stat master path/to/file4 path/to/file5 > 0002-great_new_feature_something_more.patch
.... and so on.
Note that patch file names contain numbering (like '0001') so that git send-email picks patches in correct order (more on this later).
In the beginning of each of these patches, add the following lines:
Subject: [PATCH 1/n] subject for patch1 ('n' is no. of patches)
<message body describing patch>
Signed-off-by: Your Name email@domain.com
--- (three dashes)
<actual patch as generated by git diff>
Also create 'cover letter' that introduces your patch series. Give it any filename but prefix it with '0000' (for example, 0000-cover-letter.patch), to make sure git send-email picks this patch for sending before any other patch (more on this later).

Sanity check for patches:

Verify that your patches follow all Linux kernel coding style (and some other basic sanity checks) by using checkpatch.pl script included with linux kernel:
linux_source/scripts/checkpatch.pl <patch filename>

Sending patches:

Now the patches are prepared and sanity checks are done, its time to send these patches out for review. We will use git-send-email for this.
First, prepare and address book containing entries for all the recipients. I use abook which I found very easy to use. With abook create this address book and export it as 'mutt alias' file (using 'e' for export, followed by 'c' for mutt alias format). This mutt alias file will be used by git send-email.
Now add following to linux_source/.git/config:
[user]
name = Your Name
email = email@domain.com
[sendemail]
aliasesfile = /path/to/email_aliases_file
aliasfiletype = mutt
smtpserver = smtp.gmail.com
smtpserverport = 465
smtpencryption = ssl
smtpuser = email@domain.com
to = alias1
cc = alias2
cc = alias3
cc = alias4
(these smtp settings are for gmail accounts).
where, alias1 etc. is the string appearing between 'alias' and actual email address in mutt alias file created above.
Now, we are ready to send it out to world. We want to send all patches as reply to the cover letter (a.k.a patch 0), as is the convention used on LKML. Send it with:
git send-email --no-chain-reply --suppress-cc=sob --suppress-cc=self patches_dir
Options:
    --no-chain-reply: Don't send every patch as reply to previous patch. Instead, what we want is to send all patches as reply to _first_ patch (i.e. the cover letter).
    --suppress-cc=sob: Don't Cc email mentioned in 'Signed-off-by:' line in each patch.
    --suppress-cc=self: Don't send a copy of patches to you.
(If your requirements are different, then man git-send-email is always your friend!)

git send-email seems to pick patches in alphabetical order. So, 0000-cover-letter becomes the first patch. All other patches are sent as reply to this one. At last, before actually sending patches, add ‘—dry-run’ option to git send-email and make sure everything looks okay.
That’s it!