Project

General

Profile

Doc-overview » Random » Filing » Release-todo »

Submitting Patches to Linux

We currently submit patches for Linux from our next branch. I try to give a short overview about it using some patches submitted to DaveM/Kuba.

Following steps must always be done:

  • get the linux-integration + batman-adv repository
  • get the newest patches from main and stable
  • check for compat code
  • squash/cleanup patches
  • fix "Fixes: " and other commit id references
  • check if coccinelle prints new warnings
  • check if all versions compile cleanly using sparse (cgcc)
  • do a merge in a temporary branch and compare the result with batman-adv:next.git
  • check commit messages author/signed-off-by of patches
  • check if all patches are checkpatch.pl clean
  • check (for net-next) that net-next is really open
  • prepare changelog
  • create tag for david which includes the changelog
  • format patches
  • submit cover letter + patches
  • wait until David accepted the pull request
  • update base patches

Rules for patch handling

  • new patches enter batman-adv.git main branch
  • new patches are forwarded to David from main branch
  • main branch is closed whenever net-next is closed
  • new patches for main should be avoided when Linux -rc7/final is near (which will cause the closing of net-next)

Variables

There are different basepoints/references used when preparing the

  • GIT_OM_SERVER
    • server from which the open-mesh.org repositories are retrieved
  • LM_MAIN
    • name of branch in batman-adv.git which points to the latest patch already ported to linux-merge.git:batadv/net-next
  • LM_STABLE
    • name of branch in batman-adv.git which points to the latest patch already ported to linux-merge.git:batadv/net
  • BASE_NETNEXT
    • some recent commit in net-next.git which should be used as base commit for the patches which will be applied (chose wisely)
  • BASE_NET
    • some recent commit in net.git which should be used as base commit for the patches which will be applied (chose wisely)
GIT_OM_SERVER="git+ssh://git@open-mesh.org" 
LM_MAIN=ecsv/lm_main
LM_STABLE=ecsv/lm_stable
BASE_NETNEXT=v4.16-rc1
BASE_NET=v4.16-rc1

get the linux-integration + batman-adv repository

First step is to actually get the repositories and create the branches. You may also use the --reference option for linux-merge to speedup the download when you already have some linux repository on you system

git clone "${GIT_OM_SERVER}/linux-merge.git" linux-merge
git -C linux-merge remote add net-next git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
git -C linux-merge remote add net git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
git -C linux-merge remote update --prune
git -C linux-merge branch base/net --track origin/base/net
git -C linux-merge branch base/net-next --track origin/base/net-next
git -C linux-merge branch batadv/net --track origin/batadv/net
git -C linux-merge branch batadv/net-next --track origin/batadv/net-next
git clone "${GIT_OM_SERVER}/batman-adv.git" batman-adv
git -C batman-adv branch "${LM_MAIN}" --track "origin/${LM_MAIN}" 
git -C batman-adv branch "${LM_STABLE}" --track "origin/${LM_STABLE}"

The next step is to make sure that out linux-merge.git branches are either on top or equal the chosen. It should also be checked first that it is really a basepoint in the chosen branches

git -C linux-merge remote update --prune
git -C linux-merge branch -r --contains="${BASE_NET}" net/main
git -C linux-merge branch -r --contains="${BASE_NETNEXT}" net-next/main

git -C linux-merge switch base/net
git -C linux-merge merge --ff-only "origin/base/net" 
git -C linux-merge merge --ff-only "${BASE_NET}" 

git -C linux-merge switch base/net-next
git -C linux-merge merge --ff-only "origin/base/net-next" 
git -C linux-merge merge --ff-only "${BASE_NETNEXT}" 

git -C linux-merge push origin base/net base/net-next

Problems during this have to either be fixed by using a correct base commit, fixing the local branches or by rebasing the remaining patches on the batadv/* branches on top of the base commits (think twice before rebasing).

The same has to be done for the batman-adv repository. Here it is is important to update the stable branch manually after releases. Otherwise it would look like patches which were already submitted to net-next have to be send again to net. But make sure that there are no actual fixes between ${LM_STABLE} and origin/stable which were missed earlier

git -C batman-adv remote update --prune

git -C batman-adv switch "${LM_STABLE}" 
git -C batman-adv merge --ff-only "origin/${LM_STABLE}" 

git -C batman-adv switch "${LM_MAIN}" 
git -C batman-adv merge --ff-only "origin/${LM_MAIN}"

get the newest patches

from main

cd batman-adv
git switch -C rebase --no-track origin/main
GIT_SEQUENCE_EDITOR="sh -c 'git rev-list --reverse --no-merges '\\''${LM_MAIN}..origin/main'\\'' --not origin/stable|awk '\\''{ print \"pick \"\$1}'\\'' > \$1' \$@" \
 git rebase -i origin/stable --onto "${LM_MAIN}" --no-ff

This patches have to be cleaned up. First these these changes have to be squashed so that "Fixes: " patches are squashed together with the patches which introduced the problem (when they exist). Also compat code has to be removed. Both things can be identified with git log:

Patches with "Fixes: " lines:

git log --grep='^Fixes: ' "${LM_MAIN}"..rebase

The extra stuff which should not be submitted can be identified via:

git log --stat --oneline "${LM_MAIN}"..rebase -- compat-sources compat-include README.external CHANGELOG Makefile .gitignore .gitattributes compat.h gen-compat-autoconf.sh README
git log -p -SUGLY_HACK "${LM_MAIN}"..rebase

This can all be cleaned up by using git-rebase and then marking the relevant commits for editing. It may also be a good idea to move patches like "Start-new-development-cycle" to the front

git rebase -i "${LM_MAIN}" 

# edit the commit message to point to the correct upstream commit -- just search in linux-merge for the commit name
git commit --amend

# remove changes either reverting to previous versions (new files have to be deleted)
git restore -s HEAD~1 README.external.rst CHANGELOG.rst Makefile .gitignore .gitattributes compat.h gen-compat-autoconf.sh README.rst

# delete some new file (example)
git rm -f compat-include/net/genetlink.h

# at the end continue with the rebase to the next commit
git rebase --continue

It is a good idea to check again if some file was missed. It is also a good idea to manually check the commit messages for other commit references:

git log --stat "${LM_MAIN}"..rebase

If everything looks good then these patches should be formatted (make sure that you don't have some old patches in the directory). The branch of already prepared patches has to be updated

git format-patch -s "${LM_MAIN}"..rebase
git switch "${LM_MAIN}" 
git merge --ff-only origin/main
git push origin "${LM_MAIN}"

Save the generated patches to a save place. Lets call this place $EXPORTED_PATCHES_MAIN

from stable

Make sure that ${LM_MAIN} was updated after a release correctly and ${LM_MAIN}..origin/stable is containing patches which were send to net-next

The rest of the rebase process works similar to the next process

cd batman-adv
git switch -C rebase --no-track origin/stable
git rebase -i "${LM_STABLE}"

This patches have to be cleaned up. First these these changes have to be squashed so that "Fixes: " patches are squashed together with the patches which introduced the problem (when they exist). Also compat code has to be removed. Both things can be identified with git log:

Patches with "Fixes: " lines:

git log --grep='^Fixes: ' "${LM_STABLE}"..rebase

The extra stuff which should not be submitted can be identified via:

git log --stat --oneline "${LM_STABLE}"..rebase -- compat-sources compat-include README.external CHANGELOG Makefile .gitignore .gitattributes compat.h gen-compat-autoconf.sh README
git log -p -SUGLY_HACK "${LM_STABLE}"..rebase

This can all be cleaned up by using git-rebase and then marking the relevant commits for editing. It may also be a good idea to move patches like "Start-new-development-cycle" to the front

git rebase -i "${LM_STABLE}" 

# edit the commit message to point to the correct upstream commit -- just search in linux-merge for the commit name
git commit --amend

# remove changes either reverting to previous versions (new files have to be deleted)
git restore -s HEAD~1 README.external.rst CHANGELOG.rst Makefile .gitignore .gitattributes compat.h gen-compat-autoconf.sh README.rst

# delete some new file (example)
git rm -f compat-include/net/genetlink.h

# at the end continue with the rebase to the next commit
git rebase --continue

It is a good idea to check again if some file was missed. It is also a good idea to manually check the commit messages for other commit references or missing signed-off-by:

git log --stat "${LM_STABLE}"..rebase

If everything looks good then these patches should be formatted (make sure that you don't have some old patches in the directory). The branch of already prepared patches has to be updated

git format-patch -s "${LM_STABLE}"..rebase
git switch "${LM_STABLE}" 
git merge --ff-only origin/stable
git push origin "${LM_STABLE}"

Save the generated patches to a save place. Lets call this place $EXPORTED_PATCHES_STABLE

Apply patches on linux-merge

from main

The patches were already exported. So they only have to be applied in the correct branch

cd linux-merge
git switch batadv/net-next
git merge --ff-only origin/batadv/net-next
./scripts/checkpatch --strict "${EXPORTED_PATCHES_MAIN}"/*.patch
git am -s "${EXPORTED_PATCHES_MAIN}"/*.patch
git push origin batadv/net-next

from stable

The patches were already exported. So they only have to be applied in the correct branch

cd linux-merge
git switch batadv/net
git merge --ff-only origin/batadv/net
./scripts/checkpatch --strict "${EXPORTED_PATCHES_STABLE}"/*.patch
git am -s "${EXPORTED_PATCHES_STABLE}"/*.patch
git push origin batadv/net

check if coccinelle prints new warnings

make coccicheck KBUILD_EXTMOD=net/batman-adv/

check if all versions compile cleanly using sparse (cgcc)

First the linux-merge build environment has to be prepared

make allnoconfig
cat >> .config << EOF
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_SMP=y
CONFIG_EMBEDDED=n
CONFIG_EXPERT=n
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SRCVERSION_ALL=y

CONFIG_CRC16=y
CONFIG_LIBCRC32C=y
CONFIG_NET=y
CONFIG_INET=y
CONFIG_DEBUG_FS=y
CONFIG_IPV6=y
CONFIG_BRIDGE=y
CONFIG_VLAN_8021Q=y
CONFIG_WIRELESS=y
CONFIG_CFG80211=y
CONFIG_CFG80211=y

CONFIG_BATMAN_ADV=y
CONFIG_BATMAN_ADV_BATMAN_V=y
CONFIG_BATMAN_ADV_BLA=y
CONFIG_BATMAN_ADV_DAT=y
CONFIG_BATMAN_ADV_NC=y
CONFIG_BATMAN_ADV_MCAST=y

CONFIG_CC_STACKPROTECTOR_STRONG=y
CONFIG_LOCKUP_DETECTOR=y
CONFIG_DETECT_HUNG_TASK=y
CONFIG_SCHED_STACK_END_CHECK=y
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCK_STAT=y
CONFIG_DEBUG_LOCKDEP=y
CONFIG_DEBUG_ATOMIC_SLEEP=y
CONFIG_DEBUG_LIST=y
CONFIG_DEBUG_PI_LIST=y
CONFIG_DEBUG_SG=y
CONFIG_DEBUG_NOTIFIERS=y
CONFIG_PROVE_RCU_REPEATEDLY=y
CONFIG_SPARSE_RCU_POINTER=y
CONFIG_DEBUG_STRICT_USER_COPY_CHECKS=y
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_DEBUG_RODATA=y
CONFIG_DEBUG_RODATA_TEST=n
CONFIG_DEBUG_SET_MODULE_RONX=y
CONFIG_PAGE_EXTENSION=y
CONFIG_DEBUG_PAGEALLOC=y
CONFIG_DEBUG_OBJECTS=y
CONFIG_DEBUG_OBJECTS_FREE=y
CONFIG_DEBUG_OBJECTS_TIMERS=y
CONFIG_DEBUG_OBJECTS_WORK=y
CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER=y
CONFIG_DEBUG_KMEMLEAK=y
CONFIG_DEBUG_STACK_USAGE=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_GDB_SCRIPTS=y
CONFIG_READABLE_ASM=y
CONFIG_STACK_VALIDATION=y
CONFIG_WQ_WATCHDOG=y
CONFIG_DEBUG_KOBJECT_RELEASE=y
CONFIG_DEBUG_WQ_FORCE_RR_CPU=y
CONFIG_OPTIMIZE_INLINING=y
EOF
make olddefconfig
make -j$(nproc || echo 1)

Then the rebuild has to be done between batadv/net-next and base/net-next (or between batadv/net and base/net) for each commit:

git switch batadv/net-next
git rebase -i base/net-next -x 'make C=1'

check if all patches are in linux-merge

TODO

submit patches

TODO

A good commit in has to be chosen. We should not send more than 20 patches for net-next for each pull request. A lower number (6?) of patches should be used for net.

Lets assume a good commit is 4e3e823b5a503235630921287f130e1d8d22d200

TAG=batadv-next-for-netdev-20160701
git tag -s ${TAG} 4e3e823b5a503235630921287f130e1d8d22d200
# write a good commit message
git push origin ${TAG}

The patches then have to be prepared and the cover letter has to be created for the pull request. For net-next it would look like:

TAG=batadv-next-for-netdev-20160701
git format-patch --cover-letter --subject-prefix='PATCH net-next' base/net-next..${TAG}
sed -i '/\*\*\* BLURB HERE \*\*\*/q' 0000-cover-letter.patch
git request-pull base/net-next git://git.open-mesh.org/linux-merge.git ${TAG} >> 0000-cover-letter.patch
# write cover letter
git send-email --to=davem@davemloft.net --to=kuba@kernel.org --cc=netdev@vger.kernel.org --cc=b.a.t.m.a.n@lists.open-mesh.org 00*.patch

or for net:

TAG=batadv-net-for-netdev-20160701
git format-patch --cover-letter --subject-prefix='PATCH net' base/net..${TAG}
sed -i '/\*\*\* BLURB HERE \*\*\*/q' 0000-cover-letter.patch
git request-pull base/net git://git.open-mesh.org/linux-merge.git ${TAG} >> 0000-cover-letter.patch
# write cover letter
git send-email --to=davem@davemloft.net --to=kuba@kernel.org --cc=netdev@vger.kernel.org --cc=b.a.t.m.a.n@lists.open-mesh.org 00*.patch

Now I will write something like

Subject: pull request for net-next: batman-adv 2016-07-01

Hi David,

we are a bit late to submit our feature patches, but I hope we make it in time.
Antonio is taking a pause in his upstream work, so I'll be submitting our
batman-adv patches for now. It is my first time, so please bear with me. 

Please pull or let me know of any problem!

Thank you,
      Simon

The following changes since commit 99860208bc62d8ebd5c57495b84856506fe075bc:

  sched: remove NET_XMIT_POLICED (2016-06-12 22:02:11 -0400)

are available in the git repository at:

  git://git.open-mesh.org/linux-merge.git tags/batadv-next-for-netdev-20160701

for you to fetch changes up to 4e3e823b5a503235630921287f130e1d8d22d200:

  batman-adv: Add debugfs table for mcast flags (2016-06-30 10:29:43 +0200)

----------------------------------------------------------------
This feature patchset includes the following changes:

 - two patches with minimal clean up work by Antonio Quartulli and
   Simon Wunderlich

 - eight patches of B.A.T.M.A.N. V, API and documentation clean
   up work, by Antonio Quartulli and Marek Lindner

 - Andrew Lunn fixed the skb priority adoption when forwarding
   fragmented packets (two patches)

 - Multicast optimization support is now enabled for bridges which
   comes with some protocol updates, by Linus Luessing

----------------------------------------------------------------
Andrew Lunn (2):
  batman-adv: Set skb priority in fragments
  batman-adv: Include frame priority in fragment header

Antonio Quartulli (4):
  batman-adv: statically print gateway table header
  batman-adv: remove ogm_emit and ogm_schedule API calls
  batman-adv: remove useless inline attribute for sysfs helper function
  batman-adv: move GW mode and selection class to private data structure

Linus Lüssing (4):
  batman-adv: Always flood IGMP/MLD reports
  batman-adv: Add multicast optimization support for bridged setups
  batman-adv: Adding logging of mcast flag changes
  batman-adv: Add debugfs table for mcast flags

Marek Lindner (5):
  batman-adv: document sysfs files in alphabetical order
  batman-adv: update elp interval documentation
  batman-adv: refactor batadv_neigh_node_* functions to follow common
    style
  batman-adv: remove unused callback from batadv_algo_ops struct
  batman-adv: init ELP tweaking options only once

Simon Wunderlich (2):
  batman-adv: Start new development cycle
  batman-adv: remove unused vid local variable in tt seq print

 .../ABI/testing/sysfs-class-net-batman-adv         |  20 +-
 net/batman-adv/Kconfig                             |   2 +-
 net/batman-adv/bat_algo.h                          |   7 +-
 net/batman-adv/bat_iv_ogm.c                        |  72 ++-
 net/batman-adv/bat_v.c                             |  29 +-
 net/batman-adv/bat_v_elp.c                         |   4 +-
 net/batman-adv/bat_v_ogm.c                         |   4 +-
 net/batman-adv/debugfs.c                           |  23 +
 net/batman-adv/fragmentation.c                     |  12 +-
 net/batman-adv/gateway_client.c                    |  15 +-
 net/batman-adv/gateway_common.c                    |   6 +-
 net/batman-adv/hard-interface.c                    |   6 +-
 net/batman-adv/main.c                              |   2 -
 net/batman-adv/main.h                              |   6 +-
 net/batman-adv/multicast.c                         | 499 +++++++++++++++++++--
 net/batman-adv/multicast.h                         |   3 +
 net/batman-adv/originator.c                        |  40 +-
 net/batman-adv/originator.h                        |   6 +-
 net/batman-adv/packet.h                            |   7 +-
 net/batman-adv/routing.c                           |   2 +
 net/batman-adv/send.c                              |  61 +--
 net/batman-adv/send.h                              |   4 +-
 net/batman-adv/soft-interface.c                    |  10 +-
 net/batman-adv/sysfs.c                             |  22 +-
 net/batman-adv/translation-table.c                 |   2 -
 net/batman-adv/types.h                             |  32 +-
 26 files changed, 704 insertions(+), 192 deletions(-)

update base patches after successful pull request

TODO