Friday, June 25, 2021

How to cherry pick on a commit but only a few files from upstream patch

I forked a repository from upstream and there was a patch I like and I would like that patch goes into my forked repository. But at the same time, I would like to pick only a few changes, that is, not the whole commit and at the same time too, I need to make some minor modifications. So here goes how this is done.

Let's check where we are at now.

 jason@localhost:~/advanced-policy-firewall$ git branch  
 * master  
  rfxn_upstream  
  robertoberto-patch  
  support-ipv6  

Get the commit id from roberto patch, cherry pick the whole commit but do not commit yet.. just stage at the current branch (which is master)

 jason@localhost:~/advanced-policy-firewall$ git cherry-pick -n 0d1df6549820f9592aefb2353b77c52eadbe759f  
 Auto-merging files/vnet/vnetgen  
 CONFLICT (content): Merge conflict in files/vnet/vnetgen  
 error: could not apply 0d1df65... - fixed vnetgen for newer linux distros to prioritize ip over ipconfig  
 hint: after resolving the conflicts, mark the corrected paths  
 hint: with 'git add <paths>' or 'git rm <paths>'  
 jason@localhost:~/advanced-policy-firewall$ git status .  
 On branch master  
 Your branch is up to date with 'origin/master'.  
 Changes to be committed:  
  (use "git restore --staged <file>..." to unstage)  
      modified:  README  
      new file:  test/distros_examples/ifconfig_ubuntu_12  
      new file:  test/distros_examples/ifconfig_ubuntu_20  
      new file:  test/distros_examples/ip_addr_ubuntu_12  
      new file:  test/distros_examples/ip_addr_ubuntu_20  
      new file:  test/distros_examples/ip_link_ubuntu_12  
      new file:  test/distros_examples/ip_link_ubuntu_20  
 Unmerged paths:  
  (use "git restore --staged <file>..." to unstage)  
  (use "git add <file>..." to mark resolution)  
      both modified:  files/vnet/vnetgen  

Let's get the diff of the stage file

 jason@localhost:~/advanced-policy-firewall$ git diff --cached README  
 diff --git a/README b/README  
 index 07ad004..0adf39d 100644  
 --- a/README  
 +++ b/README  
 @@ -199,7 +199,7 @@ Fedora Core Any  
  Slackware 8.0+  
  Debian GNU/Linux 3.0+  
  Suse Linux 8.1+  
 -Unbuntu Any  
 +Ubuntu Any  
  TurboLinux Server 9+  
  TurboLinux Fuji (Desktop)  
  RedHat Linux 7.3,8,9  

Unstage a file because I wanna make minor modification. After changes are make, add it back to the stage

 $ git restore --staged test/distros_examples/ip_link_ubuntu_12  
 $ vim test/distros_examples/ip_link_ubuntu_12  
 $ git add test/distros_examples/ip_link_ubuntu_12  

All good now, let's commit

 $ git commit -m "cherry pick but only selected files from roberto patched"  
 [master 80830c9] cherry pick but only selected files from roberto patched  
  8 files changed, 330 insertions(+), 28 deletions(-)  
  create mode 100644 test/distros_examples/ifconfig_ubuntu_12  
  create mode 100644 test/distros_examples/ifconfig_ubuntu_20  
  create mode 100644 test/distros_examples/ip_addr_ubuntu_12  
  create mode 100644 test/distros_examples/ip_addr_ubuntu_20  
  create mode 100644 test/distros_examples/ip_link_ubuntu_12  
  create mode 100644 test/distros_examples/ip_link_ubuntu_20  
 $ git branch  
 * master  
  rfxn_upstream  
  robertoberto-patch  
  support-ipv6  

and we push the changes to github

 $ git push  
 Enumerating objects: 19, done.  
 Counting objects: 100% (19/19), done.  
 Delta compression using up to 16 threads  
 Compressing objects: 100% (13/13), done.  
 Writing objects: 100% (14/14), 3.68 KiB | 3.68 MiB/s, done.  
 Total 14 (delta 7), reused 0 (delta 0), pack-reused 0  
 remote: Resolving deltas: 100% (7/7), completed with 4 local objects.  
 To https://github.com/jasonwee/advanced-policy-firewall.git  
   f98e9bb..80830c9 master -> master  

in case you want to see the final result, please visit this github commit.

Monday, June 21, 2021

Continuous Integration with GitHub Action

This is a contribution to the SUSE Cloud Native Foundations Scholarship Program which I received and some of the peers want a CI online demo. So here I will share on my knowledge and experience on continuous integration using a public repository and public runner. I will start with a sample project creation, GitHub project setup and end with GitHub action setup and runner. 


What is CI? 

In software engineering, continuous integration (CI) is the practice of merging  all developers' working copies to a shared mainline several times a day.[1]


So essentially, it is a routine where every developer has to go through after they made code changes. Example, syntax check, linting, code compiling, multiple tests, package building, perhaps also support different runtime versions and/or operating systems.

There are many CI software available, please choose the one which match your requirements. I particularly like Jenkins a lot but Jenkins would require you to setup the runner on your machine. Since the objective of this blog is all public, hence, I picked GitHub action.

Let's begin with initial project setup. Reference if you want to know more.

https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html

https://docs.github.com/en/actions/guides/building-and-testing-java-with-maven


```

$ export JAVA_HOME=/usr/lib/jvm/jdk-11.0.5/

$ mvn --version

Apache Maven 3.6.3

Maven home: /usr/share/maven

Java version: 11.0.5, vendor: Oracle Corporation, runtime: /usr/lib/jvm/jdk-11.0.5

Default locale: en_US, platform encoding: UTF-8

OS name: "linux", version: "5.10.0-7-amd64", arch: "amd64", family: "unix"

$ mvn archetype:generate -DgroupId=ch.weetech.app -DartifactId=demo_ci -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false

[INFO] Scanning for projects...

[INFO] 

[INFO] ------------------< org.apache.maven:standalone-pom >-------------------

[INFO] Building Maven Stub Project (No POM) 1

[INFO] --------------------------------[ pom ]---------------------------------

...

...

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  5.555 s

[INFO] Finished at: 2021-06-20T14:53:50+08:00

[INFO] ------------------------------------------------------------------------

$ cd demo_ci

$ tree .

.

├── pom.xml

└── src

    ├── main

    │   └── java

    │       └── ch

    │           └── weetech

    │               └── app

    │                   └── App.java

    └── test

        └── java

            └── ch

                └── weetech

                    └── app

                        └── AppTest.java


11 directories, 3 files

$ mvn package


[INFO] Scanning for projects...

[INFO] 

[INFO] -----------------------< ch.weetech.app:demo_ci >-----------------------

[INFO] Building demo_ci 1.0-SNAPSHOT

[INFO] --------------------------------[ jar ]---------------------------------

[INFO] 

[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ demo_ci ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

...

...

...

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  2.221 s

[INFO] Finished at: 2021-06-20T14:56:41+08:00

[INFO] ------------------------------------------------------------------------

$ java -cp target/demo_ci-1.0-SNAPSHOT.jar ch.weetech.app.App

Hello World!

```

Once we have the sample project setup, test and compile code locally okay. Then it is time that we initialize git repository locally and push to GitHub. But before  we do of the following, you need to create a new repository on your GitHub.


```

$ mvn clean

[INFO] Scanning for projects...

[INFO] 

[INFO] -----------------------< ch.weetech.app:demo_ci >-----------------------

[INFO] Building demo_ci 1.0-SNAPSHOT

[INFO] --------------------------------[ jar ]---------------------------------

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  2.079 s

[INFO] Finished at: 2021-06-20T15:24:46+08:00

[INFO] ------------------------------------------------------------------------

$ git init 

hint: Using 'master' as the name for the initial branch. This default branch name

hint: is subject to change. To configure the initial branch name to use in all

hint: of your new repositories, which will suppress this warning, call:

hint: 

hint: git config --global init.defaultBranch <name>

hint: 

hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and

hint: 'development'. The just-created branch can be renamed via this command:

hint: 

hint: git branch -m <name>

$ git status -m master

$ git status .

On branch master


No commits yet


Untracked files:

  (use "git add <file>..." to include in what will be committed)

pom.xml

src/


nothing added to commit but untracked files present (use "git add" to track)

$ git add pom.xml src/

$ git commit -m "initial"

[master (root-commit) 2cd8641] initial

 3 files changed, 108 insertions(+)

 create mode 100644 pom.xml

 create mode 100644 src/main/java/ch/weetech/app/App.java

 create mode 100644 src/test/java/ch/weetech/app/AppTest.java

$ git remote add origin https://github.com/jasonwee/demo_ci.git

$ git branch -M main

$ git push -u origin main

Enumerating objects: 16, done.

Counting objects: 100% (16/16), done.

Delta compression using up to 16 threads

Compressing objects: 100% (6/6), done.

Writing objects: 100% (16/16), 1.76 KiB | 899.00 KiB/s, done.

Total 16 (delta 0), reused 0 (delta 0), pack-reused 0

To https://github.com/jasonwee/demo_ci.git

 * [new branch]      main -> main

Branch 'main' set up to track remote branch 'main' from 'origin'.

$ git status .

On branch main

Your branch is up to date with 'origin/main'.


nothing to commit, working tree clean


```

Now that you pushed everything to GitHub, next is to setup GitHub Actions.

1. Locate Actions from the project tab.

2. Picked 'Java with Maven' as this best match the current requirement.

3. Check action configurations are okay and commit it. 



4. once you commit, the action will kickstart the first run.


Congratulation! This is the first CI that you setup. The rest, as you may have already guessed will be the same as every developer will do, i.e. making code changes.

```

$ vim README.md

$ cat README.md

a contribution to https://www.udacity.com/scholarships/suse-cloud-native-foundations-scholarship

$ git status .

On branch main

Your branch is up to date with 'origin/main'.


Untracked files:

  (use "git add <file>..." to include in what will be committed)

README.md


nothing added to commit but untracked files present (use "git add" to track)

$ git add README.md 

$ git commit -m "added readme" README.md 

[main a966f09] added readme

 1 file changed, 1 insertion(+)

 create mode 100644 README.md

$ git push

Enumerating objects: 4, done.

Counting objects: 100% (4/4), done.

Delta compression using up to 16 threads

Compressing objects: 100% (3/3), done.

Writing objects: 100% (3/3), 410 bytes | 410.00 KiB/s, done.

Total 3 (delta 0), reused 0 (delta 0), pack-reused 0

To https://github.com/jasonwee/demo_ci.git

   a273126..a966f09  main -> main

```





That's it, in case you figure where can I find this repository to begin mine? https://github.com/jasonwee/demo_ci