Kernel Patch Management

Hello Readers,

Recently on open source mailing lists, there was a big discussion on patch related issues. I just thought of writing something on Linux “Patch management”

·         What is patch

·         How patch works

·         How to reverse the changes  

Patch: - The commands diff and patch form a powerful combination. They are widely used to get differences between original files and updated files in such a way that other people who only have the original files can turn them into the updated files with just a single patch file that contains only the differences. The patch program applies diff files to originals. The diff command is used to compare an original to a changed file. Diff lists the changes made to the file. A person who has the original file can then use the patch command with the diff file to add the changes to their original file (patching the file). Patch should be installed because it is a common way of upgrading applications.

If the patch file has extension gz or bz2, it need to be uncompressed before applying patch:-

·         gunzip patch-x.y.z.gz

·         bunzip2 patch-x.y.z.bz2

To apply a patch:-
patch -p1 </patch/to/file

Patches can be undone, or reversed, with the '-R' option:
patch -R </path/to/file

For complete details:-
man patch

Common errors and issues when applying patch:-

When a patch applies a patch file it verifies the sanity of the file in different ways

a.      Checking that the file looks like a valid patch file 

b.      Checking the code around the bits being modified matches the context 

provided in the patch 

If patch encounters something that doesn't look quite right it has two options:-

·         It can either refuse to apply the changes and abort

·         It can try to find a way to make the patch apply with a few minor changes.

Scenario 1:-

One example of something that's not 'quite right' that patch will attempt to fix 

if all the Context matches, the lines being changed match, but the 

line numbers are different. This can happen, for example if the patch 

makes a change in the middle of the file but for some reasons a few lines have 

been  added or  removed near the beginning of the file. In that case everything 

looks good it has just moved up or down  a bit, and patch will usually adjust the 

line  numbers  and apply the patch.

      Scenario 2:-

When patch encounters a change that it can't fix up with fuzz it rejects it outright 

and leaves  a file with a .rej extension (a reject file). You can 

read this file to  see exactly what change couldn't be 

applied, so you can  go fix it up manually.

      Scenario 3:- 

If patch stops and presents a "File to patch:" prompt, then patch could not find 

a  file to be  patched. Most likely you forgot to specify -p1 or you are in the wrong 

directory. Less often,  you'll find patches that need  to be applied with -p0 

instead  of -p1 (reading the patch file  should reveal if this is the case -- if 

so, then this is an error by the person who created the patch 

but it is not fatal).

     Scenario 4 :-

If you get "Hunk #2 succeeded at 1887 with fuzz 2 (offset 7 lines)." or a message 

similar  to that,  then it means that patch had to adjust the location 

of the change  (in this example it needed to move 7 lines from 

where it  expected to make the change  to make it fit). The resulting file may or 

may not be OK, depending on the reason the file was different 

than expected. This often happens if you try to apply a patch  that 

was generated against a different  kernel version than the one you are trying 

to patch.

Scenario 5 :-

If you get a message like "Hunk #3 FAILED at 2387.", then it means that the patch 

could not  be applied correctly and the patch program was unable 

to fuzz its way through. This will generate  a .rej file with the      

change that caused the patch to fail and also a .orig file showing you the  original 

content that couldn't be  changed. 

Scenario 6 :-

 If you get "Reversed (or previously applied) patch detected!  Assume -R? [n]" then 

patch detected that the change contained in the patch seems to have already been 

made. If you actually did apply this patch previously and you just 

re-applied it in error, then just say [n]o and abort this patch. 

If you applied this  patch previously and actually intended to revert it, but forgot 

to specify -R, then you can say [y]es here to make patch revert it for 

you. This can also happen if the creator of the  patch reversed 

the source and  destination directories when creating the patch, and in that  case 

reverting the patch will in fact apply it.

       Patch Work project: - http://patchwork.ozlabs.org/
       

       Hope you like this article; Feel free to share your feedback. 

LInux Kernel Configuration and Compilation

Hello Readers,

New to Linux Kernel??

In this post, we will be learning about how to compile Linux kernel, how to get the source and make the kernel and to understand entire ecosystem of Linux kernel and to understand the configuration aspect.

KERNEL COMPILATION

The Linux kernel is the core of the operating system, acting as a layer between the hardware and all other processes. The kernel provides for memory management, multi-tasking, input/output, networking, and many other functions. Since Linux is open source software, access to the source code for the Linux kernel is freely available. This means that anyone is free to customize and recompile the kernel to suit their specific needs. In fact, the kernel is designed in a modular fashion that allows users to remove parts of the kernel that are not needed for the intended purpose of the machine

When thinking of customizing and recompiling the kernel, you may have visions of sorting through thousands of lines of C code, but that is not the case. Unneeded kernel modules can be removed without editing a single line of code. But, recompiling the kernel is not something that needs to be done frequently. It is a rather complex process. A combination of failing to make proper backups of the old kernel and errors in the compilation process can result in a system that does not function correctly or that does not boot up at all!

Kernel Version Numbers

The Linux kernel version numbers consist of three numbers separated by decimals, such as 2.6.14.

·         The first number is the major version number.

·         The second number is the minor revision number.

·         The third number is the patch level version.

At any given time there is a group of kernels that are considered “stable releases” and another group that is considered “development.”

If the second number of a kernel is even, then that kernel is a stable release. For example, the 2.6.14 kernel is a stable release because the second number is even. If the second number is odd, then that kernel is a development release. For example, the 2.5.51 is a development release because the second number is odd. Once the 2.5.x branch is considered finished, and then it will become the 2.6.0 kernel. Patches will then appear for the 2.6.x branch and development work will begin on the 2.7.x branch. If the 2.7.x advancements are significant enough to be considered a major revision, the 2.3.x branch will become 3.0.0 and development work will begin on the 3.1.x branch.

When should I upgrade to new kernel:-

New kernels are released rather frequently and the difference between two consecutive patch levels is usually minimal. Updating your kernel every time a new kernel is released is usually pointless unless the new version addresses an issue that directly affects you.

In the past, the kernel was not as modular as it is today. This means that older kernels used to load many unneeded modules into memory, thus increasing system load and also increasing the chances that bugs in those unneeded modules could adversely affect the system.

Recompiling the kernel to remove the unneeded modules had noticeable benefits. Newer kernels, however, usually load modules into memory only when they are needed. Manually removing these modules has little positive effect since they are not called by the kernel anyway.

Recompiling the kernel may be necessary if new hardware is added to the system that the current kernel does not support. For example, a system with a dual processor motherboard but only one processor installed most likely has a kernel that supports only one processor. If a second processor is installed at a later date, the kernel must be recompiled to support symmetric multi-processing (SMP) in order to utilize the second processor.

The process of compiling the kernel places a heavy load on the system, especially the RAM. On a busy server, there is being a noticeable degradation of system performance. Also, after the kernel has been compiled and installed, the system must be rebooted so that the new kernel can be used. Depending on the role of the machine, the downtime involved with rebooting the server can be costly. Consideration should be given to the items listed above before recompiling the kernel.

If you wish to upgrade to a newer kernel, you can patch your current kernel instead of downloading an entire new kernel. By patching your existing kernel, you retain your settings from previous kernel compilations. Patching the kernel is a good choice if you wish to upgrade from your current patch level to the next consecutive patch level. For example, patching kernel 2.2.5 to 2.2.6 involves applying one patch. However, if you wish to upgrade the 2.2.5 kernel to 2.2.14, then a patch for each patch level must be applied sequentially. In this case, it may be better to download the entire 2.2.14 kernel

Now we know the compilation process in theory, let’s compile the Linux kernel:-

Acquire Source

The best place to get the latest source at is http://kernel.org/. Choose the version you would like to use, either in the 2.6 or 2.4 branches and click on the F link on the right side. Save that file to the /usr/src directory on your machine. If you do not have a src directory, then simply create it.

Extract and Patch

Extract the source by running the following command:

---------------

[root@localhost root]# tar -xvjf linux-2.6.6.tar.bz2

---------------

If you downloaded any additional patch sets, then you will need to unzip them as follows:

---------------

[root@localhost root]# bunzip2 patch-2.6.6-bk5.bz2

---------------

After uncompressing the patches, then you can apply them to the source. Remember to apply them in order if you have more than one.

--------------------------------------------------------

[root@localhost root]# cd linux-2.6.6
[root@localhost root]# patch -p1 <../patch-2.6.6-bk5

--------------------------------------------------------

Assign a Unique Name

This is completely optional, but if you end up having to make several different revisions of the kernel, then you will find this quite helpful. Inside of the kernel directory is a file called Makefile. You should consider updating the EXTRAVERSION line to include a revision number, such as:

---

VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 6
EXTRAVERSION = -1

---

Backup .config

If you have configured your kernel previously, then you should consider backing up your previous configuration as follows:

---

[root@localhost root]# cd linux
[root@localhost root]# cp .config config.save

---

Configure the Kernel :-  There are 4 main frontends to configuring the kernel: config, oldconfig, menuconfig, and xconfig. You simply choose one of these and type, for example, make menuconfig.

a. config is the least user-friendly option as it merely presents a series of questions that must be answered sequentially. Alas, if an error is made you must begin the process from the top. Pressing Enter will accept the default entry, which is in upper case.

b. oldconfig will read the defaults from an existing .config and rewrite necessary links and files. Use this option if you've made minor changes to source files or need to script the rebuild process. Note that oldconfig will only work within the same major version of the kernel. You cannot, for example, use a 2.4.x .config with the 2.6.x kernel.

c. menuconfig is probably the best option if you do not have X-Windows available.

d.   xconfig, as the name suggests, is an X-Windows based frontend. It uses the Tcl/TK libraries to present a nice GUI that is similar in structure to the ncurses config used with the menuconfig. Below is an example screen:

Once you have made all of your configurations, you can save your settings. By default, it will save as .config in the topmost directory. You can save this here, but I advise you to copy this file after you exit this utility.

Build Dependencies

The next step is create all of the necessary include files that the kernel needs to run and compile. If you are compiling a 2.4.x kernel, then run the following:

---

[root@localhost root]# make dep

[root@localhost root]# make clean

---

With the 2.6.x kernels, these 2 steps can be skipped.

Build the Kernel

We are now (finally) ready to start the actual kernel build. At the prompt type:

---

[root@localhost root]# make bzImage

---

Build the Modules

Although you have finished building the kernel, without all of the modules that the kernel loads and runs, the system will be pretty useless. You need to run the following 2 commands to build and install the modules:

----

[root@localhost root]# make modules

[root@localhost root]# make modules_install

----

Install

Not sure if this works or not on the 2.4.x kernels, but running the following command on a 2.6.x kernel will make the appropriate boot images and update grub as needed to include this kernel.

---

[root@localhost root]# make install

---

Reboot and Test

Simply reboot the system and then from the grub menu, choose the kernel that you built. If all goes well, then you should be up and running with all of your peripherals responding correctly and without any crashes. If, however, something breaks, you still have your prior kernels you can go back to at any time.

Hope you liked the article; feel free to post your comments/suggestions if any.

Articles related to system and kernel boot process

Hello Readers,

In this post i would like to share link to some of the best articles i have read in recent time:-

a. Article related to "Motherboard chipset and Memory map"
http://duartes.org/gustavo/blog/post/motherboard-chipsets-memory-map

b. How PC Boots up
http://duartes.org/gustavo/blog/post/how-computers-boot-up

c. The Kernel boot process
http://duartes.org/gustavo/blog/post/kernel-boot-process

I am quite positive you guys will love these write-up's. I will be sharing more of such links in future posts. Let's share more and learn more.

GITWEB

Once we have the GIT server up and running with the repository structures in place, we need to explore more about GITWEB, so as to make browsing of repositories easier. This post will provide pointers related to setting up GITWEB and its configuration.

 

What is GITWEB??

• 'Gitweb' is a Git web interface, the one working on http://www.kernel.org/git
• It is written in Perl, and can be used as a CGI script, or as a mod_perl legacy script
• It allows browsing a git repository (or a set of git repositories) using a web browser.
• Using gitweb we can browse directory trees at arbitrary revisions, view contents of files, see log or examine commits, commit messages and changes made by a given commit.
• We can search for commits by an author, added to repository by a comitter, commit with commit message (commit description) which includes some text.

GITWEB CONFIGURATION

Install Apache

First, you need to ensure apache2 is installed. Check to see if you have an /etc/apache2 folder. If you do, you should be okay. If you don’t, you need to install it.  sudo apt-get install apache2

Once you have that installed and the daemon is restarted, type the ip address in the browser and you should something like “It Works” 

Issues related to Apache

There can be lot of issues related to apache configuration, some are listed below:-

There may be a firewall running on your server, and it might be blocking traffic to the standard web server ports, port 80 (for regular connections) and port 443 (for secure connections).

Check the current firewall rule:-

sudo  /sbin/iptables –L

IP table rules to open the ports

sudo /sbin/iptables -I INPUT -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I INPUT -p tcp --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo /sbin/iptables -I OUTPUT -p tcp --sport 443 -m state --state ESTABLISHED -j ACCEPT

Logs

Apache logs are stored under:- /var/log/apache2     

Test the server

If you navigate to your server's IP address in a web browser: http://<ip address>

You should see a default page telling you apache is working

INSTALL GITWEB

sudo apt-get install gitweb

This will install gitweb into /var/www/gitweb, create a conf file at /etc/gitweb.conf, and add three files to /usr/share/gitweb (git-favicon.png, git-logo.png, gitweb.css).

If for some reason, the /var/www/gitweb folder is not created, you can clone the git source (git clone git://git.kernel.org/pub/scm/git/git.git) and then just copy the git/gitweb folder to /var/www/gitweb

cp –R git/gitweb /var/www/

You only need to edit the $projectroot variable in the /etc/gitweb.conf to point to your repositories folder:

/etc/gitweb.conf

=============================== 

# path to git projects (<project>.git)
$projectroot = "/project";
# directory to use for temp files
$git_temp = "/tmp";
# target of the home link on top of all pages
#$home_link = $my_uri || "/";
# html text to include at home page
$home_text = "indextext.html";
# file with project list; by default, simply scan the projectroot dir.
$projects_list = $projectroot;
# stylesheet to use
$stylesheet = "/gitweb/gitweb.css";
# logo to use
$logo = "/gitweb/git-logo.png";
# the 'favicon'
$favicon = "/gitweb/git-favicon.png";
#enable human readable URLs
$feature{'pathinfo'}{'default'} = [1];

===============================

Since Ubuntu includes the entire /etc/apache2/conf.d directory, so we just added the following information into /etc/apach2/conf.d/gitweb:-

 /etc/apache2/conf.d/gitweb.conf

===============================
AddHandler cgi-script .cgi
#</Directory>
ServerName hostname1
<Directory /var/www/gitweb>
RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT]
Options Indexes FollowSymlinks ExecCGI
DirectoryIndex /cgi-bin/gitweb.cgi
AllowOverride None
</Directory>

 /etc/apache2/sites-enabled/000-defaul

===============================
 Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

ScriptAlias /git "/usr/lib/cgi-bin/gitweb.cgi"
        <Directory "/var/www/gitweb">
                Options Indexes FollowSymLinks ExecCGI
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>
===============================

Restart Apache to make sure everything works....

Congratulation, the gitweb is up and running on your system now. Hope you find this article useful, If there are any issues or concerns feel free to share your feedback.

Open source version control management

GIT is a distributed revision control system with an emphasis on speed.
GIT was initially designed and developed by Linus Torvalds for Linux kernel development.
Later on, GIT became the de facto standard in the open source community.

The main design goals behind GIT are:-

·         Support for distributed workflow

·         Very high performance

·         Very strong safeguards against corruption

GIT is free software. It is distributed under the terms of GNU General Public License v2. The GIT provides only the basic functionality for SCM operations. It can be seen as a versioning file-system. GIT does not have any notion of users or access rights. Nor it has any graphical user interface. All these are built on top of GIT by additional tools, like Gerrit.

Let's get started:-
Initializing the build environment

64-bit systems:

---------------------------------
$ sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev libgl1-mesa-dev
---------------------------------

More information on initializing the build environment can be found at:-
http://source.android.com/source/initializing.html

Download the GIT Source
Use your favorite package installer to install git-core and gitosis packages:
sudo apt-get -y install git-core

Installing Repo
Repo is a tool that makes it easier to work with Git in the context of Android. For more information about Repo
To install, initialize, and configure Repo, follow these steps

Make sure you have a bin/ directory in your home directory, and that it is included in your path:

-----
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl http://android.git.kernel.org/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
-----

After installing Repo, set up your client to access the android source repository:-

Create an empty directory to hold your working files:

Run repo init to bring down the latest version of Repo with all its most recent bug fixes. We must specify a URL for the manifest, which specifies where the various repositories included in the Android source, will be placed within your working directory.

$ repo init -u git://hostname/local/platform/manifest.git

To check out a branch other than "master", specify it with -b:

When prompted, configure Repo with your real name and email address. To use the Gerrit code-review tool, you will need an email address. Make sure this is a live address at which you can receive messages.

A successful initialization will end with a message stating that Repo is initialized in your working directory. Your client directory should now contain a .repo directory where files such as the manifest will be kept.

To pull down files to your working directory from the repositories as specified in the default manifest, run $repo sync

Let’s learn something about manifest file, how it works and what it contains?

What is a Manifest file??
It’s a xml file, which contains information related to:-
·         Remote
·         Revisions
·         Project path
·         Project name


Manifest file is a xml file which contains all the information about:-
Projects:-
Which are the git projects that have to be synced, where those projects are located, which revision of those git projects should be picked while syncing to local directory.

Remote:-
Points to the generic path of the git repositories.

Revision: -
Specifies which branch or tag or commit needs to be picked up while syncing.

Useful Links to explore mode on GIT:-
http://www.gitref.org
http://www.kernel.org/pub/software/scm/git/docs/user-manual.html
 http://progit.org/book/
 http://git-scm.com/
http://www.github.com