Adding a simple system call to the Linux 3.2.0 Kernel from scratch

As a part of my Operating System lab works, I had recently implemented a simple system call to the Linux 3.2.0 Kernel.

We can add changes in the kernel only if it is compiled or not. So in our case before adding a system call, we need to compile the kernel in our machine. Remember, you need to have at least 8 GB of free space in your root folder before you start compilation. Otherwise the compilation will end up with showing the error message that there’s not enough disk space in your machine and you will not be able to use that linux machine anymore. So it is preferred to test it in a virtual box with atleast 8 GB of free space!


Lets install all the dependency packages (gcc and libncurses5-dev)

$ sudo apt-get install gcc libncurses5-dev

Then update and install all the upgrade your machine.

$ sudo apt-get update && sudo apt-get upgrade

I am trying with Linux kernel 3.2 there are various other kernels available like 3.0, 3.4, 3.6 etc. You can download the kernel from .

$ wget

Extract the tarball to /usr/src/linux-3.X/

$ sudo tar -xvf linux-3.X.tar.bz2 -C /usr/src/linux-3.X/

Change the directory to /usr/src/linux-3.x

$ cd /usr/src/linux-3.x


Lets configure our new Linux Kernel with the configuration of old kernel (which is already installed in our machine)

$ sudo make oldconfig

Linux kernel source compilation

$ sudo make -j2

‘-j2’ is useful for multiprocessing in multiprocessor machines.

Compilation errors and solutions

There’s a possibility that you may come across an ERROR message like this while compiling

ERROR: "__modver_version_show" [drivers/staging/rts5139/rts5139.ko] undefined!
WARNING: modpost: Found 4 section mismatch(es).
To see full details build your kernel with:
make[1]: *** [__modpost] Error 1
make: *** [modules] Error 2

I got it to work by changing the value of CONFIG_RTS5139=n in .config file in source pre directory. This will disable that Realtek RTS5139 USB card reader support module which is not necessary for kernel compilation.
Again I compiled the kernel with CONFIG_DEBUG_SECTION_MISMATCH flag enabled. And this time, it worked for me! (it took me almost 23 minutes to compile in a machine with a 3 GHz xeon processor and with 4 GB of physical memory).

To Install

Next step is to install the compiled kernel

$ sudo make modules_install install

The above command will install the Linux Kernel 3.0 into your system. It will create some files under /boot/ directory and it will automatically make a entry in your grub.cfg. Check whether it made correct entry and check for the files it created.
The files under /boot/ directory are,

  • vmlinuz-3.0.0
  • initrd.img-3.0.0
  • config-3.0.0

If you unable to find the initrd.img-3.0.0 file then create it. Run the below command to create it.

$ sudo update-initramfs -u -k 3.X.X

To Update

If the above command unable to update / generate the initramfs then run the following command to create new initramfs.

$ sudo update-initramfs -c -k 3.X.X

You can update the grub in-order to make changes in grub

$ sudo update-grub2

Restart your system and from the previous kernels you can choose the newly installed kernels name. To check after booting open a terminal and type “uname -r“.

Writing simple “hello world” system call

Now lets add the system call!!

Writing the system call itself is pretty much the simplest part of this whole process, so we shall start there. First, create a new directory in the root of a clean copy of the kernel sources created for this tutorial. Call the new directory “hello”. In this directory, we need to create two files. The first of which is the implementation of the system call itself, hello.c:

#include <linux/kernel.h>:

asmlinkage long sys_hello(void) {
    printk("Hello World\n");
    return 0;

After creating the system call, we need to set up the Makefile to build it. Add the following to a Makefile in the same directory as hello.c:

obj-y := hello.o

This is a very simple Makefile, since the build system of the kernel takes care of most of the work for you. This concludes the new files that will need to be added to the kernel sources to make a new system call.

Kernel changes needed to reflect the new system call

There are a few source files in the kernel that will need to be updated in order to reflect the new system call to be added to the kernel. The first, and simplest of these is the root-level Makefile. Find the following line, around line 711 of the root-level Makefile:

core-y := kernel/ mm/ fs/ ipc/ security/ crypto/ block/

Assuming you created the new hello directory, add a hello/ to the end, as shown:

core-y := kernel/ mm/ fs/ ipc/ security/ crypto/ block/ hello/

It would be a good idea to set the EXTRAVERSION in the Makefile as well, to differentiate it from your normal kernel.

Next, open the include/linux/syscalls.h file, and add a definition for your new system call. Right after the line mentioning sys_setns (around line 849 of the file), add the following:

asmlinkage long sys_hello(void);

This step makes your system call available to other parts of the kernel that may wish to call it. While this is not necessary for this simple system call, it is still good practice for when adding a real system call to the kernel.

Next, open the arch/x86/include/asm/unistd_32.h file, and search for the following lines:

#define __NR_setns 346
#ifdef __KERNEL__
#define NR_syscalls 347

Between the __NR_setns and #ifdef __KERNEL__ lines, add the following:

#define __NR_hello 350

In addition, update the NR_syscalls line to 348, since you have added a new system call.

The final file that needs to be updated is the system call table, which resides in arch/x86/kernel/syscall_table_32.S. Add the definition for your new system call, by adding the following:

 .long sys_hello

You should add this after the definition for the previous latest syscall, which should be a line that mentions sys_setns.

Congratulations, you have now added a new system call to the Linux kernel. Follow the normal steps to build the kernel, and fix any problems you may find in compilation (if you followed this exactly, you should not have any).

Testing the new system call

The next step is to test out your system call to make sure it works. Be sure to reboot into your new kernel, then write the following in a new .c file (not in the kernel source tree):

#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <linux/kernel.h>
#include <sys/syscall.h>
#define __NR_hello 350

long int hello_syscall(void) {
    return syscall(__NR_hello);

int main(int argc, char *argv[]) {
    long int output = hello_syscall();
    printf ("The hello_syscall() returned %ld\n", output);
    return 0;

Compile the program, run it and check the dmesg command to see the “Hello World” output!

$ gcc hello.c -o hello
$ ./hello && dmesg
The hello_syscall() returned 0

References: [1]



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s