In this post I will describe the steps to be made in order to create a rootfs in order to boot a VM (virtual machine) using qemu. The main goal that we want to achieve be doing this is to create an environment where the host machine is the one running ceph and the VM is the client.

In order to boot a VM using qemu we need 2 things:

* a root filesystem - an image that contains the structure of
  folders and subfolders used be a linux system

* a kernel image corresponding with our needs - we created this in
  the last tutorial

In order to complete the entire setup, my mentor for this project provided me with some notes that he took when creating his setup. Those notes suggested the fact the I should use a script found in the ceph project in ‘src/script/’. Unfortunately, the script in question was deleted by in some previous commit and currently it is no longer in ceph.

To retrieve this script I have used git and its powerful commands. The steps are described below:

cd ~/work/ceph git rev-list -n 1 HEAD – src/script/

The last command should give you the hash of the commit in question (that deleted the script). The hash is: c9a6e60eee8719472bcf71fffb20eb2557703f9d

Now we are ready to retrieve the file and place it back where it belongs.

git checkout c9a6e60eee8719472bcf71fffb20eb2557703f9d^ – src/script/

Voila! We have the script back!

Now is the appropriate time to know that this script uses another one that was also deleted in that same commit (I found it the hard way - by running the script). So let’s just recover the second script also : network-from-cmdline

git checkout c9a6e60eee8719472bcf71fffb20eb2557703f9d^ – src/script/network-from-cmdline

Now that we have everything that we need lets get to work:

cd ~/work/ceph cd src/script ./ rootfs.img

This will create in the current directory a file an ext3 filesystem data file describing a debian image. The script uses debootstrap to download and install the necessary packages in the new rootfs so you should make sure you have it installed:

sudo apt-get install debootstrap

If everything is ok you will now have a file called rootfs.img in your current folder.

Unfortunately, I bumped into a problem exactly at the last step and the output was this:

+ umount /tmp/1227
umount: /tmp/1227: device is busy.
		(In some cases useful info about processes that use
				 the device is found by lsof(8) or fuser(1))

It meens that the filesystem was created succesfully but the folder were the file (rootfs.img) was mounted could not be unmounted. The step to solve this are:

* find out what are the processes using files from that folder:

ioana@ceph:~/work/ceph/src/script$ sudo lsof /tmp/1227/
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
	  Output information may be incomplete.
	  sshd    18908 root  cwd    DIR    7,0     4096     2 /tmp/1227
	  sshd    18908 root  rtd    DIR    7,0     4096     2 /tmp/1227
	  sshd    18908 root  txt    REG    7,0   475896 23639
	  sshd    18908 root  mem    REG    7,0    51728 57368
	  sshd    18908 root  mem    REG    7,0    43552 57401
	  sshd    18908 root  mem    REG    7,0    31616 57375

* as we can see 'sshd' is the only process that restricts us from
  unmounting the rootfs... so let's just terminate it.

  ioana@ceph:~/work/ceph/src/script$ sudo kill -9 18908

* finally unmount the rootfs:

ioana@ceph:~/work/ceph/src/script$ sudo umount  /tmp/ 1227

* check if everything is ok:

ioana@ceph:~/work/ceph/src/script$ file rootfs.img
rootfs.img: linux rev 1.0 ext3 filesystem data,
uuid=7a3dd817-e960-400e-b5d3-69072b08aae3 (large files)
ioana@ceph:~/work/ceph/src/script$ du -sh rootfs.img	689m    rootfs.img

Et voila! We have our own root filesystem that we are going to use to boot our virtual machine in some next tutorial.

Hi, people!

This post is the first from a series of posts in which I will describe in a detailed manner how to create the testing environment for this project.

First, I should mention the fact that I worked on a machine that had Ubuntu 14.04 and that I do not recomend Arch or something like it for doing this kind of setup if you are not ready to spend several days just to catch and find solution for numerous bugs (trust me, I went down that road but my mentor Alex Elder advised me not to fool around and just change the OS to a LTS Ubuntu).

Step 1: Install prerequisite packages

Here is a snippet from a script that I wrote that checks to see if
your system has all the packages and if not it installs them. If you
preffer installing them by hand below is a list.

packages_general=(git vim quilt);
packages_build=(autoconf automake autotools-dev debhelper libatomic-ops-dev
				libboost-dev libcrypto++-dev libcurl4-gnutls-dev libedit-dev
				libexpat1-dev libfcgi-dev libfuse-dev libgoogle-perftools-dev
				libgtkmm-2.4-dev libkeyutils-dev libtool pkg-config python
				python-dev python-pip python-support python-virtualenv uuid-dev
				cython libsnappy-dev libleveldb-dev libblkid-dev libudev-dev
				libnss3-dev libcurl4-nss-dev libaio-dev xfslibs-dev

				libboost-thread-dev libboost-random-dev libboost-regex-dev
				libboost-program-options-dev liblttng-ust-dev libbz2-dev);

Step 2: Get the source code

Go to and as you
see there will be more projects that the ceph community is working
on. At the moment, we are interested in ceph and ceph-client.

Just to make everything clean and easy I will describe my folder
structure. In my home directory I have an empty folder 'work' where
I want to place ceph derived things.

# it will take some time depending on the internet connection
cd ~/ceph
git clone
git clone

Step 3: Build ceph

We will build ceph using Autotools and the below instructions can
also be found in ceph/README along other useful things such as
building ceph using cmake (if you prefer it that way).

There are two steps involved in building anything: configuring and
actually building it.

# get to the right directory
cd ~/work
cd ceph

# handle the configuration part
./ -d3 

# now build ceph


or if you have multiple cores/threads it would be better to use

make -j $num_procs

where 'num_procs' is the number of processors of your system. What
it does is to specify the right level of conccurency and therefore speedup
your build. In my case it is: make -j4.

Now you should just go watch a movie, read a book or anything else...
because it may take a while.

Step 4: Build ceph-client

If you browse a little bit the ceph-client folder you may discover
that ceph-client is actually a kernel source appended with anything
that the ceph project needs.

Now I should mention the fact that this step needs another package
installed, so:

sudo apt-get install libncurses5-dev  # in order to user ncurses headers

The first thing to do is to get to the right folder in order to
configure and build the kernel.

cd ~/work/ceph-client

Now is the moment to mention the fact that the only thing that
distinguishes this build from any other one is the right
configuration that can be achieved with the following steps.

Creates a default configuration file .config based
on your system architecture

make defconfig

Add the specific bits that ceph client is interested in, more
'make menuconfig' is using ncurses in order to provide an interface
to the configuration process

make menuconfig # only accepted way of editing kernel conf file

It will show a window in the current terminal. If you type  '/' a
search window will appear. The next step is to search for the three
key words and just activate their config options by hitting the 'y'
button. Save and exit. After this in your .config file from the
ceph-client root folder is a fully functional configuration file.

Now lets build the kernel:

make -j4 # by this time you know why is there a '-j4' option

Congratulations! You just finished the first tutorial in order to setup the environment for ceph development and testing.

See you next time!

Hi, I am Ioana and for the next 3 months I will be an intern at the Linux Foundation through the Outreachy program.

After the application period, that lasted for about 1 mounth, I have been selected to work closely with Alex Elder and the Ceph community.

This blog was created with the main intent to keep you informed about the my progress during this internship.