Trudy in Linuxland

LinuxCon Europe 2016 ( aka Presenting for the first time at a conference)

2016-10-14T00:00:00-00:00

Berlin was the host of LinuxCon Europe this year, 4-6 october 2016. I had the opportunity to attend this conference as part of the Outreachy internship with the Linux Kernel and took the chance to be part of the Outreachy Kernel Report Panel session and present the work that I did, what I’ve learnt and how did this internship helped me.

Unlike Linaro Connect, the conference that I attended sometime in March, I had a bunch of people with me at the LinuxCon Europe. First of all, I met Julia, our project coordinator, and all the other participants from various rounds on the Outreachy Kernel internship. Also, some of my classmates where there as a reward for winning an Operating Systems contest that is help in my university.

The Outreachy Kernel Report session was held in the first day of the conference. We did not expect many people to attend it and so it was but those that were there are the people that are actually interested in how the kernel community welcomes the newbies. Because, let’s face it.. even though some of us now have full-time jobs we are still newbies. Julia presented the program as a whole, all the interns presented their work in about 5 minutes each and then the questions began. One of my favorite one is about the ramp-up process that a newbie goes through on order to get the hand of creating a patch, sending it as an text email on some mailing list, review it, etc. I, personally, have no particular problem with the mailing list way of work of the kernel, it seems to me that it’s as clean as possible and easy to use. But if you want to hear Greg KH’s opinion on this check out this video

You can find the slides here. My part won’t be there (at least when I published this).. because I am that kind of person that waits to get things done in the last minute.

When the hard part was done - we all presented ouw progress - the fun part began. I attended as many sessions. In the next part I will present some of the most interesting ones.

Geo-Replication and Disaster Recovery for Cloud Object Storage with Ceph Rados Gateway - Orit Wasserman

During the time that I worked on Ceph - the distributed filesystem run by RedHat and Sage Weil - I only investigated and tried to improved cluster-wide performance without knowing or going into the concept of Geo-Replication. Learn a lot of new things and the direction Ceph is going at.
Ceph and Flash - Allen Samuels

Efficiency doesn’t only come from smart algorithms, sometimes you also need to choose your hardware appropriatly. Allen thaught us how to improve our Ceph clusters by working at the storage hardware.

lguest: A Journey of Learning the Linux Kernel Internals - Daniel Baluta

Daniel is a teaching assistent in my university, an Intel employee and also a mentor in the Outreachy Kernel. He was the one that held a kernel entry-level workshop in our faculty to introduce on the linux field from early on. This session was another attept to give newbies a welcoming entry in the kernel. lguest is basic hypervisor that introduces people into many important topics such as - interrupts, memorry assembly, stack trace etc. Great to watch.

Cgroups and namespaces, The Building blocks of linux containers - Rami Rosen

Used Docker in order to create and run multiple containers that would create the Ceph cluster. Eventually you have to learn also how these containers are created and managed by the kernel. If you also have the same opinion, check Rami’s talk.

Build your own ChromeOS distro and Image Server - Ronald G. Minnich

Being able to run your own software from firmware to OS is really an incredible thing. Intersting to see how some companies give the end-user the possibility to change everything in their systems. Unlike the majority, Google is doing that.

CephFS and LXC: Container High Availability and Scalability, Redefined - Florian Haas

Learnt about ansible playbooks in corelation with containers and LXC. Will have a look at that in the near future.

Locking down your Systemd services - Lennart Poettering

Last but not least, systemd - the controversial init system that lately becomes much more than that. In this talk, Lennard presented new options for service sandboxing. I think that going before people and owning your opinions and decisions no matter what takes a great man.

Fun fact:

25 years of Linux. I remember that during one of the keynotes, the presenter ask to raise the hand the people that were no older than 25. Well, not so many as I would expect but some. Oh, and he reminded nnot be too eager.. we will also get old.

Fun fact #2:

Linus did not attend LinuxCon.., instead it went to Linaro Connect LAs Vegas where he presented its opinion of the ARM ecosystem. Google it.

All in all, the LinuxCon experience was one of a kind. I hope to attend it also next year and to be a better part of the community as I grow as a person and a engineer.

LinuxCon Europe 2016 ( aka Presenting for the first time at a conference) was originally published by IoanaCiornei at Trudy in Linuxland on October 14, 2016.

Linaro Connect. Do's and Dont's

2016-03-28T00:00:00-00:00

The big day has arrived and on 5th of March at noon I was about to get in the plane to Bangkok to attend Linaro Connect. It was an aventured that lasted about 13 hours but Sunday, 6th of March, at 9am I was in Bangkok with an arrival visa in my hands.

The next step was to get to Centare Grand Hotel, to settle in and to meet Alex Elder, my mentor.

Somewhere around noon I had the occasion to meet Alex and get to spend some time visiting the surroundings with him. It was a great experience, better that I have expected (and I was expecting to be great). Alex had some great advice about what I should do to make the most of the following week.

What TO DO when attenting your first technical conference?

1. Talk to as many people as you encounter.
2. Go to the technical sessions and ask questions. People will
   be more than happy to answer them.
3. If you know beforehand a person that is attending that
   conference, ask him/her to introduce you to people. It will
   make a difference in the future.
4. Don't panic if you don't know everything that is discussed
   there. Later you will find out that you are not the only one.
   Best approach is to ask.

What NOT TO DO?

1. Don't be shy. Tell your story, ask for opinion and advice.
2. Don't be afraid to speak your mind (in a civilized manner).
3. Don't just spend time in your hotel. Get out, plan meetings
   and go to meet people. It will help your career but also you
   will have a great time.

I will be adding more tips when I remember them.. but in the mean time.. That’s it.

Linaro Connect. Do's and Dont's was originally published by IoanaCiornei at Trudy in Linuxland on March 28, 2016.

Linaro Connect. When, were, how?

2016-03-27T00:00:00-00:00

In this series of blog posts I will tell you more about my first technical conference that I attended - Linaro Connect - Bangkok 2016.

How I ended up doing to Linaro Connect?

As I was saying in some previous blog posts, my mentor for the Outreachy program, Alex Elder, is currently working at Linaro. In some of our first meeting he confronted me with a big question: “How do you feel about flying to Thailand in march?”. I didn’t know how to react to this until he told me that me and one other Outreachy intern were given the possibility to attend the this year Linaro Connect in order to get to know our mentors and any other engineers that were participating.

What do I know at this moment?

I have the opportunity to go to Linaro Connect on Bangkok
It will be my first flight
I am going to go by myself

Hmmm.. I cannot deny that I had my concerns and I was just a bit anxious about this trip but I could not say no to this huge opportunity.

So, the next week I responded to Alex’s question with a big YES and many other questions.

That’s it for this blog post. I will continue my story in the following ones.

See you soon!

Linaro Connect. When, were, how? was originally published by IoanaCiornei at Trudy in Linuxland on March 27, 2016.

Bugfix in the launch script

2016-02-18T00:00:00+02:00

As the title suggests there were some problems in the script that launched the qemu instance. This post is trying to get solve that problem.

When launching the qemu instance you might have observed the fact that there you were not able to run smoothly any command and the password promt or the login one just showed up over and over again.

It seems that there were 2 instances of the “getty” process that just got bits an pieces from what you were typing and never enough to make a proper login.

I chose to solve this issue by redirecting the output from the virtual machine into the host terminal so that you do not have to deal with another window or any login problems.

The revised script is below:

#! /bin/sh

brctl addbr br0
ip addr add 192.168.5.4/24 broadcast 192.168.5.255 dev br0
ip link set br0 up

ip tuntap add dev tap0 mode tap
ip link set tap0 up promisc on

brctl addif br0 tap0

qemu-system-x86_64  -M pc -m 1024 -no-reboot \
-nographic \
-kernel boot_files/bzImage  -append "console=ttyS0,115200 highres=off \
root=/dev/sda rootfstype=ext3" \
-hda  boot_files/rootfs.img \
-net nic,macaddr="52:54:be:36:42:a9" -net tap,ifname=tap0,script=no,downscript=no

ip link set br0 down
brctl delbr br0

ip link set tap0 down
ip link delete tap0

Thanks again.

See you next time.

Bugfix in the launch script was originally published by IoanaCiornei at Trudy in Linuxland on February 18, 2016.

Setup the environment - Run VM in qemu

2016-01-20T00:00:00+02:00

If you followed the previous two blog posts now you should have a rootfiles system, a kernel image and a compiled ceph to run on your host.

Now that we have everything that we need we should get to work in running the VM, the ceph filesystem and mount the filesystem into the guest VM.

The first step is to run the virtual machine using qemu. I have created a small script that just creates and configures a network bridge, a tap connected to it and to the VM and starts the virtual machine.

For this to happen smoothly you should install some packages:
	* bridge-utils  (in order to create a bridge)
	* isc-dhcp-server (to start a dhcp server on the bridge
	  interface)

In order to configure the dhcp server you should add the following entry at the end of /etc/dhcp/dhcpd.conf file:

    subnet 192.168.2.0 netmask 255.255.255.0 {
		range dynamic-bootp 192.168.2.5 192.168.2.150;
		option broadcast-address 192.168.2.255;
		option routers 192.168.2.4 ;
	}

The folder structure in which I copied the rootfs and the kernel is
the following:

drwxrwxr-x  5 ioana 1000 4096 Jan 20 15:36 ./
drwxr-xr-x 25 ioana 1000 4096 Jan 20 13:17 ../
drwxrwxr-x 24 ioana 1000 4096 Dec 30 02:09 ceph/
drwxrwxr-x 26 ioana 1000 4096 Jan 20 16:28 ceph-client/
drwxr-xr-x  3 root  root 4096 Jan 20 16:45 testing/

drwxr-xr-x 3 root  root  4096 Jan 20 16:45 ./
drwxrwxr-x 5 ioana  1000 4096 Jan 20 15:36 ../
drwxr-xr-x 2 root  root  4096 Jan 20 16:08 boot_files/
-rwxrwxr-x 1 ioana users  614 Jan 20 16:46 launch.sh*

root@yoga:/home/ioana/work/testing# ll boot_files/
drwxr-xr-x 2 root root       4096 Jan 20 16:08 ./
drwxr-xr-x 3 root root       4096 Jan 20 16:45 ../
-rw-r--r-- 1 root root    6307248 Jan 20 16:28 bzImage
-rw-r--r-- 1 root root 1073741824 Jan 20 16:47 rootfs.img


The script that you need to use in order to launch the qemu VM is:

launch.sh:

#! /bin/sh

brctl addbr br0
ip addr add 192.168.5.4/24 broadcast 192.168.5.255 dev br0
ip link set br0 up

ip tuntap add dev tap0 mode tap
ip link set tap0 up promisc on

brctl addif br0 tap0

qemu-system-x86_64  -M pc -m 1024 -no-reboot -kernel \
boot_files/bzImage  -append "highres=off \
root=/dev/sda rootfstype=ext3" \
-hda  boot_files/rootfs.img \
-net nic,macaddr="52:54:be:36:42:a9" -net tap,ifname=tap0,script=no,downscript=no

ip link set br0 down
brctl delbr br0

ip link set tap0 down
ip link delete tap0

This script will open a new qemu window from where you can interract with your new vitual machine.

The default user is root and it does not have a password.

Until next time!

Setup the environment - Run VM in qemu was originally published by IoanaCiornei at Trudy in Linuxland on January 20, 2016.

Setup the environment - Create the rootfs

2016-01-06T00:00:00-00:00

In this post I will describe the steps to be made in order to create a rootfs in order to boot a VM (virtual machine) using qemu. The main goal that we want to achieve be doing this is to create an environment where the host machine is the one running ceph and the VM is the client.

In order to boot a VM using qemu we need 2 things:

* a root filesystem - an image that contains the structure of
  folders and subfolders used be a linux system

* a kernel image corresponding with our needs - we created this in
  the last tutorial

In order to complete the entire setup, my mentor for this project provided me with some notes that he took when creating his setup. Those notes suggested the fact the I should use a script found in the ceph project in ‘src/script/build_debian_image.sh’. Unfortunately, the script in question was deleted by in some previous commit and currently it is no longer in ceph.

To retrieve this script I have used git and its powerful commands. The steps are described below:

cd ~/work/ceph git rev-list -n 1 HEAD – src/script/build_debian_image.sh

The last command should give you the hash of the commit in question (that deleted the script). The hash is: c9a6e60eee8719472bcf71fffb20eb2557703f9d

Now we are ready to retrieve the file and place it back where it belongs.

git checkout c9a6e60eee8719472bcf71fffb20eb2557703f9d^ – src/script/build_debian_image.sh

Voila! We have the script back!

Now is the appropriate time to know that this script uses another one that was also deleted in that same commit (I found it the hard way - by running the script). So let’s just recover the second script also : network-from-cmdline

git checkout c9a6e60eee8719472bcf71fffb20eb2557703f9d^ – src/script/network-from-cmdline

Now that we have everything that we need lets get to work:

cd ~/work/ceph cd src/script ./build_debian_image.sh rootfs.img

This will create in the current directory a file an ext3 filesystem data file describing a debian image. The script uses debootstrap to download and install the necessary packages in the new rootfs so you should make sure you have it installed:

sudo apt-get install debootstrap

If everything is ok you will now have a file called rootfs.img in your current folder.

Unfortunately, I bumped into a problem exactly at the last step and the output was this:

+ umount /tmp/1227
umount: /tmp/1227: device is busy.
		(In some cases useful info about processes that use
				 the device is found by lsof(8) or fuser(1))

It meens that the filesystem was created succesfully but the folder were the file (rootfs.img) was mounted could not be unmounted. The step to solve this are:

* find out what are the processes using files from that folder:

ioana@ceph:~/work/ceph/src/script$ sudo lsof /tmp/1227/
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
	  Output information may be incomplete.
	  COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
	  sshd    18908 root  cwd    DIR    7,0     4096     2 /tmp/1227
	  sshd    18908 root  rtd    DIR    7,0     4096     2 /tmp/1227
	  sshd    18908 root  txt    REG    7,0   475896 23639
	  /tmp/1227/usr/sbin/sshd
	  sshd    18908 root  mem    REG    7,0    51728 57368
	  /tmp/1227/lib/libnss_files-2.11.3.so
	  sshd    18908 root  mem    REG    7,0    43552 57401
	  /tmp/1227/lib/libnss_nis-2.11.3.so
	  sshd    18908 root  mem    REG    7,0    31616 57375
	  /tmp/1227/lib/libnss_compat-2.11.3.so

* as we can see 'sshd' is the only process that restricts us from
  unmounting the rootfs... so let's just terminate it.

  ioana@ceph:~/work/ceph/src/script$ sudo kill -9 18908


* finally unmount the rootfs:

ioana@ceph:~/work/ceph/src/script$ sudo umount  /tmp/ 1227
ioana@ceph:~/work/ceph/src/script$

* check if everything is ok:

ioana@ceph:~/work/ceph/src/script$ file rootfs.img
rootfs.img: linux rev 1.0 ext3 filesystem data,
uuid=7a3dd817-e960-400e-b5d3-69072b08aae3 (large files)
ioana@ceph:~/work/ceph/src/script$ du -sh rootfs.img	689m    rootfs.img

Et voila! We have our own root filesystem that we are going to use to boot our virtual machine in some next tutorial.

Setup the environment - Create the rootfs was originally published by IoanaCiornei at Trudy in Linuxland on January 06, 2016.

Setup the environment - Build ceph and ceph-client

2016-01-06T00:00:00+02:00

Hi, people!

This post is the first from a series of posts in which I will describe in a detailed manner how to create the testing environment for this project.

First, I should mention the fact that I worked on a machine that had Ubuntu 14.04 and that I do not recomend Arch or something like it for doing this kind of setup if you are not ready to spend several days just to catch and find solution for numerous bugs (trust me, I went down that road but my mentor Alex Elder advised me not to fool around and just change the OS to a LTS Ubuntu).

Step 1: Install prerequisite packages

Here is a snippet from a script that I wrote that checks to see if
your system has all the packages and if not it installs them. If you
preffer installing them by hand below is a list.

packages_general=(git vim quilt);
packages_build=(autoconf automake autotools-dev debhelper libatomic-ops-dev
				libboost-dev libcrypto++-dev libcurl4-gnutls-dev libedit-dev
				libexpat1-dev libfcgi-dev libfuse-dev libgoogle-perftools-dev
				libgtkmm-2.4-dev libkeyutils-dev libtool pkg-config python
				python-dev python-pip python-support python-virtualenv uuid-dev
				cython libsnappy-dev libleveldb-dev libblkid-dev libudev-dev
				libnss3-dev libcurl4-nss-dev libaio-dev xfslibs-dev

				libboost-thread-dev libboost-random-dev libboost-regex-dev
				libboost-program-options-dev liblttng-ust-dev libbz2-dev);

Step 2: Get the source code

Go to https://github.com/ceph/ and as you
see there will be more projects that the ceph community is working
on. At the moment, we are interested in ceph and ceph-client.


Just to make everything clean and easy I will describe my folder
structure. In my home directory I have an empty folder 'work' where
I want to place ceph derived things.

# it will take some time depending on the internet connection
cd ~/ceph
git clone https://github.com/ceph/ceph.git
git clone https://github.com/ceph/ceph-client

Step 3: Build ceph

We will build ceph using Autotools and the below instructions can
also be found in ceph/README along other useful things such as
building ceph using cmake (if you prefer it that way).

There are two steps involved in building anything: configuring and
actually building it.

# get to the right directory
cd ~/work
cd ceph

# handle the configuration part
./do_autogen.sh -d3 

# now build ceph

make

or if you have multiple cores/threads it would be better to use

make -j $num_procs

where 'num_procs' is the number of processors of your system. What
it does is to specify the right level of conccurency and therefore speedup
your build. In my case it is: make -j4.


Now you should just go watch a movie, read a book or anything else...
because it may take a while.

Step 4: Build ceph-client

If you browse a little bit the ceph-client folder you may discover
that ceph-client is actually a kernel source appended with anything
that the ceph project needs.

Now I should mention the fact that this step needs another package
installed, so:

sudo apt-get install libncurses5-dev  # in order to user ncurses headers

The first thing to do is to get to the right folder in order to
configure and build the kernel.

cd ~/work/ceph-client

Now is the moment to mention the fact that the only thing that
distinguishes this build from any other one is the right
configuration that can be achieved with the following steps.

Creates a default configuration file .config based
on your system architecture

make defconfig

Add the specific bits that ceph client is interested in, more
exactly CEPH_LIB, CEPH_LIB_PRETTYDEBUG and CEPH_FS.
'make menuconfig' is using ncurses in order to provide an interface
to the configuration process

make menuconfig # only accepted way of editing kernel conf file

It will show a window in the current terminal. If you type  '/' a
search window will appear. The next step is to search for the three
key words and just activate their config options by hitting the 'y'
button. Save and exit. After this in your .config file from the
ceph-client root folder is a fully functional configuration file.

Now lets build the kernel:

make -j4 # by this time you know why is there a '-j4' option

Congratulations! You just finished the first tutorial in order to setup the environment for ceph development and testing.

See you next time!

Setup the environment - Build ceph and ceph-client was originally published by IoanaCiornei at Trudy in Linuxland on January 06, 2016.

Hello, World!

2015-11-22T00:00:00-00:00

Hi, I am Ioana and for the next 3 months I will be an intern at the Linux Foundation through the Outreachy program.

After the application period, that lasted for about 1 mounth, I have been selected to work closely with Alex Elder and the Ceph community.

This blog was created with the main intent to keep you informed about the my progress during this internship.

Hello, World! was originally published by IoanaCiornei at Trudy in Linuxland on November 22, 2015.