Btrfs:RAID Setup
================
:author: Aaron Ball
:email: nullspoon@iohq.net


I recently became very interested in
http://en.wikipedia.org/wiki/Logical_Volume_Manager_%28Linux%29[LVM] and its
ability to have a volume that spans multiple drives. I was just about to do an
LVM setup when I began researching
https://btrfs.wiki.kernel.org/index.php/Main_Page[btrfs] in more depth.  It is
rumored to be the next replacement for
https://ext4.wiki.kernel.org/index.php/Main_Page[ext4], the default Linux
filesystem (in most cases). It also happpens to support volumes that span
multiple devices (http://en.wikipedia.org/wiki/Standard_RAID_levels[raid],
though software raid albeit), aside from a whole
https://btrfs.wiki.kernel.org/index.php/Main_Page#Features[list] of other
functionalities.

Being a person who really enjoys trying new, cool, and often unstable things
(who doesn't love a good learning experience), I decided to set up a
http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5[raid 5] using btrfs
with three whopping one terabyte drives. If all goes well, I should be able to
loose one drive and still have 2 terabytes ( [3-1]*1000 = 2000 ) fully
functional.


[[getting-started]]
== Getting Started

Creating a btrfs filesystem is as simple as creating an ext4 filesystem (or any
other filesystem for that matter). You use the mkfs command.  However, I
created a raid setup, so I needed a few more parameters.  Here's what I used.

----
mkfs.btrfs -m raid5 -d raid5 /dev/sdb /dev/sdc /dev/sdd
----

Well that was easy. What'd we just do?

[cols=",",options="header",]
|================================================================
|mkfs.btrfs |duh
|-m raid5 |Sets the metadata up to use raid 5
|-d raid5 |Set the data up to use raid 5
|/dev/sdb /dev/sdc /dev/sdd |Span our volume across these devices
|================================================================

With that, you should now [very quickly] have a new raid 5 (or whatever you
selected for your raid levels). To mount it, run the mount command on any of
the raw devices in your raid.

----
mount -t btrfs /dev/sdb /mnt/oh_heyyy
----


[[compression]]
== Compression

Btrfs supports various kinds of seamless compression. The default is none since
compression will cause a performance hit (naturally). I thought I'd give it a
try anyways. I set up lzo compression (supposedly the fastest compression, but
less effective) about half way through my sync job (forgot to do it initially).
The original total size of the files in each home directory came to 386 GB
(lots of users for a home system). The end result after compression was 377 GB,
so I ended up saving 9 GB of space while still getting an amazing transfer rate
(see the link:#Benchmarks[ benchmarks] section). Keep in mind though that I
enabled compression after I had already synced a good 100 GB of files, so there
is a good portion of that data that isn't compressed. Despite that though, 9 GB
of space isn't too bad, especially given the performance.


[[handy-commands]]
== Handy Commands

Here's what commands I'm using most frequently up to this point.

* **btrfs fi**[lesystem] **show**: Shows a list of filesystems and their
corresponding devices.
* **btrfs fi**[lesystem] **label <dev> <label>**: Changes the label of
the specified raid device.
* **btrfs fi**[lesystem] **df /path/to/mount**: Displays real df data
about the mounted volume.


[[benchmarks]]
== Benchmarks

I know there are other ways to benchmark storage io, but I wanted to see what
the maximum write speed would be and I don't have a second raid set up to get a
high transfer rate in, so my fastest option at this point is /dev/zero. Here's
my setup (again).

* My btrfs raid 5 is mounted at /home/. The raid is made up of three 1
TB Western Digital Green drives, each at 7200 rpm and it is mounted with "-o
compress=lzo".
* The OS itself ( / ) is installed on a single HDD, a 7200 rpm 500 GB
Maxtor (slightly olde).

[[btrfs-raid-performance]]
=== Btrfs Raid Performance

First, we test writing 2000 1M blocks of zeros to /home/, the raid.

----
[root@zion ~]# dd if=/dev/zero of=/home/bench.test bs=1M count=2000
2000+0 records in 2000+0 records out 2097152000 bytes (2.1
GB) copied, 6.24284 s, 336 MB/s
----

336 MB/s! Not bad for a homemade drive array using software raid and
some desktop drives.


[[non-raid-single-hdd-performance]]
=== Non-Raid Single HDD Performance

Starting with the same as the last but writing to /root/, the single HDD, we
get...

----
[root@zion ~]# dd if=/dev/zero of=/root/bench.test bs=1M count=2000
2000+0 records in 2000+0 records out 2097152000 bytes
(2.1 GB) copied, 30.5043 s, 68.7 MB/s
----


[[impressions]]
== Impressions

I might not be well versed enough in the area of storage, but setting up a
btrfs raid was really easy. I did have to learn all the different raid levels
to decide which I wanted to use of course, but I would have done that anyways.
The filesystem (again, spanning 3 TB) was created ridiculously fast (as fast as
I hit the enter key). I performed an rsync from my old drive (a single 500 GB
HDD, 7200 rpm, 3 Gbit/s) to the new raid (2 TB across 3 HDDs, 7200 rpm, 6
Gbit/s) volume and got about a 31 MB per second transfer rate, which is the max
transfer rate that my single 500 GB drive has ever done anyways, so at least
btrfs can perform that well (not that that's particularly amazing). I was also
very impressed by the 336 MB/s write speed of the raid array. Perhaps I'm
ignorant at this point in time, but that seems pretty impressive for some cheap
off-the-shelf desktop drives. They're not even 10k rpm, let alone 15k. I would
certainly say that from a performance perspective, btrfs is definitely ready
for home use. It may be a little new for enterprise use, but that's up to the
enterprise. For me though, I will keep using it until I see any problems. Even
then, I'll still troubleshoot and then probably continue using it.

Finally, I have to give some serious credit to the guys who wrote the b-tree
filesystem (oddly Oracle sponsored it). It's this kind of open source that
drives the world of technology (not that others don't of course) to expand
beyond "what the consumer wants". You guys are innovating in the coolest ways
and best of all, you're making it freely available. Many thanks guys!


// vim: set syntax=asciidoc: