1 Btrfs:RAID Setup
2 ================
3 :author: Aaron Ball
4 :email: nullspoon@iohq.net
5
6
7 == {doctype}
8
9 I recently became very interested in
10 http://en.wikipedia.org/wiki/Logical_Volume_Manager_%28Linux%29[LVM] and its
11 ability to have a volume that spans multiple drives. I was just about to do an
12 LVM setup when I began researching
13 https://btrfs.wiki.kernel.org/index.php/Main_Page[btrfs] in more depth. It is
14 rumored to be the next replacement for
15 https://ext4.wiki.kernel.org/index.php/Main_Page[ext4], the default Linux
16 filesystem (in most cases). It also happpens to support volumes that span
17 multiple devices (http://en.wikipedia.org/wiki/Standard_RAID_levels[raid],
18 though software raid albeit), aside from a whole
19 https://btrfs.wiki.kernel.org/index.php/Main_Page#Features[list] of other
20 functionalities.
21
22 Being a person who really enjoys trying new, cool, and often unstable things
23 (who doesn't love a good learning experience), I decided to set up a
24 http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5[raid 5] using btrfs
25 with three whopping one terabyte drives. If all goes well, I should be able to
26 loose one drive and still have 2 terabytes ( [3-1]*1000 = 2000 ) fully
27 functional.
28
29
30 [[getting-started]]
31 == Getting Started
32
33 Creating a btrfs filesystem is as simple as creating an ext4 filesystem (or any
34 other filesystem for that matter). You use the mkfs command. However, I
35 created a raid setup, so I needed a few more parameters. Here's what I used.
36
37 ----
38 mkfs.btrfs -m raid5 -d raid5 /dev/sdb /dev/sdc /dev/sdd
39 ----
40
41 Well that was easy. What'd we just do?
42
43 [cols=",",options="header",]
44 |================================================================
45 |mkfs.btrfs |duh
46 |-m raid5 |Sets the metadata up to use raid 5
47 |-d raid5 |Set the data up to use raid 5
48 |/dev/sdb /dev/sdc /dev/sdd |Span our volume across these devices
49 |================================================================
50
51 With that, you should now [very quickly] have a new raid 5 (or whatever you
52 selected for your raid levels). To mount it, run the mount command on any of
53 the raw devices in your raid.
54
55 ----
56 mount -t btrfs /dev/sdb /mnt/oh_heyyy
57 ----
58
59
60 [[compression]]
61 == Compression
62
63 Btrfs supports various kinds of seamless compression. The default is none since
64 compression will cause a performance hit (naturally). I thought I'd give it a
65 try anyways. I set up lzo compression (supposedly the fastest compression, but
66 less effective) about half way through my sync job (forgot to do it initially).
67 The original total size of the files in each home directory came to 386 GB
68 (lots of users for a home system). The end result after compression was 377 GB,
69 so I ended up saving 9 GB of space while still getting an amazing transfer rate
70 (see the link:#Benchmarks[ benchmarks] section). Keep in mind though that I
71 enabled compression after I had already synced a good 100 GB of files, so there
72 is a good portion of that data that isn't compressed. Despite that though, 9 GB
73 of space isn't too bad, especially given the performance.
74
75
76 [[handy-commands]]
77 == Handy Commands
78
79 Here's what commands I'm using most frequently up to this point.
80
81 * **btrfs fi**[lesystem] **show**: Shows a list of filesystems and their
82 corresponding devices.
83 * **btrfs fi**[lesystem] **label <dev> <label>**: Changes the label of
84 the specified raid device.
85 * **btrfs fi**[lesystem] **df /path/to/mount**: Displays real df data
86 about the mounted volume.
87
88
89 [[benchmarks]]
90 == Benchmarks
91
92 I know there are other ways to benchmark storage io, but I wanted to see what
93 the maximum write speed would be and I don't have a second raid set up to get a
94 high transfer rate in, so my fastest option at this point is /dev/zero. Here's
95 my setup (again).
96
97 * My btrfs raid 5 is mounted at /home/. The raid is made up of three 1
98 TB Western Digital Green drives, each at 7200 rpm and it is mounted with "-o
99 compress=lzo".
100 * The OS itself ( / ) is installed on a single HDD, a 7200 rpm 500 GB
101 Maxtor (slightly olde).
102
103 [[btrfs-raid-performance]]
104 === Btrfs Raid Performance
105
106 First, we test writing 2000 1M blocks of zeros to /home/, the raid.
107
108 ----
109 [root@zion ~]# dd if=/dev/zero of=/home/bench.test bs=1M count=2000
110 2000+0 records in 2000+0 records out 2097152000 bytes (2.1
111 GB) copied, 6.24284 s, 336 MB/s
112 ----
113
114 336 MB/s! Not bad for a homemade drive array using software raid and
115 some desktop drives.
116
117
118 [[non-raid-single-hdd-performance]]
119 === Non-Raid Single HDD Performance
120
121 Starting with the same as the last but writing to /root/, the single HDD, we
122 get...
123
124 ----
125 [root@zion ~]# dd if=/dev/zero of=/root/bench.test bs=1M count=2000
126 2000+0 records in 2000+0 records out 2097152000 bytes
127 (2.1 GB) copied, 30.5043 s, 68.7 MB/s
128 ----
129
130
131 [[impressions]]
132 == Impressions
133
134 I might not be well versed enough in the area of storage, but setting up a
135 btrfs raid was really easy. I did have to learn all the different raid levels
136 to decide which I wanted to use of course, but I would have done that anyways.
137 The filesystem (again, spanning 3 TB) was created ridiculously fast (as fast as
138 I hit the enter key). I performed an rsync from my old drive (a single 500 GB
139 HDD, 7200 rpm, 3 Gbit/s) to the new raid (2 TB across 3 HDDs, 7200 rpm, 6
140 Gbit/s) volume and got about a 31 MB per second transfer rate, which is the max
141 transfer rate that my single 500 GB drive has ever done anyways, so at least
142 btrfs can perform that well (not that that's particularly amazing). I was also
143 very impressed by the 336 MB/s write speed of the raid array. Perhaps I'm
144 ignorant at this point in time, but that seems pretty impressive for some cheap
145 off-the-shelf desktop drives. They're not even 10k rpm, let alone 15k. I would
146 certainly say that from a performance perspective, btrfs is definitely ready
147 for home use. It may be a little new for enterprise use, but that's up to the
148 enterprise. For me though, I will keep using it until I see any problems. Even
149 then, I'll still troubleshoot and then probably continue using it.
150
151 Finally, I have to give some serious credit to the guys who wrote the b-tree
152 filesystem (oddly Oracle sponsored it). It's this kind of open source that
153 drives the world of technology (not that others don't of course) to expand
154 beyond "what the consumer wants". You guys are innovating in the coolest ways
155 and best of all, you're making it freely available. Many thanks guys!
156
157
158
159 Category:Linux
160 Category:Btrfs
161 Category:Storage
162 Category:RAID
163
164
165 // vim: set syntax=asciidoc:
|
