Chunk Size Of a RAID0 Volume On a QNAP NAS – What’s The Sweet Spot?

Prologue

By default the QNAP’s AJAX based volume management wizard uses a 64KB chunk size (aka cluster or block size) when it creates a RAID0 volume. I have also noticed that the wizard uses the same chunk size for RAID5 and RAID6 volumes.

For my VMware home lab I wanted to change that value because mainly my typical IO length is 4KB. Also I wanted to test other values to better understand the impact on my VMware home lab environment. Thus I decided to deep dive this well know software RAID management tool for many Linux based installation and especially this QNAP device, that is MDADM.

So as I told you by default the nice QNAP’s AJAX based wizard set the chunk size to 64KB. Is it a good value? Is it an one size fit all? That’s what I wanted to figure out. So I managed to create a software RAID0 volume, every time with a different chunk size value, and I tested each scenario with my IOmeter’s special icf file. What I have found out is that in many different scenarios, 64KB happens not to be the sweet spot! Before I go on, let’s answer some questions.

what is the chunk size?

The chunk size is in multiples of two and units of kilobytes, and it defines the size of the stripes for the array. If the chunk size is too small, then the RAID subsystem will have to spend a lot of time talking to the Linux kernel layer. If the chunk size is too large, then the RAID subsystem will have to spend too much time waiting on the physical storage layer.

How could you change the chunk size when  QNAP’s AJAX based wizard set the chunk size to 64KB by default and there is no way to change that?

Basically it is a two-step process. First step you create manually the RAID0 volume using the MDADM tool and not the QNAP’s AJAX based wizard. MDADM allows you to specify the chunk size whilst creating the RAID0 volume. And step two, you finalize the process with the QNAP’s AJAX based wizard. The difference is that the software RAID volume is already setup therefore the wizard skips the ‘create volume’ steps leaving the chunk size the way it was set in step one.

First step: command line

mdadm –create /dev/md0 -l 0 -c 4 -n 2 /dev/sda3 /dev/sdb3

-c or –chunk= That’s the parameters to define the chunk size

mdadm: /dev/sda3 appears to contain an ext2fs file system
size=582934400K mtime=Thu Nov 25 17:40:43 2010
mdadm: /dev/sda3 appears to be part of a raid array:
level=raid0 devices=2 ctime=Thu Nov 25 17:37:15 2010
mdadm: /dev/sdb3 appears to be part of a raid array:
level=raid0 devices=2 ctime=Thu Nov 25 17:37:15 2010
Continue creating array? y
mdadm: array /dev/md0 started.

Second step: the QNAP’s AJAX based wizard

Once the manual step is done, back to the QNAP’s AJAX based console, go to Disk management, them Volume management and click in the right pane on Create one striping disk volume.

Select the two disks that are part of the striping disk volume and click on Create.

You can see that the Striping Disk Volume is being initialized…

Then the QNAP’s AJAX based wizard formats the striping volume. This can take a while depending on the size of the volume.

Format is done. Note that I’m using EXT4 and that the default cluster size is 4KB. That’s something to keep in mind when later on we will see the benchmark results. If I would have formated with another cluster size, my bet the benchmark results would have been different…

To confirm the chunk size, let’s go back at the command line for a second. You can get the details of a volume by using the -D parameter. As you can see on the screenshot above, the chunk size is 4K.

And the QNAP’s AJAX based wizard has properly amended the /etc/mtab with a mount point to the newly created striping volume. Job done.

The benchmark test results

Now here are the results I got with three different chunk size values; 4KB, 64KB and 2048KB. As I told you at the beginning of this blog post, 64K is definitely not the best chunk size in my environment although it is the default value for a 100% QNAP’s AJAX based wizard driven software RAID setup.

What we can observe from the graph, is that the 4KB wins in almost any IOmeter tests except for Series 1 where amazingly a chunk size of 2048KB, that is 2MB, wins by less than 100 IOPS over a total of 2769, that is less than 2,5%.

A 64KB chunk size never won any of the eleven different tests I have conducted. Although it is the default value when creating a software RAID volume, it is not the best one when compared to a 4KB chunk size. Why QNAP picked up that value? That’s an open question so far…

The 2048KB chunk size is large, very large for such a NAS storage device with no hardware RAID controller but eventually it is better that a 64KB chunk size in three out of eleven IOmeter tests. Who could imagine that!

Epilogue

Because there is no easy way to determine what a good chunk size is for a particular RAID array and because it’s heavily dependent on the hardware being used and the intended use of the array. I strongly suggest you benchmark several scenario’s, that is several chunk size, and pick up the best one that suits your needs and requirements and especially your typical IO length. For my VMware home lab, I built a RAID0 volume with a 4KB chunk size because my typical IO length is 4KB.

And you, do you know your typical IO length? Maybe it’s time to check that up 🙂