ZFS: WriteBack versus SSD-ZIL

[solaris-Z8efaSeK1ezqlBn2x/YWAg at public.gmane.org (Edward Ned Harvey)]

In this email, when I say PERC, I really mean either a PERC, or any other
hardware WriteBack buffered raid controller with BBU.

 

For future server purchases, I want to know which is faster:  (a) A bunch of
hard disks with PERC and WriteBack enabled, or (b) A bunch of hard disks,
plus one SSD for ZIL.  Unfortunately, I don't have an SSD available for
testing.  So here is what I was able to do:

 

I measured the write speed of the naked disks (PERC set to WriteThrough).
Results were around 350 ops/sec.

I measured the write speed with the WriteBack enabled.  Results were around
1250 ops/sec.

 

So right from the start, we can see there's a huge performance boost by
enabling the WriteBack.  Even for large sequential writes, buffering allows
the disks to operate much more continuously.  The next question is how it
compares against the SSD ZIL.

 

Since I don't have an SSD available, I created a ram device and put the ZIL
there.  This is not a measure of the speed if I had an SSD; rather, it is a
measure which the SSD cannot possibly achieve.  So it serves to establish an
upper bound.  If the upper and lower bounds are near each other, then we
have a good estimate of the speed with SSD ZIL . but if the upper bound and
lower bound are far apart, we haven't gotten much information.

 

With the ZIL in RAM, results were around 1700 ops/sec.

 

This measure is very far from the lower bound.  But it still serves to
provide some useful knowledge.  The take-home knowledge is:

.         There's a lot of performance to be gained by accelerating the ZIL.
Potentially up to 6x or 7x faster than naked disks.

.         The WriteBack raid controller achieves a lot of this performance
increase.  About 3x or 4x acceleration.

.         I don't know how much an SSD would help.  I don't know if it's
better, the same, or worse than the PERC.  I don't know if the combination
of PERC and SSD together would go faster than either one individually.

 

I have a hypothesis.  I think the best configuration will be to use a PERC,
with WriteBack enabled on all the spindle hard drives, but include an SSD
for ZIL, and set the PERC for WriteThrough on the SSD.  This has yet to be
proven or disproven.