Using SSD for redo on Exadata - pt 2
Tuesday, December 6, 2011 at 3:31PM In my previous post on this topic, I presented data showing that redo logs placed on an ASM diskgroup created from exadata griddisks created from flash performed far worse than redo logs placed on ASM created from spinning SAS disks.
Of course, theory predicts that flash will not outperform spinning magnetic disk for the sequential write IOs experienced by redo logs, but on Exadata, flash disk performed much worse than seemed reasonable and worse than experience on regular Oracle with FusionIO SSD would predict (see this post).
Greg Rahn and Kevin Closson were both kind enough to help explain this phenomenon. In particular, they pointed out that the flash cards might be performing poorly because of the default 512 byte redo block size and that I should try a 4K blocksize. Unfortunately, at least on my patch level (11.2.2.3.2), there appears to be a problem with setting a 4K blocksize
ALTER DATABASE add logfile thread 1 group 9 ('+DATA_SSD') size 4096M blocksize 4096
*
ERROR at line 1:
ORA-01378: The logical block size (4096) of file +DATA_SSD is not compatible with the disk sector size (media sector size is 512 and host sector size is 512)
According to Greg, the F20 SSD cards are incorrectly reporting their physical characteristics and this is fixed in the current patch level. Luckily, you can override the check by setting
ALTER SYSTEM SET "_disk_sector_size_override"=TRUE SCOPE=BOTH;
Greg and Kevin really know their stuff: setting a 4k redo log block size resulted in dramatic improvements to redo log throughput – elapsed time reduced by 70%:
As expected, redo log performance for SSD still slightly lags that of SAS spinning disks. It’s clear that you can’t expect a performance improvement by placing redo on SSD, but at least the 4K blocksize fix makes the response time comparable. Of course, with the price of SSD being what it is, and the far higher benefits provided for other workloads – especially random reads – it’s hard to see an economic rationale for SSD-based redo. But at least with a 4K blocksize it’s tolerable.
When our Exadata system is updated to the latest storage cell software, I’ll try comparing workloads with the Exadata smart flash logging feature.
Reader Comments (1)
Hi Guy
A few configurations I would do in this setup:
1) Format the fusion card to use 4K blocks (this should also get rid of the error message)
2) pin the Fusion driver to the same core that runs the log writer thread
I am afraid I don't know to find out where the log writer run in Oracle. But I know this works well for SQL server. You pin the driver using the fio-config tool