« Getting started with Apache Pig | Main | Amazon Elastic Map Reduce (EMR), Hive, and TOAD »
Tuesday
Dec062011

Using SSD for redo on Exadata - pt 2

In my previous post on this topic, I presented data showing that redo logs placed on an ASM diskgroup created from exadata griddisks created from flash performed far worse than redo logs placed on ASM created from spinning SAS disks.

Of course, theory predicts that flash will not outperform spinning magnetic disk for the sequential write IOs experienced by redo logs, but on Exadata, flash disk performed much worse than seemed reasonable and worse than experience on regular Oracle with FusionIO SSD would predict (see this post).

Greg Rahn and Kevin Closson were both kind enough to help explain this phenomenon.  In particular, they pointed out that the flash cards might be performing poorly because of the default 512 byte redo block size and that I should try a 4K blocksize.   Unfortunately, at least on my patch level (11.2.2.3.2), there appears to be a problem with setting a 4K blocksize

ALTER DATABASE add logfile thread 1 group 9 ('+DATA_SSD') size 4096M blocksize 4096
*
ERROR at line 1:
ORA-01378: The logical block size (4096) of file +DATA_SSD is not compatible with the disk sector size (media sector size is 512 and host sector size is 512)

According to Greg, the F20 SSD cards are incorrectly reporting their physical characteristics and this is fixed in the current patch level.   Luckily, you can override the check by setting

ALTER SYSTEM SET "_disk_sector_size_override"=TRUE SCOPE=BOTH;

Greg and Kevin really know their stuff:  setting a 4k redo log block size resulted in dramatic improvements to redo log throughput – elapsed time reduced by 70%:

image

As expected,  redo log performance for SSD still slightly lags that of SAS spinning disks.     It’s clear that you can’t expect a performance improvement by placing redo on SSD, but at least the 4K blocksize fix makes the response time comparable.  Of course, with the price of SSD being what it is, and the far higher benefits provided for other workloads – especially random reads – it’s hard to see an economic rationale for SSD-based redo.    But at least with a 4K blocksize it’s tolerable.

When our Exadata system is updated to the latest storage cell software, I’ll try comparing workloads with the Exadata smart flash logging feature.

References (23)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    Exadata storage software 11.2.2.4 introduced the Smart flash logging feature. The intent of this is to
  • Response
    Exadata storage software 11.2.2.4 introduced the Smart flash logging feature. The intent of this is to
  • Response
    Rather great post, really useful stuff. Never ever believed We would obtain the data I want listed here. Photograph looking on net for some time now and was starting to get frustrated. Fortunately, I came across your web site and received what Im struggling to find.
  • Response
    It’s been tough to find time to do actual performance research of late, but I have managed to get a test
  • Response
    It’s been tough to find time to do actual performance research of late, but I have managed to get a test
  • Response
    It’s been tough to find time to do actual performance research of late, but I have managed to get a test
  • Response
    Of all the claims I make about SSD for Oracle databases, the one that generates the most debate is that
  • Response
    Of all the claims I make about SSD for Oracle databases, the one that generates the most debate is that
  • Response
    Of all the claims I make about SSD for Oracle databases, the one that generates the most debate is that
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: Forskolin Amazon
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: Phen375 Australia
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: you could try here
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: Instagram
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: Instagram
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: Online advertising
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Response: Lenkrad
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2
  • Response
    Guy Harrison - Yet Another Database Blog - Using SSD for redo on Exadata - pt 2

Reader Comments (1)

Hi Guy

A few configurations I would do in this setup:

1) Format the fusion card to use 4K blocks (this should also get rid of the error message)
2) pin the Fusion driver to the same core that runs the log writer thread

I am afraid I don't know to find out where the log writer run in Oracle. But I know this works well for SQL server. You pin the driver using the fio-config tool

October 8, 2012 | Unregistered CommenterThomas Kejser

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>