Storage vMotion taking twice as long when block sizes do not match

While trouble shooting performance issues at a customer I noticed in esxtop that the MB written/s were double the MB read/s when performing a Storage vMotion when the block sizes of the source and target datastores where different.

The customer has two storage devices connected to the same Storage Area Network (SAN); an IBM DS4700 and an IBM DS3400. The DS4700 has Fibre Channel disks in it and the DS3400 SATA disks.

I performed a number of tests with a VM having a 22GB fat vmdk file, using the following datastores: –

VMFS_VOL2 Block Size 8MB FC Disks on the DS4700
VMFS_VOL4 Block Size 4MB FC Disks on the DS4700
VMFS_VOL12 Block Size 4MB SATA Disks on the DS3400

Details of the original tests I performed are shown in the image below, click it to see a larger version.

Tests1

I then changed vmfs_vol12 to an 8MB block size to see what difference that made. Transfer times between datastores with the same block size, i.e. 8MB, where the same as the above times for transfers between datastores with matching 4MB block size. Times when the block sizes where different were double the times when the block sizes matched. Transferring to the SATA disks was slower than transferring off the SATA disks as I expected would be the case. When the block size did not match then the MB written per second where always double the MB read per second regardless of whether the transfer was from the 8MB block size to the 4MB block size or from the 4MB block size to the 8MB block size. See details below

Tests2

Now I wondered what would happen if the block sizes where something other than double the other, so I deleted vmfs_vol12 and recreated it with a 1MB block size, my thoughts were that when transferring between an 8MB block size and a 1MB block size I would see the MB written being 8 times the MB read and the transfer time being 8 times the time when the block sizes matched, and when migrating between a 4MB block size and a 1MB block size the figures would be 4 times. However the MB written were once again double the MB read and almost matched the figures above for when the block sizes where different, as detailed below.

Tests3

The figures in the tables above are rough figures as they fluctuated during the migration.

Here are a couple of screenshots from esxtop from when the block sizes match and when they are different.

BlockSizeDifferent

BlockSizeMatch

It does not seem right that if you migrate a VM between datastores with different block sizes then the transfer takes twice as long as when the datastores have the same block size.

Further investigated required. I have posted details on the VMware ESX 4 VMTN Community, see http://communities.vmware.com/thread/293823?tstart=0

Advertisements
This entry was posted in VMware, vSphere. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s