These instructions are for VMware vSphere ESX 4.x, and they explain how to reformat an existing local VMFS-3 partition on your ESX server, which allows you to change the block size.
Note: These instructions do not apply to ESXi 4.x versions.
When performing a default installation of ESX 4.x, the installer formats the VMFS-3 partition with a 1 MB block size. Unfortunately, this only supports individual file sizes up to 256 GB, and this means you cannot have a VMDK file greater than that size.
Situation
P2V an old Windows SQL Server with a combined RAID-5 disk of over 500 GB. There’s currently no room on the SAN, so until additional disk can be added to the SAN, the source physical server needs to end up on the local disk of an ESX server in the cluster.
PROBLEM
A default installation was performed on the ESX server, and therefore, the VMFS partition on the local disk has a block size of 1 MB.
RESOLUTION
Most of what follows is explained in VMware KB article 1013210. After reading how this situation was resolved, read the KB article. It includes a lot of additional information for which you should be familiar.
WHAT’S REQUIRED
- Physical access to the ESX server.
- Physical console connection.
- Root password for the ESX server.
- An ESX 4.x installation CD.
- Pencil and paper.
- A remote SSH server with at least 12 GB of free space, a gigabit network connection between it and your ESX server, and preferably on the same network segment.
- Approximately 1 hour of downtime.
PROCESS SUMMARY
- Boot the ESX server with the installation CD.
- Copy the contents of the Service Console OS directory to a remote SSH server.
- Reformat the original VMFS volume with the desired block size.
- Copy the Service Console OS directory back to the ESX server.
- Reconfigure the ESX server.
- Verify that the ESX server can boot without intervention or errors.
The Process
Preparation:
From within the vSphere Client, go to the Configuration tab of the ESX host, choose Storage, select the local datastore that needs to be reformatted, open its Properties, and write down its Extent Name.
It should look similar to this:
(mpx.vmhba0:C0:T0:L0):5
– or this –
(naa.58039d39a98fd...145ef87):5
Also note the Adapter Details of the NIC that will be used to connect to the remote SSH server. You may need this if you plan to keep multiple NICs connected during this process. Especially if the other NICs are not on the same network segment.
If you have a clustered environment, migrate the VMs off the ESX server. If you have VMs on the datastore you wish to reformat, you will need to move them to another datastore. Otherwise, shut the VMs down.
- Put the ESX server into Maintenance Mode, and then shut it down.
- Optional: Disconnect any unneeded NICs and SAN connections.
- Boot into the ESX 4.x installer, and select the Graphical Installer.
- Work through each stage of the installer, and when prompted about the license, choose the Evaluation Mode.
- Complete the network configuration steps, and then click the Test button. This will commit the network settings.
- Open an alternate TTY session (Ctrl+Alt+F2), and press Enter.
- Test network connectivity. Can you ping the SSH server?
- Copy the contents of the ESX console OS directory to the SSH server.
scp -r /vmfs/volumes/<Storage Name>/esxconsole-<Console UID> <username>@<SSH Server IP>:/<Destination Directory>
Reformat the VMFS volume:
First verify the Extent Name of the local storage VMFS partition:
ls /vmfs/devices/disks/
The output of the ls command should contain the same Extent Name that was written down earlier (minus the parenthesis). If not, reboot the ESX server normally, and re-verify the Extent Name.
If you have a match, reformat the original VMFS volume with the desired block size.
vmkfstools -C vmfs3 -b <Block Size> -S <Storage Name> /vmfs/devices/disks/<Extent Name>
Block Size Choices:
Block Size Largest File Size 1m 256 GB 2m 512 GB 4m 1 TB 8m 2 TB
When the reformatting is complete, vmkfstools will output the new Volume UID. Carefully write it down, and then verify that it’s correct.
Copy the Service Console OS directory back from the SSH server:
scp -r <username>@<SSH Server IP>:/<Copied Directory> /vmfs/volumes/<Storage Name>
Reconfigure the ESX server:
List the contents of the VMFS volume that has the Service Console OS directory, and carefully write down the UID included in the Service Console directory name. Then verify that it’s correct.
ls /vmfs/volumes/<Storage Name>
Eject the installation CD, and then reboot the ESX server.
During start up, the GRUB boot loader shows some boot-up options, and gives a 5 second countdown. Interrupt the countdown by hitting one of the cursor keys (aka. Arrow keys).
Use the Up/Down arrow keys to highlight ESX Server 4.x, and then hit “a” on the keyboard to edit the kernel boot parameter.
Add the now-required “/boot/cosvmdk” parameter to the kernel boot line by using the Volume UID and Service Console OS directory UID that was written down earlier, and then verify it.
What it looks like before editing the parameter:
grub append> ro root=UUID=<Root UUID> mem=300M quiet
What it looks like after editing the parameter:
grub append> /boot/cosvmdk=/vmfs/volumes/<Volume UID> /esxconsole-<Console UID>/esxconsole.vmdk ro root=UUID=<Root UUID> mem=300M quiet
Note: Double check what’s inputted. Errors may result in a boot-up failure. If an error occurs, re-verify what was entered in the kernel boot parameter.
Press Enter to save the changes. The boot process will resume.
When the ESX server boots up cleanly, login to the console with root credentials. Alt-F1 will show the login prompt.
Note: If the ESX server is a member of a cluster, “vmware-aam” will timeout with a failure. This is ok.
Using a text editor, edit /etc/vmware/esx.conf, locate the line that specifies the Console OS vmdk path, and replace the old Volume UID with the new Volume UID. The Service Console UID should be the same. Verify the changes.
/boot/cosvmdk = "/vmfs/volumes/<New Volume UID>/esxconsole- <Console UID>/esxconsole.vmdk"
Save the changes, and exit out of the text editor to return to the prompt. Then run the following command to update the ESX server boot configuration and initial RAM disk image:
esxcfg-boot -b
Reboot the ESX server to verify that the server boots without error.
Note: If the ESX server is a member of a cluster, “vmware-aam” will timeout with a failure. This is ok.
Take the ESX server out of Maintenance Mode, and then start up the VMs or migrate them back.