www.alexpage.de

Posted: **Tue Apr 12, 2016 11:09 am**

I have four servers running VMWare 5.5
I'm studing cloud.
I'd like to backup the USB pen drive, which contains configuration and other data related to the VMWare host.
Here follows data about data storage

FD01: \\?\usbstor#disk&ven_kingston&prod_datatraveler_2.0&rev_pmap#50e549c695a9bd4089410930&0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
FD02: Kingston DataTraveler 2.0 USB Device
FD04: Device ID is USBSTOR\DISK&VEN_KINGSTON&PROD_DATATRAVELER_2.0&REV_PMAP\50E549C695A9BD4089410930&0
FD05: Media Type is 11
FD06: Location DataTraveler 2.0
FD07: Hardware ID USB\Vid_0951&Pid_1665&Rev_0100
FD08: Device Number is 2
FD09: FOUND!

Each server has 'hyperisor image saved on 16GB pen drive
The servers are operating normally
I used your sightseeing program for backing

I can not explain why, with 2 USB pen drive, I was able to do the backup, but with other 2 did not succeed. It appears "cyclic redundancy error"
Ideas?

yet the pen drives work. The proof is that the server turns on, reads data from the pen drive and it works
Any help will be appreciated

Posted: **Tue Apr 12, 2016 7:55 pm**

The protocol is just logging the device detection.

I don't have experience with VMWare Hypervisor myself, but I know that other people tried to use USB Image Tool for that purpose. Maybe Google has some hits when searching for USB Image Tool and VMWare ESX.

Theoretically, the image should be a 1:1 byte copy of the USB drive.

Posted: **Thu Apr 14, 2016 2:50 pm**

Hi flaviog2005,

I've also an EXSi 5.5 running from 16GB SD-Card and tried to make a backup. First I cloned the SD-Card to an image using dd. After that I tried to restore the image using USB Image Tool but the destination sd card did not boot. The Error was

Code: Select all

BANK5: invalid configuration
BANK6: not a VMware boot bank
No hypervisor found.

After investigation of user 'fgrehl' (see link at the end) we figured out that the problem rises due to a screwed up the partition numbering. While the source sd-card hat partitions 1,5,6,7,8,9 the destination sd-card had partitions 1,2,3,4,5,6. ESXi expects it boot partitions on partition 5 and 6 so it is obvious why booting fails. The start- and end sectors of the different partitions where right, but the numbers within the GPT were incorrect.
Using dd to restore the image does not have that problem, it doesn't even matter, that the destination sd-card has some more sectors although both are mentioned to have 16GB space.
I don't know if that might help for your problem but at least there seems to be some problems with current version 1.72 of USB Image Tool to make a real 1:1 byte copy.

So here is what I did:
1) dd if=/dev/disks/mpx.vmhba32:C0:T0:L0 | gzip > /vmfs/volumes/WHATEVER/esx55.img.gz
2) restore with USB Image Tool using compressed gzip image file (don't worry about the progress bar running over 100% up to 500% and more)
3) I repaired the wrong GTP with gdisk on a live linux by writing down the sector numbers of partition 2-6 deleting partition 2-6 and recreating them with the original numbers 5-9 and the saved sector numbers

You can see the full tutorial and the reports on that bug here
http://www.virten.net/2014/12/clone-esx ... sh-drives/

Posted: **Thu Apr 14, 2016 6:37 pm**

Thanks for reporting that link. I will try to figure out what happens here with the partition numbering. Intentionally USB Image Tool doesn't touch partitions in device mode, it's operating on the whole device as provided by Windows API. Maybe there are some side effects, that come into effect here. I will try to recreate the issue with multiple partitions on a SD Card.

Posted: **Fri Apr 15, 2016 7:49 am**

Let me know if you need additional informations.
I used Win 7 64Bit version on a Lenovo X201 (which is still BIOS driven). Maybe that is a side effect, because the sd-card uses a GPT but as far as I know Windows 7 can only boot from GPT disk on a UEFI system. For sure I did not try to boot (I just wanted to clone the sd-card) but maybe the combination of BIOS / GPT raises some problems? I'm no expert in that...

PS: Googling how to clone an ESXi sd-card shows up some more threads in which users are struggling with the same issue:
https://community.spiceworks.com/topic/ ... ry-3368881
http://www.y-m-e.net/2014/06/vmware-esx ... nd-apu1c4/

Posted: **Wed Mar 15, 2017 1:29 pm**

Hi Alex,

did you have time to investigate into that issue or are there any other new informations?
If I remember correctly you mentioned that you wanted to figure out, what is going wrong with the GPT numbering when when performing 1:1 byte copy.

Posted: **Mon Jul 03, 2017 6:28 pm**

While searching the internet for USB Image Tool use cases, I found a possible reason/solution for that issue. It could be, that a backup GUID partition table at the end of the flash drive interferes with the new image. This might happen, when you restore a smaller image to a larger flash drive. Next option of USB Image Tool will contain an option to fill the remaining bytes if the new flash drive with zeroes. Current option would (possibly) be to do a full re-format of the flash drive (no quick format) and then restore the image.

https://www.thomas-krenn.com/de/wiki/VM ... ick_klonen

https://kb.vmware.com/selfservice/micro ... Id=2012022

Update (2017-07-31): Version 1.74 adresses this with a new option "Initialize remaining space after restore".

Posted: **Thu Nov 09, 2017 5:49 pm**

Hi Alex,

I just tested your new version 1.74 thanks for investigating into the problem!

First the bad news:
On my Windows 7 64bit (Lenovo X201) the new option doesn't help to fix that problem. The numbers of the gpt are still messed up after writing an image back. This happens to SD Cards as well as to USB-Sticks.

The good news:
In order to narrow that problem down (I thought it is a problem of the OS Windows 7) I used a Linux VM (within Virtual Box) on my Windows 7 host. I passed the USB Device using the extension pack to the VM.
Within the VM I used 'dd' to dump my source image back to a SD Card (as well as an USB Stick). When 'dd' finished the writing process I could see the following output from kernel in my syslog:

Code: Select all

Nov  9 16:17:43 sol kernel: [40711.142290] GPT:Primary header thinks Alt. header is not at the end of the disk.
Nov  9 16:17:43 sol kernel: [40711.142297] GPT:30894079 != 61734911
Nov  9 16:17:43 sol kernel: [40711.142297] GPT:Alternate GPT header not at the end of the disk.
Nov  9 16:17:43 sol kernel: [40711.142298] GPT:30894079 != 61734911
Nov  9 16:17:43 sol kernel: [40711.142298] GPT: Use GNU Parted to correct GPT errors.
Nov  9 16:17:43 sol kernel: [40711.142315]  sdd: sdd1 sdd5 sdd6 sdd7 sdd8 sdd9
Nov  9 16:55:17 sol kernel: [42965.876403] GPT:Primary header thinks Alt. header is not at the end of the disk.
Nov  9 16:55:17 sol kernel: [42965.876407] GPT:30894079 != 61734911
Nov  9 16:55:17 sol kernel: [42965.876407] GPT:Alternate GPT header not at the end of the disk.
Nov  9 16:55:17 sol kernel: [42965.876408] GPT:30894079 != 61734911
Nov  9 16:55:17 sol kernel: [42965.876408] GPT: Use GNU Parted to correct GPT errors.
Nov  9 16:55:17 sol kernel: [42965.876424]  sdd: sdd1 sdd5 sdd6 sdd7 sdd8 sdd9

It can be seen, that the source image contains the correct partitions (1,5 to 9) but due to some little space difference of the original source drive compared to the destination drive the gpt header is not at the end of the new disk. So far that is not a problem. If you now fix that error as recommended using 'parted' or 'gparted' everything is fine and you're done. The new drive regardless if SD Card or USB Stick will boot an ESXi hypervisor as expected.

BUT:
If you don't correct that gpt error and disconnect the USB device from the VM so that it is passed back to the Windows 7 host and after that (without any further action) connect the device back to the Linux VM you can see the following output from kernel:

Code: Select all

Nov  9 16:56:58 sol kernel: [43066.348328] scsi 5:0:0:0: Direct-Access     JetFlash Transcend 32GB   1.00 PQ: 0 ANSI: 6
Nov  9 16:56:58 sol kernel: [43066.361045] sd 5:0:0:0: [sdd] 61734912 512-byte logical blocks: (31.6 GB/29.4 GiB)
Nov  9 16:56:58 sol kernel: [43066.366923] sd 5:0:0:0: [sdd] Write Protect is off
Nov  9 16:56:58 sol kernel: [43066.372798] sd 5:0:0:0: [sdd] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA
Nov  9 16:56:58 sol kernel: [43066.510941]  sdd: sdd1 sdd2 sdd3 sdd4 sdd5 sdd6
Nov  9 16:56:58 sol kernel: [43066.544906] sd 5:0:0:0: [sdd] Attached SCSI removable disk

The gpt table now contains partitions (1 to 6) and with that you will not be able to boot an ESXi hypervisor. You will get the well known error when trying to boot

Code: Select all

BANK5: not a VMware boot bank
BANK6: not a VMware boot bank
No hypervisor found.

because the ESXi hypervisor is expected on partition 5 or 6 which now corresponds to partition 8 and 9. It seems to me, that Windows 7 OS fixes the gpt error in background without any notice to the user and thereby messing up the partition numbers.

In order to confirm that theory I used 'sgdisk --backup=FILE_NAME /dev/DRIVE_NAME' in my Linux VM to dump the gpt (after 'dd' had finished and before disconnecting the USB device from the Linux VM) to a file. With this backup I could easily repair the incorrect gtp from my Windows 7 host using

Code: Select all

sgdisk --load-backup=FILE_NAME /dev/DRIVE_NAME

After that the SD Card or USB Stick can be used again to boot an ESXi hypervisor.

Q: Where is the link to USB Image Tool?
A: USB Image Tool does a low level copy and writes the correct gpt table from an image to a new device (SD Card / USB Stick). But as the gtp table is not "error free" Windows 7 fixes the error in background by renumbering the partition numbers. I don't know if there is a chance to handle that within USB Image Tool. This is just the result of my evaluations. Does that make sense?

Kind regards
KingTutt

Posted: **Thu Nov 09, 2017 6:48 pm**

It sounds definitely like this could be the reason for that behaviour. I had similar issues with locking and Windows modifying the device before verification could take place (http://www.alexpage.de/forum/viewtopic.php?f=1&t=526).

The new option from version might prevent an old backup GPT from directly at the end of the device will interfere with the new image after restore, but if Windows tries to repair the new GPT which could be at any position on the new device and messes up partition numbering, it will become tricky. I would have to make sure there is a valid backup GPT at the end of the device after each restore. Until now I tried not to modify the device except plain writing of the image. Looks like I'll have to investigate the GPT format more in detail see what I can do about that behaviour. Maybe it's an option to duplicate the master GPT at the end of the device.

Thanks for the detailed evaluation!

Posted: **Mon Nov 13, 2017 6:30 pm**

But I'm not sure if the problem is, that "there is an old GPT somewhere on the device". It seems to, because the kernel complains about that, but:
After writing an image to a new disk which is bigger than the original device first there is not a backup of the GPT at the end of the new device and second the content of the primary GPT is wrong regarding the amount of available sectors.
Here is an example where the source device hat 16GB and the destination device 32GB of memory.

Code: Select all

[root@test]# sgdisk --print /dev/sdd
Disk /dev/sdd: 61734912 sectors, 29.4 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 72FC21C9-FF06-496C-BA07-108A1DA284C8
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 30894046
Partitions will be aligned on 32-sector boundaries
Total free space is 23808125 sectors (11.4 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64            8191   4.0 MiB     EF00  
   5            8224          520191   250.0 MiB   0700  
   6          520224         1032191   250.0 MiB   0700  
   7         1032224         1257471   110.0 MiB   FC00  
   8         1257504         1843199   286.0 MiB   0700  
   9         1843200         7086079   2.5 GiB     FC00  
[root@test]#

The partition table is right regarding the numbering but while the device has got 61734912 sectors only 30894046 sectors are used. When now using gparted or fdisk, both tools ask the user for correcting the available amount of sectors and of cause by writing they will write implicitly a backup GPT at the end of the device.

Using gparted:

Code: Select all

gparted /dev/sdd
Not all of the space available appears to be used, you can fix GPT to use all
of the space (an extra 30840832 blocks)

Using fdisk

Code: Select all

[root@test]# fdisk -l -u /dev/sdd
GPT PMBR size mismatch (30894079 != 61734911) will be corrected by w(rite).
Festplatte /dev/sdd: 29,4 GiB, 31608274944 Bytes, 61734912 Sektoren
Einheiten: Sektoren von 1 * 512 = 512 Bytes
Sektorgröße (logisch/physikalisch): 512 Bytes / 512 Bytes
E/A-Größe (minimal/optimal): 512 Bytes / 512 Bytes
Festplattenbezeichnungstyp: gpt
Festplattenbezeichner: 72FC21C9-FF06-496C-BA07-108A1DA284C8

Gerät       Anfang    Ende Sektoren Größe Typ
/dev/sdd1       64    8191     8128    4M EFI-System
/dev/sdd5     8224  520191   511968  250M Microsoft Basisdaten
/dev/sdd6   520224 1032191   511968  250M Microsoft Basisdaten
/dev/sdd7  1032224 1257471   225248  110M unbekannt
/dev/sdd8  1257504 1843199   585696  286M Microsoft Basisdaten
/dev/sdd9  1843200 7086079  5242880  2,5G unbekannt
[root@test]#

According to the "nature of MS Windows systems" I would guess that the real problem is that the GPT does not contain the correct available amount of sectors for the device. Windows simply fixes that by writing a new GPT (without asking the user) and thereby messing up the numbering of the partitions instead of just fixing the amount of sectors.

www.alexpage.de

USB with Hypervisor ESXi 5.1 image

USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image

Re: USB with Hypervisor ESXi 5.1 image