Automatic Coredump Expansion in vSphere
For information regardring vSphere 7 changes to system partitions, see this link.
Coredump partitions or a coredump file service is used in the case of a host failure (hardware or software) with a purple diagnostic screen. This screen is often referred to as a “PSOD” or “purple screen of death” with ESXi. A coredump is generated, which contains a wealth of information about the state of the system when a kernel crash occurs. This helps VMware Global Support Services (GSS) determine the cause of the crash.
VMware vSphere® 6.7 adds a new feature for automatically expanding the coredump partition on the boot device of a vSphere when the configuration device is suitable. When upgrading an older version of vSphere to version 6.7 or higher, the expansion will be performed during boot up and no additional configuration step is required. Installing a new instance of vSphere 6.7 will naturally size the coredump partition appropriately based on the capacity of the boot device.
Device Table Space Consumption
|Boot Device Size > 3.4GB||Boot Device Size < 3.4 GB|
|USB CDROM installation||110MB + 2.5GB towards the partition end if space||110MB, no change if auto-partition option is not used|
|USB / SSD installation (updates)||110MB initially, expanding to 2.5GB||110MB, no additional partition, but uses all available space|
The main difference between USB and any other block device used for the boot up process is the last partition on the device. Partition expansion will take place when the coredump is the last partition on the device. Block storage devices such as M.2, SSD, and magnetic drives will most likely have a VMFS partition at the end of the device for scratch partition (logs) and for coredump file based.
Partition Layout and Device Type
Figure 1. Flash device partition - USB, SD-Card
Figure 2. Block device - magnetic or solid-state drive
In Figure 1, you can see the base structure of a USB device, as an example. If the device capacity is less than 3.4GB, only the base coredump partition, VMKcore, will be applied. Figure 2 shows a boot device with more capacity such as an SSD. This device will have a Scratch partition and a VMFS partition that fill the remainder of the free space on the drive.
Coredump Partition Calculation
In summary, the following factors are important for calculating coredump partition size:
• ESXi host physical memory
• All SSDs placed in ESXi host (vSphere 6.7 release)
See this knowledge base article for exact calculations: KB 2147881
Available Coredump Namespace
There are three options when selecting the destination for coredump information. Let’s take a closer look at these options.
File: Placing coredump to a different vmfs volume
Network: Sending the coredump, over the network to one of the VirtualCenter server
Partition: Set/unset local partition table
File and coredump partition use is mainly automatic now and no other action steps needs be taken besides if you want to use a different partition. See also KB 2004299 or KB 2077516 for the manual procedure.
Visibile ESXi Log Output
Insufficient device size for coredump partition
Output in jumpstart-esxcli-stdout.log
2018-02-26T15:00:50.379Z| VmkCtl: Cannot expand the coredump partition, since the coredump partition is not located at the end of the disk t10.ATA_____INTEL_SSDSC2BA200G4_____________________BTHV72910AFJ200MGN__.
Additional logs: vobd.log, hostd.log
/var/log/vobd.log:2018-02-27T07:54:56.179Z: [GenericCorrelator] 75630750us: [vob.user.coredump.capacity.insufficient] The storage capacity of the coredump targets is insufficient to capture a complete coredump. Recommended coredump capacity is 6494 MiB.
Successful expansion of the coredump partition
2018-02-26T17:14:42.649Z| VmkCtl: Trying to expand coredump partition from 2560 MiB to 5077 MiB .
2018-02-26T17:14:45.623Z| VmkCtl: Disk mpx.vmhba32:C0:T0:L0 contains a large enough coredump partition of size 5323620352, recommended size 5323620352 .
Coredump expansion on removable media such as USB or SD cards, does not require manual configuration in vSphere 6.7 and higher versions. Where possible, available space on the boot device will automatically be configured for coredump usage. File and network based coredump destinations are used and configured same way. Automatic core dump configuration and expansion helps ensure hosts are properly configured to collect diagnostic
About the Author
This content in this document was assembled using content from various resources from vSAN Engineering and vSAN Product Management.
Andreas Scherr is a Sr. Solutions Architect, working in the Storage and Availability Business Unit at VMware, Inc. He specializes in vSAN Proof-of-Concepts, enterprise architectures, data center analytics, software-defined storage, and hyperconverged Infrastructures. Andreas can also be found on twitter at @vsantester .