vSphere 9.1 Performance Best Practices: What Changed, What Matters, and What to Do First
VMware just released the updated Performance Best Practices guide for vSphere 9.1 — 94 pages of tuning guidance covering CPU, memory, storage, and networking. This article breaks down the most impactful changes from 9.0 to 9.1, curates the recommendations that matter most, and tells you where to start.
The Guide You Should Actually Read
VMware publishes a lot of documentation. Most of it is reference material — you consult it when you have a specific question. The Performance Best Practices for VMware vSphere guide is different. It is one of the few VMware documents that is worth reading cover to cover before deploying anything, because the decisions it influences — BIOS settings, NUMA configuration, memory sizing, storage adapter choices, power management policies — are the ones you make early and live with for years.
The vSphere 9.1 edition↗ was released on July 13, 2026. It covers virtual hardware version 22, expanded processor-specific guidance for Intel Granite Rapids and AMD EPYC, new storage capabilities, and significant updates to memory tiering, power management, and lifecycle operations.
This article breaks down what changed from 9.0 to 9.1, highlights the recommendations that deliver the most impact, and gives you a prioritized starting point.
What Is New in vSphere 9.1
Intel QAT for Encrypted vMotion
This is the single most operationally impactful addition. Intel QuickAssist Technology (QAT), available on recent Intel processors, offloads cryptographic operations from general-purpose CPU cores to dedicated hardware accelerators. vSphere 9.1 leverages QAT to significantly reduce the CPU cost of encrypted vMotion.
Why this matters: encrypted vMotion is no longer optional in most enterprise environments — compliance mandates it. But encryption adds CPU overhead to every migration, which means either slower migrations or fewer concurrent migrations. QAT eliminates that trade-off. If you are purchasing new hardware, choosing processors with QAT support is one of the highest-ROI decisions you can make.
You can verify QAT support on an existing host with: lspci | grep -i qat
Intel Granite Rapids C-State Guidance
The guide adds specific C-state recommendations for Intel Granite Rapids processors. C-states control how deeply a processor core sleeps when idle — deeper sleep saves more power but takes longer to wake up, introducing latency on the first instruction after idle.
Granite Rapids processors have a different C-state architecture than previous generations, and the optimal settings depend on your workload profile. The guide provides a decision table mapping workload types to recommended C-state configurations. For latency-sensitive workloads (real-time analytics, financial trading, AI inference), disabling deeper C-states is still the right call. For general enterprise workloads, the Balanced power policy — which is the default — handles C-states intelligently and rarely needs manual tuning.
Memory Tiering Improvements
Memory tiering — using NVMe storage as a secondary memory tier behind DRAM — was introduced in vSphere 9.0. In 9.1, it gets two significant operational improvements. First, setup no longer requires manual ESX CLI commands and scripts; configuration is now done at the cluster level through vSphere Configuration Profiles. Second, hosts only need to enter maintenance mode for changes to take effect — a reboot is no longer required. The release also adds a new software mirroring feature for redundancy.
The performance guidance remains consistent: keep active memory at no more than 50% of DRAM capacity (though some workloads tolerate up to 75%), keep NVMe read latency below 200 microseconds, and ensure enough CPU headroom for the tiering overhead. For hardware selection, choose NVMe devices capable of at least 100,000 writes per second with 3+ DWPD endurance.
For a deeper dive into memory tiering architecture and sizing, see: NVMe Memory Tiering in VCF 9.1.
vSAN ESA as the Recommended Architecture
The guide now explicitly recommends vSAN Express Storage Architecture (ESA) over Original Storage Architecture (OSA) for new deployments. vSAN OSA hybrid is deprecated and will be discontinued in a future release. The performance guidance for vSAN has been fully rewritten around ESA, with OSA guidance moved to the prior version's document.
Key ESA performance points: use PVSCSI adapters (default queue depth of 64 vs. 32 for legacy adapters), consider increasing PVSCSI queue depth for OLTP workloads, use up to four virtual SCSI adapters per VM for multi-VMDK configurations, and consider Virtual NVMe devices in ESA clusters for reduced controller overhead. vSAN over RDMA (RoCE v2) improves network CPU efficiency but requires lossless network configuration — don't enable it unless your network is properly set up for it.
Lifecycle Manager Live Patch
Live Patch allows security patches and urgent bug fixes to be applied with minimal disruption to running workloads — no host reboot, no VM migration. Combined with Quick Boot (which reboots ESX without firmware initialization) for patches that do require restart, the lifecycle management story in 9.1 is materially faster and less disruptive than any prior version.
VCF Single Sign-On Performance
The SSO guidance has been broadened with specific recommendations for modern identity provider federation. VCF 9.1 supports Okta, Microsoft Entra ID, and on-premises PingFederate via SCIM 2.0 provisioning. The guide emphasizes optimizing the network path to your OIDC identity provider and minimizing automated login frequency — audit your scripts for forgotten or over-frequent login patterns that load the SSO server unnecessarily.
Virtual Hardware Version 22
vSphere 9.1 introduces virtual hardware version 22 with additional capabilities. The trade-off: VMs on hardware version 22 can only run on ESX 9.0 or later hosts, which constrains vMotion in mixed-version clusters. Plan your upgrade path before upgrading VM hardware versions.
The Recommendations That Matter Most
The guide contains hundreds of recommendations. Here are the ones that deliver the most impact across the widest range of environments.
CPU: Do Not Over-Provision vCPUs
This is the most common performance mistake in virtualized environments, and the guide addresses it repeatedly. Configuring a VM with more vCPUs than its workload can use does not make it faster — it makes it slower. Extra vCPUs consume timer interrupts, increase memory coherency overhead, cause guest OS scheduler migration between cores (losing cache locality), and in older operating systems can trigger busy-wait idle loops that waste CPU cycles.
The guide recommends leaving some CPU resources for ESX itself rather than fully committing all cores to VMs. Monitor with esxtop: if the load average is at or above 1.0, the system is overloaded. CPU usage at 80% is a reasonable ceiling; 90% is a warning.
For NUMA-aware workloads, size VMs to fit within a single NUMA node when possible. VMs that span NUMA nodes ("wide VMs") experience higher average memory access latencies. If you must create wide VMs, vNUMA (enabled by default for VMs with more than 8 vCPUs) helps the guest OS manage memory locality.
Memory: Understand the Overcommit Hierarchy
ESX uses five memory management mechanisms in a specific order: page sharing, ballooning, memory compression, swap to host cache, and regular host-level swapping. The first four allow significant memory overcommitment with little or no performance impact. The fifth — regular swapping — can dramatically degrade performance.
The practical rule: overcommit memory freely, but monitor for regular host-level swapping. In the vSphere Client, check the Swapped and Compressed values under Monitor > Utilization > Guest Memory. Non-zero swapped values indicate the host has exhausted all gentler reclamation methods.
For large pages: ESX assigns 2MB pages to VMs whenever possible, significantly reducing TLB misses. But large pages are not shared between VMs, so page sharing only kicks in when memory pressure forces large pages to be broken into small pages. This is by design and is the right trade-off for most workloads.
Storage: Choose PVSCSI and Consider vNVMe
The default virtual storage adapter for most guest operating systems is LSI Logic. The Paravirtual SCSI (PVSCSI) adapter offers significantly lower CPU utilization and potentially higher throughput. For I/O-intensive workloads, switching to PVSCSI is one of the simplest performance improvements available.
The Virtual NVMe (vNVMe) adapter, introduced in vSphere 6.5, provides even lower latency for flash-based storage. Compared to virtual SATA, vNVMe accesses local PCIe SSD devices with much lower CPU cost per I/O and significantly higher IOPS. For vSAN ESA clusters, vNVMe is worth evaluating.
Use eager-zeroed thick disks for workloads that need the best first-write performance. VAAI-capable storage can offload the zeroing operation to the array, making eager-zeroed thick disk creation fast.
Networking: DirectPath I/O and SR-IOV for Maximum Throughput
For workloads that need bare-metal network performance, DirectPath I/O (using VT-d or AMD-Vi) allows VMs to directly access physical network adapters, bypassing the hypervisor's virtual switch entirely. This is the same technology that VCF Private AI Services uses for GPU passthrough to inference VMs — and it applies equally to network-intensive workloads. For more on DirectPath I/O in the AI context, see: VCF 9.1 Private AI Services.
SR-IOV (Single Root I/O Virtualization) offers a middle ground: it allows a single physical NIC to present multiple virtual functions, each assigned directly to a VM, providing near-native performance while supporting more VMs per physical adapter than DirectPath I/O.
For latency-sensitive network workloads, the guide recommends disabling interrupt coalescing, using the High Performance power policy, disabling deeper C-states in BIOS, and reserving memory and CPU for the VM to prevent ballooning and CPU ready time.
Power Management: Balanced Is Usually Right
The guide's most counterintuitive recommendation: the Balanced power policy (the default) often outperforms High Performance. On systems where some cores are idle part of the time, Balanced allows active cores to enter higher Turbo Boost states than are available under High Performance — because power budget freed by idling some cores is redirected to the active ones.
High Performance is still the right choice for fully loaded systems where all cores run at or near 100% utilization. But for typical enterprise workloads with variable load, Balanced delivers both better power efficiency and, sometimes, better performance. Intel HWP (Hardware P-States), supported on fourth-generation and later Xeon Scalable Processors, further improves this by letting the CPU manage its own P-states based on ESX hints.
BIOS Settings: Get These Right Before You Deploy
BIOS settings are the foundation everything else builds on. Get them wrong and no amount of software tuning will compensate. The guide specifies:
For Intel processors: Enable VT-x, EPT, VT-d, AES-NI, and QAT. For HWP, ensure BIOS does not disable it and allows frequency control by the OS. Configure C-states based on your workload profile using the Granite Rapids decision table.
For AMD processors: Enable AMD-V, RVI, AMD-Vi. Set NPS to NPS-1 and disable CCX-as-NUMA (the defaults) — these provide the best performance for nearly all workloads. Only consider NPS-4 for workloads that are simultaneously high-bandwidth, low-latency, highly NUMA-optimized, and can be pinned to specific nodes.
Power management: Set BIOS power management to "OS Controlled" to allow ESX to manage P-states and C-states. If the BIOS controls power management, ESX cannot implement its Balanced policy, and you lose the Turbo Boost benefits.
Where to Start
If you are deploying new vSphere 9.1 infrastructure, here is the priority order:
First, get BIOS settings right — power management to OS Controlled, all virtualization extensions enabled, QAT enabled. These cannot be changed without a reboot and often require a maintenance window.
Second, set the ESX power policy to Balanced (the default). Only switch to High Performance if you have confirmed through monitoring that all cores are consistently saturated.
Third, right-size your VMs. Audit vCPU allocations against actual usage. This single action — reducing over-provisioned vCPUs — often delivers more performance improvement than any other tuning.
Fourth, switch I/O-intensive VMs to PVSCSI adapters. This is a VM-level change that can be done during a maintenance window.
Fifth, if using memory tiering, configure it through vSphere Configuration Profiles at the cluster level and ensure your NVMe devices meet the 100K IOPS / 3 DWPD / sub-200μs latency thresholds.
Sixth, read the full guide. This article covers the highlights, but the complete document↗ has depth that a summary cannot replicate — particularly around vMotion tuning, DRS configuration, Storage vMotion, Fault Tolerance, and Content Library optimization.
References:
Discussion
No comments yet. Be the first to start the discussion.