NVMe/TCP vs iSCSI

Performance Deep Dive

The performance gap between NVMe/TCP and iSCSI is not a marketing claim — it is a structural consequence of protocol design. iSCSI was built on top of SCSI, a command interface designed for spinning disks in the early 1990s. Every I/O operation on an iSCSI connection must be translated through a SCSI emulation layer: the NVMe command is converted to a SCSI command, encapsulated in iSCSI PDUs, sent over TCP, received, unwrapped, and re-translated back to an NVMe command before reaching the drive. This round-trip through the emulation layer adds 60–80 µs of consistent overhead — overhead that compounds under mixed workloads and high queue depths.

NVMe/TCP eliminates this translation entirely. NVMe commands travel natively from initiator to target with no SCSI intermediary, reducing round-trip latency to 25–40 µs on a local Ethernet segment. But the deeper advantage is parallelism. iSCSI supports a single command queue per LUN with a maximum depth of 128 commands. Modern NVMe drives support up to 65,535 queues with 65,535 commands each — and NVMe/TCP exposes that full queue architecture over the network. Under a database workload generating thousands of concurrent I/Os, iSCSI serializes what NVMe/TCP can truly parallelize, which is why real-world IOPS measurements show a 3–4× advantage for NVMe/TCP even on identical hardware and the same Ethernet infrastructure.

It is worth noting that both protocols run over standard Ethernet and require no specialized hardware beyond a capable NIC. The performance delta comes entirely from protocol efficiency, not from network upgrades. Organizations already running 25 GbE or 100 GbE for iSCSI can deploy NVMe/TCP on the same switches, the same cabling, and the same NICs — while immediately reclaiming the latency and throughput that SCSI emulation has been silently consuming.

Use Case Matrix

Migration Path: iSCSI to NVMe/TCP

Because both protocols run on standard Ethernet, organizations do not need to choose between an immediate cutover and staying on iSCSI indefinitely. A phased migration is practical and widely used. In Phase 1, NVMe/TCP targets and iSCSI targets coexist on the same network — new workloads provision from NVMe/TCP while existing iSCSI volumes remain in service. Most Linux kernels since 5.0 and Windows Server 2022 include NVMe/TCP initiator support, so no new software purchases are required.

Phase 2 involves live migration of non-critical workloads: snapshot the iSCSI volume, restore to an NVMe/TCP-backed LUN, validate performance, and re-point the application. For databases, this typically means a maintenance window of minutes rather than hours. Phase 3 retires the iSCSI targets as the last workloads complete migration. Throughout the process, the network fabric is untouched — only the storage software stack changes. Teams with strong iSCSI operational experience find the operational model familiar enough that the learning curve is minimal.

Conclusion

NVMe/TCP and iSCSI share the same hardware foundation, which makes the performance delta all the more striking: eliminating the SCSI emulation layer unlocks latency and parallelism that iSCSI cannot reach regardless of how much you tune it. For organizations building new storage infrastructure today, the calculus strongly favors NVMe/TCP. For those with stable, functioning iSCSI environments, the migration is practical but should be driven by genuine performance need rather than novelty. For teams deploying Kubernetes workloads, simplyblock.io provides production-ready NVMe/TCP block storage with a Kubernetes-native CSI driver — delivering iSCSI-compatible migration paths alongside native NVMe performance.