Nvidia is reportedly seeking SSDs from its partners with a performance of up to 100 million IOPS, which is more than 30 times the current level of solutions. This was revealed by Silicon Motion's CEO, Wallace Kou, in a recent statement. Such a significant increase is necessary to address one of the main bottlenecks in AI systems: the slow storage not being able to keep up with accelerators like the H200 and B200, which have memory bandwidth of up to 8 TB/s.
Currently, PCIe 5.0 SSDs only provide 2-3 million IOPS, and even experimental drives like Kioxia's XL-Flash are capped at 10 million IOPS. According to Kou, achieving 100 million IOPS based on NAND memory without a significant increase in cost and power consumption is almost impossible. Possible solutions include either a completely new type of memory or the use of multiple parallel storage devices.
Rumors suggest that the first specialized 'AI SSDs' may appear in 2025, coinciding with the launch of Nvidia's new platform, Vera Rubin.