At the rates we're talking about? Not really.
A theoretical example. Let's say a game that's running at a constant 60FPS needs to grab 1GB of assets to put into RAM for immediate use. Let's also absolutely no compression takes place, so it's the worst case scenario.
On the Series X, this process takes 25 frames, or 0.42 seconds. On PS5, the same scenario takes a touch under 11 frames, or 0.18 seconds. A difference of 14 frames, or 0.24 seconds
On PS4 and Xbox One, the above scenario takes 1229 frames, or 20.48 seconds.
Which really gets to the heart of the scenario. Yes, the transition to SSDs is a major change in how resources are gathered and will have a material effect to game design. However, the advantage of PS5's SSD will probably wind up being academic at best.