Yes, but the most important thing will be to see the difference between a 2.4 GB / s vs 5.5 GB / s SSD in game.
Because on PC an 860 Evo SSD (550Mb / s) and a 970 EVO Nvme (3.5GB / s) the difference is very minimal while the NVME is 6x faster theoretically.
I'm sure the biggest difference will be on installing a game on the disc, downloads to PS5 will be faster.
With cross-gen games, I can absolutely see this being the case, since these games are designed with HDD's in mind, hence not nearly as taxing in terms of SSD bandwidth.
Where things may get trickier is predicting how the difference will fair in next-gen only titles that have NVMe drives (not Sata SSD) as the baseline, and are designed around facilitating ultra fast SSD's the way this Unreal 5 demo appears to be.
In other words, they're going to be relying heavily on constant streaming of super high quality assets and subsequently ones with much larger data sizes.
Star Citizen is the only PC game I know that is designed with Sata SSD's (not NVMe) in mind as the baseline, and that basically doesn't run properly on normal HDD's, where you get awful frame tanks, stuttering, freezing etc. In other words, it's data streaming pipeline is designed with Sata SSD as the baseline, but not NVMe.
Now imagine games (much like this demo) designed around NVMe drives as the baseline instead of Sata drives, and designed to scale in a way that delivers more asset quality or density, the faster your drive or memory or whatever, can stream it in.
How much data could devs feed it? And could the speed of your drive (along with GPU/CPU performance and ram amount) in someway dictate how many assets, or how much geometry, density etc was possible on the said platform? These are the questions we don't yet have answers to.
We also don't know how the Series X drive (or PC SSD's for that matter) might affect multiplatform devs making better use of the PS5's SSD, as they still have to cater to the lowest common denominator, so perhaps it's an advantage we only see in first party titles, who knows.
But what we do know is that video game devs have stated the faster the SSD they can get, the better, the more video ram the better too (eg, diminishing the need for SSD altogether), the more GPU performance the better and so on (all within mass market budgeting of course). Some devs are always going to find ways to maximise whatever ram, SSD bandwidth, GPU performance etc they have available, and I'm sure next gen these SSD speeds, ram amounts etc will all seem paltry or slow in comparison.