In this day and age, think most of us can download games absurdly quickly and that will just get better with time. Hell, I download and delete games off Game Pass all the time, it's no big deal.
Of course a next gen system that costs $300 will cut corners, its $300! I would expect that the second revision of Series S, when SSD prices decrease, will either increase the SSD to 1 TB or reduce the system cost to $200 or $250.
You can store 3 or 4 huge games at once with 364 GB, and if you play indies and smaller titles, you can likely fit over 10 games on that SSD at once. I have over 30 games installed on my Xbox One X right now and its only the 900 GB of space that the console has available.
You can also cold store the games on an external, cheap hard drive. You can get 4 TB hard drives for like $80.