While I'm not an active game developer, I am an extremely senior technologist with a ton of programming experience at a low level and more than a passing familiarity with the practices relevant to the subject. So I'll give it a shot, but keep in mind that I haven't gone through the production grind on a game so others may have a more nuanced view. That said, this is a pretty general question.
Cerny's talk regarding the SSD the other day actually addressed most of this pretty directly. Developers wanted an SSD worse than just about anything else because they've been working around both limited bandwidth and seek times for multiple generations, and both have had a huge impact on the design of games. Seek times and fragmentation are inter-related because without seek times fragmentation becomes effectively a non-issue. (1) That's why Cerny was able to promise that game patching wouldn't involve the copying step we see today, and suggested that asset duplication wouldn't be as necessary.
Just because SSDs are fast, though, doesn't mean their access time is negligible. You really can't treat it like RAM and just load what you need when you need it, or you will still wind up spending an unacceptable amount of time waiting. What you can do, though, is start streaming a particular piece of content
much closer to the time you need it (even just a couple of frames ahead of time), and discard from memory anything you won't need for the next few frames. Practical implications include:
- Less speculation about what assets you'll need. The chances of guessing right when you have to start loading a second or two ahead of time aren't great, so you can wind up loading things you never use. That means that in practice the 100x increase in bandwidth gets further boosted by being right more often about what you need and wasting less bandwidth.
- Less memory wasted on speculative assets. Another side effect of increasing the certainty that you've loaded just what you need is that it occupies less memory than if you loaded a ton of assets just in case but never wound up using them at all.
- Loading assets on demand. If some rare event occurs you can probably get away with loading audio or visual assets in response to the event, which would be impractical if you have to start loading them much earlier. This could lead to more situation specific commentary lines in sports titles, death animations in a wide range of titles, or far more variations on a musical theme that kick in just when they're appropriate.
Does that help clarify things?
(1) Not strictly true as flash memory is still accessed in blocks and sequential access remains faster than random access. Writes of a partial block are especially slow because they involve reading what's there currently, and then writing the whole block back. Random access reading of sub-block granularity isn't nearly as fast as reading entire blocks, and flash memory controllers can generally still provide better throughput for reading sequential blocks rather than pure random access. The overhead is orders of magnitude lower than physical seek times, however, so it's still a huge win no matter how you look at it.