It's not like 1-9 were opposed to throwing things out and trying something different. 2 and 8 in particular could definitely fall into the "wild experimentation" category. 7 and 8 also have much more in common with the stories of some of the games that come after than they do with the games before.
"Ignoring MMOs" also kind of undermines your point a little bit considering those are far and away the biggest Final Fantasies ever from both a content and a business point of view.
2 and 8 really are not what I would call "wild experimentation" compared to the later games. They were different, sure, but only in the same way that 3, 6, or 7 were. They tried new ideas, but still used old ones. 2 took the basic battle system and abilities from 1, but mixed up how you learned and strengthened them. FF8 tries some new things with how magic works, but the GF and Junction system are fundamentally the same idea as Jobs, Magicite, or Materia, just repackaged and iterated. The only real difference is the addition of a few extra steps required to covert the abilities of the GF into direct stat changes.
Compare those to something like FF12, which fundamentally changes core elements of the combat system, or FF15 that is basically a different genre entirely. Especially since FF15 doesn't really clearly draw any gameplay ideas from 12 or 13 in the same way that even 8 clearly borrows ideas from 6 and 7.
And 7 and 8 don't really share much in common with the post-10 stories. They are, fundamentally, still focused on the traditional heroic journey without subversion. They lack the later focus on divine powers that guide and manipulate the world behind the scenes, and their stories don't feature pretty much any of the major elements I mentioned.
And I omitted the MMOs for several reasons. First, they are fundamentally developed by different branches of the company with very different structures and conditions. The fact that they are fundamentally a different genre makes it hard to directly compare their gameplay, and the different storytelling style brought on by the expansion-based format makes it impossible to directly compare their stories. It is just apples to oranges.
You're condensing about 8 years of company history down into "the same time for the same reasons."
The major shift in FF happened as far back as FF6 when Kitase took over writing and directing duties from Sakaguchi. Younger creatives like Nomura were moving up and getting bigger roles. New blood like Nojima were brought in around that time too. Sakaguchi was less involved in FF6 and FF7 than he had been in FF1-5, and then barely involved in FF8 at all while he took a more active role in FF9 instead.
The Square Enix merger happened two years after FF10 was released. FF10 would have been almost finished development by the time Squaresoft and Enix even began discussing a merger. Sakaguchi didn't leave the company until 2004 and he was involved in two more Final Fantasy games after FF10 as well as Kingdom Hearts and he oversaw FF11's first two expansions.
Wada was never a replacement for Sakaguchi. They worked entirely different jobs in the company. Sakaguchi was a game director and producer. Wada was a corporate executive. He was made CEO four years before Sakaguchi left.
I mostly agree, but FF12 is a pretty direct follow-up to the SNES/PS1 era of games. Matsuno's storytelling and Sakimoto's music gave it a bit of a different feel, but in terms of setting and gameplay, it was basically FF6 in 3D. It felt closer to the series history than FF10 did. And I think FF16 is going to be a throwback in a lot of ways too. FF14 by the same team certainly is. FF14 is the most Sakaguchi-ass FF game since FF5.
Anyway, I think the main takeaway is that the stewardship of Final Fantasy under CBU1 (and formerly BD1-3) is a complete mess. Kitase clearly lost the plot decades ago and hopefully Yoshi-P interrupts him every time he tries to speak at board meetings.
It is a simplification, yes, but you have to simplify a little. But I'll both disagree with you and clarify a few points.
First, the shift from FF1-5 to FF6 is indeed a shift in the direction of the series, but it isn't the only one, and it is much less significant than the one I was talking about. FF6-10 is its own arc that features very different worlds and stories than before, focusing less on chosen heroes and more on human conflict, where magic is its own major force in the world rather than just a presumed thing. But it still fits in the larger paradigm of gameplay iteration and story variety.
And yes, the big transition in the company took place after FF10, but I included 10 in the list of later games because most later FF games just... copy FF10. FF10 itself is a transition game. It is the product of the same gameplay iteration that produced the earlier games, but the core recurring things seen in later FF games are all found in 10. The things that distinguish 10 from earlier games are the same things that form the template for the later games. Thus, evil manipulative religious figures, a subverted heroic journey, rebellion against the core ideals of the world, sphere-grid like progression mechanics, and major overhauls to core battle mechanics become the norm.
Either the people in charge of FF right now really love FF10, or it is the only kind of FF they know how to make, but it is pretty much the definitive game for the series over the last few decades. Well, it and secondary influence from Kingdom Hearts, but that discussion is more complicated.
As for your comments on 12, 14, and 16...
FF12 is a terrible trashfire of a game. It isn't a throwback to the older games, because even with the Zodiac Job System it is barely any more mechanically complicated than FF1. Its core gimmick is the completely overhauled core battle system that prioritizes flash and animations over fun fights. It abandons the well-designed progression systems based on opportunity cost from older games in exchange for a shallow retread of the sphere grid where everyone gets every skill all the time. Its main story is a mess in which the main characters have no agency and barely register in the plot. And it is fundamentally built to be the anchor for a bunch of spinoff games like FFTA and Revenant Wings. Nothing about it has the slightest resemblance to the SNES/PS1 games.
FF14 is, as I said, hard to compare directly because it is an MMO. But I do think it that part of the reason for A Realm Reborn's rather poor story (other than the development time crunch) was its focus on big lore and worldbuilding, while completely overlooking the basics of character writing and putting together an engaging main story. This is one of the same big problems that drags down 12, 13, and 15. Luckily, the game is really good past that part.
As for 16, it is still too early to really be sure what it will be like, but that first trailer strongly indicated that it is doing the same thing as the last few titles. Rather than iterate on the good parts of either FF12 or 7R's battle systems like I hoped, it again is doing its own thing. And again we have a world at the mercy of big divine things, and s hero who is likely going to suffer tragedy on his path to fighting big divine things. It doesn't have the Type-0/15 director in charge, so the story will probably be better than those, but other than that it looks like the same thing all over again.