I highly doubt Mario Golf requires more development muscle than Fire Emblem: Three Housee, for example, which even required outside help from Koei Tecmo and the series never had stellar sales but didn't prevented Nintendo to bet on the series. In Camelot's case, the studio barely has more than 100+ employees, I don't get where your point about Mario Golf requiring so much effort is coming from.
Any way, what "critical reception flop" and "fan reception flop" are you talking about? The latest FE, Kirby, Yoshi and even Metroid games had very good reception and there's any indicative Nintendo will pull off those franchises.
It doesn't nullify the demand for the genre in the system. It's like saying ND should never make another Uncharted game because TLOU is a better seller which is a strawmanned conclusion.
Anyway, Mario Kart and Mario Party are stronger spin offs than the Sports games, it's not like there'll be a shortage of Mario titles. Well, if we're going deeper into this territory, then Nintendo should rebaptize all their mid tier franchises to the Mario brand.
You're digging your head too deeply into your market darwinism cognitive dissonance.
Outside of Pokémon, neither of those franchises are that big stellar sales either, they don't have that much market share into a 80+ million userbase and it's very unlikely they would alienate interest from new games of the genre on the system. Anyway, high demand for a genre isn't a good counter point, either, as we can see the FPS and open world genres. Very few can have the sales of GTA and COD but this isn't stopping their respective genre to have a huge offering and alternatives, you're just being assumptious with your cognite dissonance.
It's amazing how market darwinism is the crux of your points. I'm glad you aren't in charge of Nintendo or any other gaming publisher, otherwise the market would become a oligopoly of the biggest IPs.