Hmm, seems like I need to jump back in then and keep going.
Yeah, I'd recommend giving it another go and approaching the combat a little differently. Even in the beginning, battles should be incredibly snappy; it's only if you aren't interacting with the Break/Boost systems correctly that they become drawn out. The combat is about setting up Breaks on bosses for when you have your damage-dealer(s) primed to do full damage — boss debuffed, damage-dealer(s) buffed, full boost available, boom. Unlike most JRPGs, buffs and debuffs are crucial.
Any given Chapter 1 boss
could take half an hour if the player is wasting Break windows on basic attacks, but if the player starts setting maximizing their boosted damage output during breaks, bosses die in only a minute or two. Likewise, random encounters start melting away when utilizing boosted AoE (Apothecary's Last Stand and Scholar's spells are particularly potent). That's how exponential a difference there is between optimal damage output and sub-optimal.
If battles were too frequent, I'd highly recommend using the Scholar job's first passive, which reduces the encounter rate by 50%, making it so you can go entire screens without battles.