I think this is a big problem with discussions about any kind of libertarianism. Most people only have a surface idea of what a libertarian is, thinking that loudmouth internet idiots who claim to be libertarian, while push Republican talking points and occasionally saying "But both parties suck", or agreeing with a pet peeve of theirs that the Libertarian Party also pushes for - drug decriminalization, legalizing prostitution, whatever - are the whole of the party, or of the philosophy.
But there's a lot more to it than what typically comes to mind for most people. Like how many self-described libertarians support universal basic income replacing the welfare system. Or how anti-war most libertarians are, and how critical they've been of the wars and bombings in the Middle East for the past 2 decades, which is the opposite of the mainstream Republican Party no matter what Trump might try to claim.
I think it's fair to say that all (real) libertarians start from the basic starting point of wanting less federal power, and more individual liberties, but the specifics start to get muddier from there depending on who you're discussing things with. Like, most want less restrictions at the borders, but not all of them want to abolish borders altogether.
Again, the video The Albatross posted really shows the trouble with our political system. If you're not center-right, you don't exist in our system. Really a great watch.