Probably his stance/body language. Several martial artists have reacted to Ryo in similar ways in Shenmue 1 and especially 2."You look like you're from Japan. And your appearance suggests that you're a martial artist."
You mean my T-shirt and jeans? Nothing screams martial artist like tennis shoes.