Yakuza 4 is actually pretty good place to start since it introduces a bunch of new main characters and is the most refined version of the game.
Every game is stand alone but a lot of characters return and you won’t be having those “oh it’s that guy!” moments.
Yakuza 3 is like $10 at gamestop new and 4 is $20 so I say get both but if only one then maybe, yeah, start with 4.