slamdunk: I was actually planning on a Merchant so this is good news. Thanks!
In that case, you may freely dump the physical stats.
ALTHOUGH now that I think about it, there are a few places where strength/dexterity/constitution can be somewhat helpful - still not combat, and definitely not essential or main quests - just some minor stuff you may not be able to reach (I can think of at least one, maybe two).
Having said that, they're not that important (you'll just miss out on some none-essential items). I'd not worry about them too much - just flag them when you encounter them for the future (possibly a loremaster run - ideally I'd do that after trying out most of the other options first, as things will make more sense then).
This game is definitely intended to be replayed multiple times, as it's literally impossible to see all its content in a single run anyway.