bevinator: IMO the XP and Fame increases are marginal enough to be utterly forgettable. The real gem is the potion efficiency, which can make a huge difference later when you're digging through the Shadow Vault. Since the monsters essentially scale infinitely but the potions are fixed, that extra bonus can really help, especially for a Vanquisher that's more fragile than the other classes.

So I usually put a few points into it after I've (nearly) maxed out stuff like Armor Expertise, Block/Parry, and Critical Strike. Perhaps you should consider some of the trap spells, which can be very useful. Shock is great against non-resistant enemies, and Flechette+Devouring work really really well together. Alternatively, you can try some of the melee skills, though I never found anything other than Hamstring to be very useful. There are still some things in the Rogue tree that are good for everyone, though, like Wind of Justice. Don't forget the magic skills either. Many of the spells improve enormously even with just 3-4 points in their appropriate mastery.