Just forget about the details... its happening randomnly on several different weapons sporadically. I cannot get it to reproduce consistanly. Are perks being "randmonly" applied because that would make a lot more sense.
It sounds to me like a "rounding error". Perhaps something like shooting a crossbow takes 1.2 points and changing a crossbow for a sword takes 1.9 making 3.1. Whereas changing for a dagger might take only 1.4, totalling 2.6
If only the full numbers are displayed (first case: 3 second case: 2) it would appear "random".
Note: though I have the game, I have yet to play it past run - testing, so obviously the numbers above are just for discussion sake, not actual.