Does the Arms Master skill work in conjunction with the Unarmed skill, or just with the actual weapon skills (Sword, Spear, etc)?

Sooo.... no one knows?

yes it does Monks can reach master level in armsmaster

Only partly. When using his fists only, the Armsmaster skill contributes to attack skill and recovery time, but NOT damage.

Thanks to both of you. I guess my next question then would be: is it worth investing in?

absolutely fully mature monks also wield staffs after grandmastering unarmed, after grandmastering dodge--they can master leather to further raise their AC. Make sure to get and use the hammerhands spell (Body magic) it in effect doubles the damage the monks deal out.

Fully mature monks with all skills maxxed are killing machines dealing out almost as much damage as a fully mature knight dual wielding a sword and spear.