How do you drop something when you get over encumbered?

Hold shift and click on it.

There are two ways I know of. First press Tab or F2 to open inventory, then:
1) shift-click on the inventory item to drop it in front of you
2) click and hold on the inventory item to manouvre it in front of you (you can also do this with world objects by holding the Z key while focused on it)