I've noticed that if I start the game with an XBox controller plugged in, it freezes right after the "Press Any Button" screen. If I run it without the controller, it works fine, up to the point when I plug in the controller.

I've been playing with an XBox One Controller with a USB Port and everything has been running fine on my end. Only had the game crash once when I was exiting to desktop. (Running Win 10 and fully updated graphic drivers)

Have you tried disabling/removing additional USB devices just to see if any other USB items are conflicting with the game?

Yup.

I also tried different 360 controllers and my Logitech Precision, same issue.

Apparently also happens with my Switch Pro Controller (BT).

Fixed it! I had to turn on Windows 8 compatability for the main executable, that made it work fine.