Let me start by saying that this isn't entirely GOG's fault. They try to do their best with getting games to work. A lot of the noise in the forum and reviews here are a mixed bag - some people aren't having very many issues, while many people are having a lot of issues. Here is my experience though.
Street Fighter Alpha 2 is a damned good fighting game. However, the PC port sucks. It should only be bought as a pity present, along with maybe a flash drive with the arcade rom on it as an apology for the crappy game.
First of all, like many have said before me, you are going to want a controller that is compatable with the DInput API. This review was written up after playing a bit with the RetroBit Sega Saturn USB controller set into DInput mode. (Hold down Start for 5 seconds). Though, for easier access, ensure you have a Logitech F310. It's a solid Playstation style controller with a decent D-Pad and typical XInput buttons, but on the back, there's a switch to toggle the input mode.
My major gripe is that the graphics are the worst. I'd rather look at the DOS port of Street Fighter II, and that port is even worse than this. There is boatloads of screentearing and screenflickering in full-screen and the flickering persists when in windowed mode. Additionally, in windowed mode, there is a lot of color corruption that forces you to play it in full-screen.
Though, with a 720p TV connected up to the back of my gaming computer, this game looks terrific... if the screen tearing and flickering weren't an issue.
Whoever made this port should be tied up by Capcom with the remainder of Hi Tech Expressions for their 'ports' of the Megaman games for DOS and whoever programmed the US Gold DOS port of Street Fighter II.