Arkose: First of all, grab the
Combined Community Codec Pack. This provides full support for all common formats. Windows Media Player will automatically see these codecs, as will most other players, although depending on the player you may need to confirm a prompt about an unknown file type (since the player might not normally support such extensions).
Windows Media Player can handle everything with CCCP installed, but if you'd still prefer a third-party player I'd recommend
Media Player Classic Home Cinema.
I'm not sure about video files, but WiMP tends to fail at Matroska audio even with the plug ins installed. I think it has to do with the lack of support for cue sheets.
That may or may not be an issue for the OP, but it does significantly reduce the utility of using Matroska.