Azrapse: Sorry to contradict, but I consider XvT both the hardest and the lowest point of the series, gameplaywise.
All missions were designed to be played either cooperatively multiplayer (campaigns) or versus (battles), so the difficulty is expecting several human players to take care of the many things to do during missions, or the AI is unfairly abusing missiles, countermeasures, etc.
Also, the depth of the story in XvT is next to zero, and the "power creep" is to the top. It's not unusual to find TIE Interceptors with missiles and shields, or to encounter several waves of TIE Advanced in most missions, while in X-Wing, TIE Fighter and Alliance, these superfighters were reserved for particular dramatic moments or the hardest missions.
Both TIE Fighter and Alliance have a much better plot, much easier learning curve, and much more variety of missions and stories.
Myself, I would go first with X-Wing if you can see past the outdated graphics. The difficulty is uneven (some missions are really easy, some are really hard, but not harder than the ones in XvT). But at least the story is better than in XvT. And it is the first one, with the simplest controls.
Exactly my thoughts on this. Start with X-Wing.