We can both connect to other people's games, but not each other. We both tried hosting.
In my case joining my own game and then when my friend invited me I left my solo game went to menu and joined him, it worked in that case but you might want to look on the web OUTSIDE of steam discussions. If my solution works that's great just dont look at steam one's cause they are usually connected to some steam issues.
It seemed to be a problem with Discord. I closed that, then he could join, and I reopened it. And it's weird because I didn't have to close it when he rejoined later on.