Still on my first game so haven't used much diplomacy yet, and it's quite cumbersome to test some things yourself since the game is so slow...

If I want to attack another nation, but don't want to do it alone, can I do like this?
-Try to get a diplomat leader.
-Make an alliance with someone I have good relationship with.
-Try and anger the one I want to start a war with - make demands, run missions, make peace with their enemies etc, and provoking them to start a war on me, which will then make my ally fight with me against them (since I was attacked and didn't start the war)?

That possible, or not how the relationships/diplomacy/alliances work?

Cool, thanks, I'll probably give that a try sometime in the future (no diplomats in line yet though) :D

You don't need an alliance, you can just Ask to Declare War. This mission is unlocked by the tech Polis, costs 200 civics and 2 orders, and then costs either 160 + 40 * (number of cities you have) training, of double that number if you choose money.
You need to be friendly with the one you ask, and that player can't be pleased or higher with the target. But both of that can be achieved. For all settings where your nation is small and other nations start developed, this can be a huge factor in deciding the game.
It's probably to most powerful move the human player has against big AIs.

Ah ok, it might be easier to just ask a friendly nation to go to war with me, than to get an ally and then try to provoke a war...sounds reasonable, but good to have options I guess :D

Was thinking there might be a penalty from other nations for starting a war, but I don't know if that's a thing in this game?

No such penalties in the game as of writing this, no.