Sometimes the mission briefings encourages it but I cant figure it out. It always seems I have to have 4 guys.

On the agent selection screen (Where you buy weapons etc), right click on the big agent number. The number should go dim.
The agent is now NOT assigned to the mission.

Is there any actual benefit to it, rather than just flavour?

Is there any reason not to go as a full 4-pack with group control? Same as controlling one, except four times the firepower.

Im over half way done and have seen no reason NOT to use a full party.

However, as a side note, Ive heard some people have actually played through the entire game with a single agent. Which could be a fun challenge.

maybe if you seek an additional challenge? play with 4 agents the first time. second playthrough stick with 3. then maybe 2. ultimately, try with only 1 guy.

One well upgraded and equiped agent can be more effective sometimes than 4 poorly equiped agents. It allows you to progress in the early game without risking ill prepared yet valuable agents.

Good point.

I've often played with just one or two agents for this very reason. I prefer to use one or two fully upgraded agent (more durable) to four poorly upgraded agents. Off course some missions lend themselves better for this playing style than others. Making good use of 'panic mode' and of the persuadertron also helps. I've managed to beat a lot of levels using a mob of persuaded people and just one agent. One agent can also do a ton of damage with a couple of lasers, miniguns and an upgraded brain.
For those who haven't playes the game: if you activate panic mode (press both mouse buttons at once) your agent will move faster and will automatically fire at threats. The higher the level of the brain is that your agent has, the longer you can use panic mode and the longer the range at which he will automatically fire.

Yeah, I played all the time with 4 agents. Then I started only using one or two.

If you fully upgrade them, they are pretty powerful. Besides you have better control of one or two of them.

Early on, you might want to focus your cash on upgrading 1-2 agents and playing the easy missions with them. This works pretty well.

Later on, you might want to try using less agents as an added challenge.

Also, one of Syndicate's few big flaws is lack of grouping options.
You can't split your 4 agents into groups of 2. You either control everyone, or one agent.
Using only 2 agents in a mission allows you to play this way without worrying about two extra agents getting lost in the (not very good) pathfinding system.