And that can be a real pain. Going from world map directly to inner city, you probably wont have enough time to have several companions to lets talk about equipment and put away the weapon, before the trigger happy cops go all red.
AFAIR they will not notice you until you made at least one step - if you didn't move you still have time to give orders to your companions.
Good to know, but that one time I realised what was happening only when the cop starts shouting.
A few frantic attempts and then it was reload time.
What I discovered on my current playthrough, was my own ear in the inventory. Hazardous sport, boxing is.
Didn't bother reloading an hour old save, as I didn't realize the mishap when it happened.