The song is about when Damien had a girlfriend in Paris, and he was at college (university), at a party. Two hot girls came onto him, and he was this close to having a threesome with them when he remembered about this girlfriend of his.
So, everything makes sense when in that context.
'This is not what I do' - i shouldn't be behaving like this, it's not me
'It's a small crime' - it's a small crime in the grand scale of all crimes, but it is a crime in an emotional sense
'Give my gun away when it's loaded' - give, essentially, my penis to somebody when it's already been loaded - i.e taken, by somebody else
'If you don't shoot it, how am I supposed to hold it?' - You're all the way in Paris, my penis is yours, but you can't make me cum because you're not with me. So how am I supposed to hold it and not let somebody else shoot it instead?
Yay