Why wont my hamster let me pick him up?

I have had my hamster at least 4 weeks he's fine with me stroking him whilst he's in the cage. But if I try and pick him up he. Jumps and runs. Only way to get him out is in a loo roll tube he goes on my hand then jumps for it.