It's because people automatically assumed he would kick their ass, cuz he's a tall-ish man. If it had been Kierra Knightly, they would have swarmed and blamed her for being there to BE swarmed. :/