[ Charles stares briefly because no one ever asks him that before he shakes himself out of it. ]

Well, control - of powers or any kind of ability - always hinges on our emotions. So the people here communicate through contact but they have a fine control over their feelings at all times. Which means they never project anything negative unless they've been startled or surprised. Maybe perhaps if they were in danger.

The memories might be random at times, but the emotion that comes with it is not.