Direct gaze - that is, eye contact with another person - causes the subcortical system to overload, and the parts of the brain that deal with arousal and calming are failing to strike a balance. The end result is that direct eye contact triggers a physiological response which makes it physically uncomfortable to maintain eye contact.