Returns the squared distance between p and the segment a-b.
Vector3
a
b
p
The line is not considered infinite.