Lets make some random data.
X = rand(1000,2)*rand(2,3) + rand(1,3);
I have no idea what plane represents the data, but I do know there is some plane that perfectly contains all the data points, since it was constructed that way.
XC = mean(X,1);
Y=X-XC;
[~,~,V]=svd(Y,0);
V
V =
0.44144 0.67346 -0.59295
0.71352 -0.66416 -0.22313
0.54408 0.32458 0.77371
s = svd(Y,0)
s =
15.628
4.2751
2.1204e-14
So it is indeed a planar set.
A good definition of a plane uses the normal vector. Any point Z in the plane has the property that
dot(Z - XC,V(:,3)) == 0
Here V(:,3) corresponds to the vector with a zero singular value. It is the normal vector to the plane. As you can see, it kills off anything in-plane.
norm(Y*V(:,3))
ans =
2.1259e-14
However, if I pick any random point, it will generally not produce zero, unless I get very lucky.
(randn(1,3) - XC)*V(:,3)
ans =
-1.6699
So what are the other two vectors in V? As you say, they form a spanning basis for the planar subspace, although they could have been as easily been chosen differently. In a sense, the choice here is somewhat arbitrarily chosen so that V(:,1) is a vector that represents most of the mass in your data. But V(:,1:2) could be rotated arbitrarily here, as long as you care only about what they tell you about the plane. We can write any point in the plane using the generic parametric form
P(a1,a2) = XC + a1 * V(:,1).' + a2*V(:,2).'
If your data does not fall exactly on a perfect plane, then the third singular value from svd will be larger than zero. In the case I tried, it is not exactly zero, but 2e-14 is as close as we can reasonably get.