How to fit a manifold?

This is a quick introduction of manifold fitting. A PDF version is also available. More details can be found in:

Method

Let z be the point of interest, which is close to M, and z=argminzMd(z,z) be the projection of z on M. Intuitively, the estimation of manifold can be viewed as ‘‘pushing’’ z to z.

Push

This pushing process involves two key components: direction and distance. The direction should be perpendicular to TzM, which can be deduced from the local ‘‘covariance’’ structure, while the distance d(z,M) might be estimated using the local average. The following subsections will introduce some intuitive concepts related to this process. For more details, please refer to the papers mentioned previously.

Estimate Direction from Local ‘‘Covariance’’

For each point yiYN, let yi be its projection on M. Assume the normal space of M at yi has orthonormal basis {u1,,uDd}, we use

Πyi=(u1,,uDd)(u1,,uDd)=k=1Ddukuk

to represent the projection matrix onto this space. This projection matrix can be estimated from the local variation centered at yi, and the estimator of Πyi is denoted as Π^yi.

Local Covariance

Let BD(yi,r) be the D-dimensional Euclidean ball centered at yi with radius r. If r is ‘‘large’’ enough, such that yiyiξic0r, the area of BD(yi,r)M roughly has radius c1r, and the variation of M along the normal direction is less than c2r2 due to the reach. Then, since the distribution of X is smooth, the variation of Yyi along direction:

  • : is roughly in the order of (c1r)2+σ2;
  • : is roughly bounded above by the order of (c2r2)2+σ2.

Thus, we can define

Σ^r,i=j=1N(yjyi)(yjyi)I(yjyir)j=1nI(yjyir).

Then perform SVD on Σ^r,i to obtain {λ1<<λD} and {v1,,vD}, and estimate Πyi with Π^yi=(v1,,vDd)(v1,,vDd)=k=1Ddvkvk, whose estimation error can be bounded.

Smoothing System

To make the overall estimation smooth enough, the weight function for yi with respect to z is defined as

α~i(z)=(1zyi2r2)βI(zyir),αi(z)=α~i(z)i=1nα~i(z),

where β2 is a parameter corresponding to the smoothness. Then, for z, a smooth reference point can be given by μ^z=i=1Nαi(z)yi, and a smooth projection matrix is calculated as

Ψz=PDd(i=1Nαi(z)Π^yi),

where Pk(A) stands for the projection of matrix A onto the span space corresponding to its largest k eigenvalues.

The Manifold Estimator

The vector from z to z can be estimated with the bias vector

b^(z)=i=1nαi(z)Ψz(zyi)=Ψz(zμ^z),

which can be shown

  • b^(z) close to zz;
  • Jacobian matrix of b^(z) is close to Φz, i.e. Jb(z)ΨzCσ/r+op(1);
  • Hessian matrix of b^(z) is lower bounded.
Bias Vector

Finally, the manifold estimator is given by

M^={zRD:d(z,M)<cr,b^(z)=0}.

Under all the error bounds and all the smoothness, for any zM^, with high probability,

  • z is close to M;
  • in its neighborhood, b^(z) is rank Dd.

Hence, with high probability, M^ is a d-dimension manifold, close to M, and its reach can be bounded via the Hessian of b^(z).




Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • What is manifold fitting?