Basic space carving algorithm
I have the following problem as shown in the figure. I have point cloud and a mesh generated by a tetrahedral algorithm. How would I carve the mesh using the that algorithm ? Are landmarks are the point cloud ?
Pseudo code of the algorithm:
for every 3D feature point
convert it 2D projected coordinates
for every 2D feature point
cast a ray toward the polygons of the mesh
get intersection point
if zintersection < z of 3D feature point
for ( every triangle vertices )
cull that triangle.
1 answer

I see it like this:
So you got image from camera with known matrix and FOV and focal length.
From that you know where exactly the focal point is and where the image is proected onto the camera chip (Z_near plane). So any vertex, its corresponding pixel and focal point lies on the same line.
So for each view cas ray from focal point to each visible vertex of the pointcloud. and test if any face of the mesh hits before hitting face containing target vertex. If yes remove it as it would block the visibility.
Landmark in this context is just feature point corresponding to vertex from pointcloud. It can be anything detectable (change of intensity, color, pattern whatever) usually SIFT/SURF is used for this. You should have them located already as that is the input for pointcloud generation. If not you can peek pixel corresponding to each vertex and test for background color.
Not sure how you want to do this without the input images. For that you need to decide which vertex is visible from which side/view. May be it is doable form nearby vertexes somehow (like using vertex density points or corespondence to planar face...) or the algo is changed somehow for finding unused vertexes inside mesh.
To cast a ray do this:
ray_pos=tm_eye*vec4(imgx/aspect,imgy,0.0,1.0); ray_dir=ray_postm_eye*vec4(0.0,0.0,focal_length,1.0);
where
tm_eye
is camera direct transform matrix,imgx,imgy
is the 2D pixel position in image normalized to<1,+1>
where(0,0)
is the middle of image. Thefocal_length
determines the FOV of camera and aspect ratio is ratio of image resolutionimage_ys/image_xs
Ray triangle intersection equation can be found here:
If I extract it:
vec3 v0,v1,v2; // input triangle vertexes vec3 e1,e2,n,p,q,r; float t,u,v,det,idet; //compute ray triangle intersection e1=v1v0; e2=v2v0; // Calculate planes normal vector p=cross(ray[i0].dir,e2); det=dot(e1,p); // Ray is parallel to plane if (abs(det)<1e8) no intersection; idet=1.0/det; r=ray[i0].posv0; u=dot(r,p)*idet; if ((u<0.0)(u>1.0)) no intersection; q=cross(r,e1); v=dot(ray[i0].dir,q)*idet; if ((v<0.0)(u+v>1.0)) no intersection; t=dot(e2,q)*idet; if ((t>_zero)&&((t<=tt)) // tt is distance to target vertex { // intersection }