Creating a steepest descent algorithm to determine a 3D affine warp

I want to implement the 2.5D inverse compositional image alignment. For that I need to create an steepest descent image. I followed the implementation from Code Project for a 2D image alignment. But I am searching for 3D warp information and because of that also for a 3D steepest descent image.

To my project, I have a 3D model interpretation, with raycasting I am creating a rgbd-image. Now I want to search for a 3D warp, which aligns this template image with a given live image to estimate the camera position.

I have currently only the gradients in X and Y direction

cv::Sobel(grayImg_T, Grad_TX, CV_32F, 1, 0, 3);
cv::Sobel(grayImg_T, Grad_TY, CV_32F, 0, 1, 3);

And I am estimating the steepest descent as follows:

       float* p_sd_pixel = &p_sd[cols*j * 3 + i * 3];
       p_sd_pixel[0] = (float) (-cols*Tx + rows*Ty);
       p_sd_pixel[1] = (float) Tx;
       p_sd_pixel[2] = (float) Ty;

       for(int l = 0; l < 3; l++){
           for(int m = 0; m < 3; m++){
               float* p_h = (float*)(;
               p_h[3*l+m] += p_sd_pixel[l]*p_sd_pixel[m];

Both is from the 2D inverse compositional image alignment code, I have from the website of the link I posted before. I think I need also a gradient in Z direction. But I have no idea how to create the steepest descent image for 2.5D alignment and also how to determine the affine warp. How can I tackle the math or find a better way to implement this?