The following shared source code is accompanied with our CVPR paper Joint Depth and Alpha Matte Optimization via Fusion of Stereo and Time-of-Flight Sensor. Basically, it's an implementation of our algorithm on GPU using CUDA. So, it is can only compile and run on machines with NVidia graphics card with CUDA support. It includes implementation of 2 algorithms: the belief propagation algorithm and closed-form matting algorithm.

The source code is developed under MS VC 8.0 of MS VS 2005. It should compile under MSVS 2005 or higher edition. In order to compile the code, additional libraries are needed before that. Those libraries include: NVidia CUDA Toolkit, NVidia CUDA Developer SDK, OpenCV, Image Debugger(Image Debugger is already included in the package, no need to download them). Please make sure you have these libraries properly installed and included in the project.

Although the performance reported in our paper is tested under single precision floating point arithmetic, the code we shared is all under double precision floating point. Double precision gives much better accurate result, but with the loss of speed. It is reported that the performance of double precision arithmetic is only 8% of that of single precision arithmetic on GPU. If you want to make it run faster, you should change all the double type variables to float. Also note that only recently released graphics cards support double precision computation. Check your GPU to see if your graphics card support double precision.When compiling for devices without native double precision support, the double type gets demoted to float by default and the double precision math functions are mapped to their single precision equivalents.

In our paper, the dpeth map is generated by the combination of the stereo cameras and the TOF depth sensor. Since different users have different depth sensors which, in turn, have different output formats, and some users may not even have a depth sensor to do the depth fusion. To make our code more generally useful, we do not include the fusion code. The input to our system is just 2 stereo images, and the left and right depth maps are computed based on only these 2 images using stereo algorithm. However, we have to say that the depth map from the fusion is much better than that from stereo only, and better depth maps lead to better matting results. If you have a certain type of depth sensor, you just need to incorporate it into the code by adding another data term to the BP energy function.

You can download the the package here.

If you have any problem, feel free to contact miao liao at miao(dot)liao(at)uky(dot)edu.