• State Resolved
  • Locked Locked
  • Replies 11 replies
  • Answers 5 answers
  • Subscribers 33 subscribers
  • Views 1474 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

OPT8241-CDK-EVM: Depth and color mapping in hybrid stereo system

Andris Kuciks
Prodigy 160 points

Part Number: OPT8241-CDK-EVM

Hello.

I'm working on my final bachelor project. I managed to get extrinsinc and intrinsic parameters of the hybrid camera system using Matlab Stereo Calibration app. The next step is really vague. How can i align depth with color infrmation to make a colored point cloud? Most of the sources are using Kinect V2 to make these point clouds and Matlab even has some special functions to get it working. I couldn't find any topic realted to OPT8241 CDK and RGB camera mapping. Right now I'm using RasoberryPI to acquire RGB image in the same resolution as the ToF has - 320x240. Later on I'm planning to use a camera which has a greater resolution and connects to PC via USB. Wanted to find out if anyone has even tried it? Is it even accomplishable? And if it is what are the next steps I should take to get the system working?

Any information will be much appreciated!

Regards, 

Andris

  • 0
    TI__Expert 3495 points

    Andris,

    Pixel-to-pixel mapping between RGB and Depth imager in a RGBD camera can leverage stereo vision technique.  Normally stereo vision uses disparity to compute depth, you can reverse the formulation and use depth to compute disparity.  Then given one camera's image coordinate and disparity, find the other camera's pixel coordinate.  Now, this technique assumes perfect camera calibration such that simple pin-hole camera model can be used.  

    -Larry

  • 0 in reply to Larry Li
    Prodigy 160 points
    Isn't the matching possible by using the two camera intrisic and extrinsic parameters? I saw a lot of threads that used these parameters and in the end they got a pretty nice matched scene, or is this method more complex than I imagine?
  • 0 in reply to Andris Kuciks
    TI__Expert 3495 points
    Yes, you could, if properly applied. Intrinsic and extrinsic parameter fall out of the checkerboard lens calibration. Knowing the intrinsic parameters allows you to correct for distortion, so pin-hole camera model of simple triangular geometry could be used. Extrinsic defines checkerboard corner positions relative to each camera, and together with the co-linear separation distance, can be used to associate checkerboard corners between the two cameras. These associations are data points used to fit a disparity model or LUT. During run-time, you will still need to have depth to find disparity, and depth can comes from the ToF camera; or alternatively, solve the "correspondence point" problem for each pixel, which is a much harder problem.
  • 0 in reply to Larry Li
    Prodigy 160 points
    So I understood correctly, I can get disparity map just by using the obtained depth map and the intrinsic and extrinsic parameters? Could you support some site where this is discussed and imlemented if it is possible?
  • 0 in reply to Larry Li
    Prodigy 160 points
    Some things are still unclear though. In those papers they used the ''measured disparity'' if I understand it correctly it's the disparity of the sent out light against the recieved one. in voxel viewer which data stream would provide this measured disparity? In voxel viewer's user manual disparity isn't even mentioned. And I don't really understand what is meant as alpha and beta intrinsic depth cam parameters if you could please explain. thanks for the help. Andris
  • 0 in reply to Andris Kuciks
    TI__Expert 3495 points
    Andris,

    That paper is not necessarily using TI ToF solution, I only provided that as a reference. What you're requesting is a current research area, so there are multiple approach to this problem. The referenced paper is one of them. What I explained in earlier part of this thread was to provide you a general structure/approach by which a solution can be developed.

    Now, disparity in machine vision is usually referring to the image coordinate differences arise from camera separation. Given a pixel in one camera, finding the corresponding point in the other camera, is called the "corresponding point problem". What extrinsic can give you is (x, y, z) coordinate of each corner relative to the camera, and each corner has a known left-to-right correspondence, so you can compute disparity for each corner. Since you also know 'z', from extrinsic, you can create an interpolated LUT, or fit a math model using the (disparity, z) data points.
  • 0 in reply to Larry Li
    Prodigy 160 points

    Larry,

    I managed to do the transformation from 3D point to image plane and get the corresponding color value of a point using this theory: answers.opencv.org/.../. The problem is, when I transformed the 3D point to the image plane, the alignment in x axis is accurate, but in y axis it is off for about 60 pixels. Maybe you have some ideas why this happens. I'm usign stereoCalibration parameters which I obtained using Matlab stereo calib tool. One problem I can think of was that vector t from the calibration is in mm but the principial point and the focal length is in pixels. Does all of the parameters have to be in the same dimensions?  Or maybe this issue comes from the calibration, because I used amplitude images to do the calibration but afterwards I'm using only points clouds? 

  • 0 in reply to Andris Kuciks
    TI__Expert 3975 points
    Andris,

    Larry is not with TI anymore. Please give us a week to follow up on this.

    Suramya
  • 0 in reply to Suramya Gupta
    Prodigy 160 points
    Suramya,

    Never mind my problem. I managed to fix it. There was a problem with my code when I tried to do the transformation from 3D ToF frame to RGB image frame.

    Regards Andris!
  • 0
    TI__Mastermind 49176 points
    Hi Andris,

    Thanks for the update. I will close this post.