Clear Filters
Clear Filters

Question Regarding 'pcfromdepth' Function Example

49 views (last 30 days)
Hello,
I have been working with the example code provided in the 'doc pcfromdepth' documentation in MATLAB. I used the example color image 'sampleImage.png' along with depth images extracted using the 'MiDaS' and 'Depthanything v2' models to generate the point cloud. However, the point cloud I obtained was inaccurate compared to the result shown in the example.
I am curious about how the DepthImage used in the MATLAB example was extracted, and how the camera parameters (focalLength, principalPoint) were determined. Could you please provide more information on this?
Thank you for your assistance.
  2 Comments
Steven Lord
Steven Lord on 2 Oct 2024
"inaccurate" is a very general term. If you provide more information about what you receive (so we could compare it with the results shown on that documentation page) Answers readers may be better able to offer suggestions for ways to improve the agreement of your modification of the example with what's in the doc.
Seungju
Seungju on 4 Oct 2024
I am attaching the results of my three experiments.
The attachments can be found at the following Google Drive link.
First Experiment - Using SampleDepth Image from the 'doc pcfromdepth' Page
The code used in the first experiment is as follows:
depthImage = imread("sampleDepth_matlab.png");
imshow(depthImage)
colorImage = imread("sampleImage.png");
imshow("sampleImage.png")
focalLength = [535.4, 539.2];
principalPoint = [320.1, 247.6];
imageSize = size(depthImage,[1,2]);
intrinsics = cameraIntrinsics(focalLength,principalPoint,imageSize);
depthScaleFactor = 5e3;
maxCameraDepth = 5;
ptCloud = pcfromdepth(depthImage,depthScaleFactor, intrinsics, ...
ColorImage=colorImage, ...
DepthRange=[0 maxCameraDepth]);
pcshow(ptCloud, VerticalAxis="Y", VerticalAxisDir="Up", ViewPlane="YX");
I used the 'sampleDepth_matlab.png' and 'sampleImage.png' images as provided in the documentation. The resulting point cloud, which matched the documentation, is attached as '1st_matlabdoc_result.fig'.
Second Experiment - Using DepthImage Extracted by MiDaSV3.1 Model
In the second experiment, I extracted the DepthImage from 'sampleImage.png' using the MiDaSV3.1 model, which I have attached as 'sampleDepth_midasv3.png'. The code used was identical to that of the first experiment, except for the image filename. The result using this DepthImage is attached as '2nd_midasv3_result.fig'.
Third Experiment - Using DepthImage Extracted by Depthanything v2
In the third experiment, I extracted the DepthImage from 'sampleImage.png' using the Depthanything v2 model from the following link: https://huggingface.co/spaces/depth-anything/Depth-Anything-V2. The extracted DepthImage, attached as 'sampleDepth_depthanythingv2.png', was originally in color, so I converted it to grayscale before using it in the pcfromdepth function. The code used is as follows:
depthcolorImage = imread("sampleDepth_depthanythingv2.png");
depthImage = im2gray(depthcolorImage);
imshow(depthImage)
colorImage = imread("sampleImage.png");
imshow("sampleImage.png")
focalLength = [535.4, 539.2];
principalPoint = [320.1, 247.6];
imageSize = size(depthImage,[1,2]);
intrinsics = cameraIntrinsics(focalLength,principalPoint,imageSize);
depthScaleFactor = 5e3;
maxCameraDepth = 5;
ptCloud = pcfromdepth(depthImage,depthScaleFactor, intrinsics, ...
ColorImage=colorImage, ...
DepthRange=[0 maxCameraDepth]);
pcshow(ptCloud, VerticalAxis="Y", VerticalAxisDir="Up", ViewPlane="YX");
The resulting point cloud from this experiment is attached as '3rd_depthanythingv2_result.fig'.
Summary
Except for the first experiment (using the DepthImage provided in the MATLAB documentation), I was unable to achieve the same results with the other depth estimation models. Therefore, I am curious about the process used to extract the DepthImage provided in the MATLAB documentation. Additionally, I would like to know how the camera parameters were determined.
Thank you for your help.

Sign in to comment.

Accepted Answer

Gayathri
Gayathri on 7 Oct 2024 at 10:17
I understand that you want to know the source of the Depth Image and the intrinsic camera parameters used in the “pcfromdepth” documentation example.
The image and the depth image were captured using a RGB-D camera. This is mentioned in the “pcfromdepth” documentation link given below.
Also to know more about the dataset from which the image was taken, please refer to the example given in “Visual SLAM with RGB-D Camera” documentation. The link for the same is attached below for your reference.
The image is taken from the TUM RGB-D benchmark dataset. The dataset can be accessed using the below mentioned link.
The intrinsics of the dataset such as “focalLength”, “principalPoint” can be found in the below mentioned link.
Hope you find this information helpful.
  1 Comment
Seungju
Seungju on 14 Oct 2024 at 7:50
Moved: Walter Roberson on 14 Oct 2024 at 16:17
I truly appreciate your quick and helpful response to my question.
Your explanation has greatly clarified the issue I was facing, and it will be very beneficial as I move forward.
Thank you for taking the time to assist me.

Sign in to comment.

More Answers (0)

Categories

Find more on MATLAB Support Package for USB Webcams in Help Center and File Exchange

Products


Release

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!