Noise and points in the air in PLY files generated from rtabmapviz

[kosmonauta144]

Hi Everyone!

My setup
ROS Noetic
RTABMAP_ROS installed from binaries
Sensor: Azure Kinect
Topics subscribed /rgb_to_depth/image_raw + camera info
/depth/image_raw + camera info
/points2

Everything works fine with mapping, but in exported .ply files I have a "noise" or "points in the air". Points are more and more dense closer to the sensor and it makes them difficult to filter in post-processing so I would like to find a cause of this.

Here is the exported .ply with points in the air which has the same color as obstacle in the background behind them

Here are the yelow points which I suppose are the same "air points" but in rtabmapviz

and with different background color to show the black dots in the air

I don't know if I should use some specific filter in rtabmap.ini file or change the topics. I thought that I will deal with it by scan_cloud but it didn't help.

I will be more than thankfull for every idea how to improve the final pointcloud.

UPDATE:
is RTABMAP generate the pointcloud from /voxel_cloud topic? because voxel_cloud is messed by the colored data from the visual odometry

[kosmonauta144]

alright,

it appeared that in kinect_azure ros launcher (https://github.com/microsoft/Azure_Kinect_ROS_Driver/blob/melodic/launch/slam_rtabmap.launch) which I use, has two nodelets for rectifying rgb and depth image and the rectified depth image was extremely noised. After removing the nodelets, everything works great as expected, so I will close the issue as an issue with launcher from different package

[matlabbe]

For Kinect Azure under ROS, I've made a pull request long time ago: microsoft/Azure_Kinect_ROS_Driver#166
You may check this out.

Updated depth rectification interpolation method to NearestNeighbor instead of Linear. This avoids a lot of noisy points in the 3D point cloud. This would also fix microsoft/Azure_Kinect_ROS_Driver#103 at the same time.

[kosmonauta144]

For Kinect Azure under ROS, I've made a pull request long time ago: microsoft/Azure_Kinect_ROS_Driver#166 You may check this out.

Updated depth rectification interpolation method to NearestNeighbor instead of Linear. This avoids a lot of noisy points in the 3D point cloud. This would also fix microsoft/Azure_Kinect_ROS_Driver#103 at the same time.

Thank You very much for your reply!

I've taken a closer look and your commit works very well. Noise is gone.

Although I have a question, because I observe strange think and I am trying to understand.

In rtabmapviz and rviz also, the /rgb/image_rect is disturbed a little bit in the centre. I noticed it while looking at the edge of the image. I noticed also that if I am not using image_proc/rectify, the /rgb/image_raw is perfectly rectangular. @matlabbe do you know the reason why is the disturbance occur? Also do you know whether it does not affect the map?

[matlabbe]

The rectified image of the color image may cause black edges. The image_raw won't have black edges, but the images are "raw", so with distortions. You should use rectified images. It is why I say in the launch file:

When using color camera, to avoid black borders in point clouds in rtabmapviz, set ROI ratios in Preferences->3D rendering to "0.05 0.05 0.05 0.05" under Map and Odom columns.