3D reconstruction (without semantics) using Intel Realsense D435i is not working

[hieronymusma]

Hi everybody,

Unfortunately I have some problems building the 3D reconstruction using the Intel Realsense D435i.

Whenever I start Kimera-Semantics I can see the pose estimation in rviz but no reconstruction. In the Terminal I only get
[ INFO] [1613430674.227458481]: Updating mesh.

I am executing the following commands:

1. Start roscore
   roscore

2. Start camera node
   roslaunch realsense2_camera rs_camera.launch unite_imu_method:=linear_interpolation enable_infra:=true enable_infra1:=true enable_infra2:=true enable_gyro:=true enable_accel:=true align_depth:=true

3. Disable emitter
   rosrun dynamic_reconfigure dynparam set /camera/stereo_module emitter_enabled 0

4. Start Kimera-VIO
   roslaunch kimera_vio_ros kimera_vio_ros_realsense_IR.launch run_stereo_dense:=true

5. Start Kimera-Semantics
   roslaunch kimera_semantics_ros kimera_metric_realsense.launch

I've changed following parameters in the launch file of kimera semantics:

should_use_sim_time:=false
run_stereo_dense:=true # I've tested it with both false and true but neither works
metric_semantic_reconstruction:=false

sensor_frame:=left_cam_base_link

left_cam_info_topic:=/camera/aligned_depth_to_color/camera_info
left_cam_topic:=/camera/color/image_raw
left_cam_segmentation_topic:=/camera/color/semantic_image
left_cam_depth_topic:=/camera/aligned_depth_to_color/image_raw

Here is a visualization of my ROS topics:

The visualization of Kimera-VIO works fine. However I've encountered the issue, that after some time the visualization of Kimera-VIO stops when I start Kimera-Semantics but the status messages of Kimera-VIO are still printed.

I also receive sometimes the following message after startup of Kimera-VIO:
W0215 23:15:30.628022 5205 DataProviderModule.cpp:118] Asking for data before start of IMU stream, from timestamp: 1613430930557135336 to timestamp: 1613430930626134204. After removing and plugging in the camera again it mostly works.

I run the complete system inside a Docker container, could that be the problem?
Ubuntu 18.04
ROS Melodic 1.14.10
Realsense Firmware: 05.12.11.00

I'm unsure, if I haven't set a topic properly or forgot something else.

I would really appreciate it if someone could guide me into the right direction. I'm relatively new to ROS, therefore I don't know how to analyze the problem correctly.

Thank you very much,
Maurice

[mbed92]

Same here. The error "Asking for data before start of IMU stream" happens always when the align_depth:=true flag is set.

[moksh-401-511]

I also faced same issue.
I stored the data in a bag file and run the bag file at rate=0.1 . This worked for me.
I was able to generate unlabelled mesh. But failed to get labelled/semantics mesh

[CCCC-user]

hello,I have the same problem. Have you solved it?

[prmehta24]

@moksh-401-511 I am trying to do 3D reconstruction(no semantics) with an intel realsense d435i bagfile.

I am doing the following steps:

1. roscore.
2. rviz -d $(rospack find kimera_semantics_ros)/rviz/kimera_realsense_metric.rviz
3. roslaunch kimera_semantics_ros kimera_metric_realsense.launch play_bag:=true (I replaced the kimera_semantics_demo.bag with my bag. metric_semantic_reconstruction is default set to false. should_use_sim_time is set to true.)

However, I can not see the mesh reconstruction and have an error global status error - fixed frame - unknown frame world

Do you have an idea of how to solve this issue and view the 3D reconstruction?

[moksh-401-511]

@moksh-401-511 I am trying to do 3D reconstruction(no semantics) with an intel realsense d435i bagfile.

I am doing the following steps:

1. roscore.
2. rviz -d $(rospack find kimera_semantics_ros)/rviz/kimera_realsense_metric.rviz
3. roslaunch kimera_semantics_ros kimera_metric_realsense.launch play_bag:=true (I replaced the kimera_semantics_demo.bag with my bag. metric_semantic_reconstruction is default set to false. should_use_sim_time is set to true.)

However, I can not see the mesh reconstruction and have an error global status error - fixed frame - unknown frame world

Do you have an idea of how to solve this issue and view the 3D reconstruction?

Can you do rosbag info of your bag file and share the screenshot?

[prmehta24]

@moksh-401-511 I am trying to do 3D reconstruction(no semantics) with an intel realsense d435i bagfile.
I am doing the following steps:

1. roscore.
2. rviz -d $(rospack find kimera_semantics_ros)/rviz/kimera_realsense_metric.rviz
3. roslaunch kimera_semantics_ros kimera_metric_realsense.launch play_bag:=true (I replaced the kimera_semantics_demo.bag with my bag. metric_semantic_reconstruction is default set to false. should_use_sim_time is set to true.)

However, I can not see the mesh reconstruction and have an error global status error - fixed frame - unknown frame world
Do you have an idea of how to solve this issue and view the 3D reconstruction?

Can you do rosbag info of your bag file and share the screenshot?

Sure, here it is.

[moksh-401-511]

@prmehta24
As you know, to generate mesh, Kimera-semantics needs to know where your camera is w.r.t. world frame. So, you need to run kimera-vio-ros as mentioned here to track your camera position in a world frame. It will require the IMU data of d435i.

If you're using bag file then you might need to launch this file. Setup the sensor-frame and other necessary parameters. Then give it a try.