Data for "Image-based Backbone Reconstruction for Non-Slender Soft Robots"

This dataset provides the data for the forthcoming paper "Image-based Backbone Reconstruction for Non-Slender Soft Robots". The backbone reconstruction method used is based on the method described in Hoffmann et al. [1]. The modifications to this method to support the non-slender soft robot in this dataset are described in the forthcoming paper mentioned above. This dataset holds raw images of pressurized and elongated soft robots and the corresponding reconstructed backbones.

Dataset

The dataset is split into two subsets with similar structure. The first subset is contained in dataset_01. The second dataset is contained in dataset_02.

Each subset consists of five folders and one schedule file. The schedule file schedule.csv contains the index of the schedule entry, the angle $\alpha$ in degree, the pressure of each chamber $p_1$ to $p_3$ in bar and if the pressurization is active. Furthermore, the five folders of the subset can be described as follows

raw: Contains the raw cropped images. The filenames are formatted as CROPPED_C{CAMERA_INDEX}_E{SCHEDULE_ENTRY}.png with the camera index CAMERA_INDEX and the schedule entry SCHEDULE_ENTRY.
constant_curvature_slender, constant_curvature_volumetric, cubic_curvature_slender and cubic_curvature_volumetric. These folders contain the actual reconstructed backbones based on the raw data from the raw folder. A different reconstruction approach was used in each of these folders
- constant_curvature_slender - A constant curvature backbone kinematic based on the slender model,
- constant_curvature_volumetric - A constant curvature backbone kinematic based on the volumetric model,
- cubic_curvature_slender - A cubic curvature backbone kinematic based on the slender model,
- cubic_curvature_volumetric - A cubic curvature backbone kinematic based on the volumetric model.
Each of these folders contain a data and figures folder. The data folder consists of PARAMETER_E{SCHEDULE_ENTRY}.json files listing the optimization parameters for each schedule entry SCHEDULE_ENTRY in the JSON format. The figures folder contains annotated images of the reconstructed backbone on the cropped raw images. The filenames are structured ANNOTATED_E{SCHEDULE_ENTRY}_C{CAMERA_INDEX}_EPOCH{EPOCH}.png with the schedule entry SCHEDULE_ENTRY, the camera index CAMERA_INDEX and the epoch EPOCH of the optimization algorithm.

The optimization parameters include the base position base_position of the reconstructed backbone in world coordinates, the coefficients for the curvature polynomials ux and uy, and the constant coefficient for the elongation polynomial la.

Calibration Data

The calibration data is located in the calibration folder and consists of multiple .npy files in the numpy format. The corresponding camera index for the calibrated camera is abbreviated with CAMERA_INDEX in the following:

C{CAMERA_INDEX}.npy - Stores the reprojection error, camera matrix, distortion coefficients, rotation, and translation vectors as returned by the cv2.calibrateCamera [2] method.
C{CAMERA_INDEX}_camera_matrix.npy - Stores the camera_matrix as returned by the cv2.calibrateCamera [2] method.
C{CAMERA_INDEX}_distortion_coefficients.npy - Stores the distortion coefficients as returned by the cv2.calibrateCamera [2] method.
C{CAMERA_INDEX}_projection_matrix.npy - Stores the projection matrix from world space to pixel space based on the stereo camera calibration.
STEREO.npy - Stores the reprojection error, R, T, E, F as returned by the cv2.stereoCalibrate [2] method as an object datatype.

Acknowledgement

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 501861263 – SPP2353

References

[1] M. K. Hoffmann, J. Mühlenhoff, Z. Ding, T. Sattel and K. Flaßkamp. An iterative closest point algorithm for marker-free 3D shape registration of continuum robots. arXiv. https://arxiv.org/abs/2405.15336

[2] OpenCV. Camera Calibration and 3D Reconstruction. OpenCV Documentation. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html, accessed May 27, 2024.

Data for "Image-based Backbone Reconstruction for Non-Slender Soft Robots"

Dataset

The dataset is split into two subsets with similar structure. The first subset is contained in dataset_01. The second dataset is contained in dataset_02.

Each subset consists of five folders and one schedule file. The schedule file schedule.csv contains the index of the schedule entry, the angle

\alpha

in degree, the pressure of each chamber

p_1

p_3

in bar and if the pressurization is active. Furthermore, the five folders of the subset can be described as follows

raw: Contains the raw cropped images. The filenames are formatted as CROPPED_C{CAMERA_INDEX}_E{SCHEDULE_ENTRY}.png with the camera index CAMERA_INDEX and the schedule entry SCHEDULE_ENTRY.

constant_curvature_slender, constant_curvature_volumetric, cubic_curvature_slender and cubic_curvature_volumetric. These folders contain the actual reconstructed backbones based on the raw data from the raw folder. A different reconstruction approach was used in each of these folders

constant_curvature_slender - A constant curvature backbone kinematic based on the slender model,
constant_curvature_volumetric - A constant curvature backbone kinematic based on the volumetric model,
cubic_curvature_slender - A cubic curvature backbone kinematic based on the slender model,
cubic_curvature_volumetric - A cubic curvature backbone kinematic based on the volumetric model.

Each of these folders contain a data and figures folder. The data folder consists of PARAMETER_E{SCHEDULE_ENTRY}.json files listing the optimization parameters for each schedule entry SCHEDULE_ENTRY in the JSON format. The figures folder contains annotated images of the reconstructed backbone on the cropped raw images. The filenames are structured ANNOTATED_E{SCHEDULE_ENTRY}_C{CAMERA_INDEX}_EPOCH{EPOCH}.png with the schedule entry SCHEDULE_ENTRY, the camera index CAMERA_INDEX and the epoch EPOCH of the optimization algorithm.

Calibration Data

C{CAMERA_INDEX}.npy - Stores the reprojection error, camera matrix, distortion coefficients, rotation, and translation vectors as returned by the cv2.calibrateCamera [2] method.

C{CAMERA_INDEX}_camera_matrix.npy - Stores the camera_matrix as returned by the cv2.calibrateCamera [2] method.

C{CAMERA_INDEX}_distortion_coefficients.npy - Stores the distortion coefficients as returned by the cv2.calibrateCamera [2] method.

C{CAMERA_INDEX}_projection_matrix.npy - Stores the projection matrix from world space to pixel space based on the stereo camera calibration.

STEREO.npy - Stores the reprojection error, R, T, E, F as returned by the cv2.stereoCalibrate [2] method as an object datatype.

References

[2] OpenCV. Camera Calibration and 3D Reconstruction. OpenCV Documentation. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html, accessed May 27, 2024.