A robot dataset of successful and failed placement executions

The dataset contains the following data from successful and failed executions of the Toyota HSR robot placing a book on a shelf.

• RGB images from the robot's head camera
• Depth images from the robot's head camera
• Rendered images of the robot's 3D model from the point of view of the robot's head camera
• Force-torque readings from a wrist-mounted force-torque sensor
• Joint efforts, velocities and positions
• extrinsic and intrinsic camera calibration parameters
• frame-level anomaly annotations

The anomalies that occur during execution include:

• the manipulated book falling down
• books on the shelf being disturbed significantly
• camera occlusions
• robot being disturbed by an external collision

The dataset is split into a train, validation and test set with the following number of trials:

• Train: 48 successful trials
• Validation: 6 successful trials
• Test: 60 anomalous trials and 7 successful trials