EEG Speech-Robot Interaction Dataset

EEG data recorded during spoken and imagined speech interaction with a simulated robot

EEGCreative Commons Attribution 4.0 InternationalIntroduced 2024-06-03

Dataset Description

This dataset consists of Electroencephalography (EEG) data recorded from 15 healthy subjects using a 64-channel EEG headset during spoken and imagined speech interaction with a simulated robot.

Citation

The dataset recording and study setup are described in detail in the following publications:

  1. Rekrut, M., Selim, A. M., & Krüger, A. (2022, October). Improving Silent Speech BCI Training Procedures Through Transfer from Overt to Silent Speech. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2650–2656. IEEE. https://doi.org/10.1109/SMC53654.2022.9945447
  2. Selim, A. M., Rekrut, M., Barz, M., & Sonntag, D. (2024, June). Speech Imagery BCI Training Using Game with a Purpose. In Proceedings of the 2024 International Conference on Advanced Visual Interfaces (AVI ’24). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/3656650.3656654

If you use this dataset, please consider citing these works.

Study Design

Participants controlled a simulated robot on a screen in a game-like setup through a maze while EEG data was recorded. Below are the key details:

Experimental Setup

  • Interaction Modes:

    • Overt Speech: Speaking command words aloud.
    • Imagined Speech: Silently imagining the command words without any articulatory muscle movement.

  • Command Words:

    • Movement: “Left”, “Right”, “Up”
    • Actions: “Pick”, “Push”

  • Interaction Process:

    1. Participants pressed the spacebar to indicate readiness.
    2. The screen turned black for 2 seconds to prepare input.
    3. A fixation cross appeared, signalling participants to speak or imagine the desired command.
    4. The fixation cross disappeared, and the robot’s position updated accordingly.
    5. The robot always performed the correct action, ensuring participants were stress-free.

  • Session Organisation:

    • Four parts: Overt Speech, Imagined Speech, Overt Speech, Imagined Speech.
    • 80 repetitions per word and paradigm (400 imagined, 400 spoken per participant).
    • Task divided into levels of 25 interactions to reduce cognitive load.
    • Breaks between sessions allowed for rest and EEG headset impedance checks.

Participants

  • Subjects: 15 healthy individuals (11 male, 4 female), average age 26.8 years.
  • Attributes: Right-handed, fluent in English (non-native).
  • Ethics: Informed consent obtained; approved by the ethical review board of the Faculty of Mathematics and Computer Science, Saarland University.

Recording Conditions

  • Environment: Dimly lit room with minimal distractions.
  • EEG Equipment: Wireless 64-channel Brain Products Live Amp 64 system.
    • Sampling rate: 500 Hz.
    • Electrode placement: 10-20 International System.
  • Synchronisation: The robot game and EEG recording software were executed on the same computer.

Data Format

  • File Format: .fif format, containing a continuous raw data stream with event markers.
  • Event Dictionary: 
    {
    "Event_Dictionary": {
        "Empty": 1, "EndOfEvent": 2, "EndOfLevel": 3, "EndOfParadigm": 4,
        "space_bar": 5, 
        "Overt_Up": 11, "Overt_Left": 12, "Overt_Right": 13, 
        "Overt_Pick": 14, "Overt_Push": 15,
        "Silent_Up": 21, "Silent_Left": 22, "Silent_Right": 23, 
        "Silent_Pick": 24, "Silent_Push": 25
    }
}