NeurIPS 2026 · Evaluations and Datasets Track

EEGFaceSem

An EEG Benchmark with Paired Generative Latents for Semantic Visual Modeling

Jun Ma¹Michiel M. Spapé²Keith M. Davis III³Tuukka Ruotsalo^3,4

¹ University of Helsinki² University of Macau³ LUT University⁴ University of Copenhagen

EEGFaceSem records EEG while people view generated faces and attend to attributes such as smiling or blond hair. Pairing these responses with the generator’s latent vectors connects attention decoding to image editing and generation.

Dataset

EEG recordings from 30 participants across 8 attention tasks, with 32 EEG channels and paired face images and generative latents.

A latent vector is the input used by the generator to produce a face. Keeping it alongside the image and EEG connects the recorded response to an editable image representation.

Participants attend to target facial attributes during rapid serial presentation of generated faces, with images and latent vectors linked to EEG recordings. — Rapid serial visual presentation: faces appear every 500 ms while participants attend to a semantic feature. Tasks span perceived gender, age, hair colour and smiling.

What participants do

Each task specifies one attended attribute: female, male, blond hair, dark hair, smiling, not smiling, old or young. Participants mentally note matching faces while watching the image stream. No button press is required during the recording.

Each task has four blocks of 70 images: 20 targets and 50 non-targets in random order. Across eight tasks, the design presents 2,240 trials per participant.

What an EEG example means

A target label means the displayed face matches the attribute attended to in that task. Each classifier predicts target versus non-target within a task.

The paper uses 32-channel EEG epochs from 200 ms before to 900 ms after image onset. Preprocessing includes a 0.2–35 Hz band-pass filter, baseline correction and automatic blink rejection.

See the task

This short demonstration shows the rapid sequence of generated face stimuli presented during EEG recording.

EEG responses to attended faces

Event-related potentials average the EEG response relative to image onset. At the Pz electrode, target faces produce a larger positive response around 250–600 ms, consistent with a P300 effect. This is a group-average view of the attention response; the benchmark below evaluates individual EEG trials.

Target and non-target event-related potentials at Pz, with scalp maps showing the response over time. — Target and non-target ERPs averaged across participants at Pz. The scalp maps show how the response is distributed over electrodes at successive time points. Figure reused from the paper.

Benchmark

Predict target versus non-target EEG responses for each attention task. The benchmark covers learning within a participant, transfer to a held-out participant, and adaptation with that participant’s data.

Performance is measured with ROC AUC: 0.5 corresponds to chance ranking and 1.0 to perfect ranking. EEGNet has the highest mean AUC among the evaluated models in all three settings, reaching 0.8063 after participant-specific adaptation.

Mean ROC AUC across eight tasks
Model	Within-participant	Cross-subject	Subject-adapted
LDA	0.7206	0.6033	—
Logistic regression	0.7280	0.7397	—
MLP	0.6675	0.7392	0.7546
EEGNet	0.7416	0.7715	0.8063
EEGPT	0.6238	0.7003	0.6824

—: not reported for that setting. Full per-task results and variability are in the paper.

Within-participant

Five-fold evaluation using one participant’s data.

Cross-subject

Leave one participant out for testing; use another participant for validation.

Subject-adapted

Pretrain on other participants, then fine-tune on a subset of the target participant’s data.

RSVP controls and evaluation scope

Whole-block LDA evaluation yields a mean AUC of 0.7225, compared with 0.7206 using shuffled folds. Restricting the epoch to 0–0.48 s and retaining trials with non-target immediate neighbours gives 0.6002 AUC (subject-bootstrap 95% CI: 0.5859–0.6134). These controls cover LDA; they were not rerun for every model. The dataset uses controlled face stimuli and task-specific attention labels.

Applications

The paired latent vectors let models connect a neural response to an editable representation of the image that elicited it.

A joint EEG–latent UMAP embedding beside faces progressively edited along the blond semantic direction. — Left: a self-supervised EEG–latent embedding, coloured by target and non-target labels for visualization. Right: image editing using a supervised LDA–SVR direction. Increasing α moves the generated face along the learned blond-hair direction.

Learn a shared representation

Contrastive learning brings each EEG epoch together with the latent vector of its corresponding stimulus. The left panel visualizes the learned space with UMAP. Binary task labels are used to colour the points after training; the alignment itself uses EEG–latent pairs.

Learn an editing direction

An LDA model learns to distinguish target and non-target EEG responses from one participant. A linear support vector regressor then predicts the LDA scores from image latents. Its weight vector gives a direction that can be applied to new latents to change the generated face.

Conditional generation

The paper also explores an EEG decoder whose predicted task labels condition an adversarial autoencoder in the generator’s latent space. Changing the conditioning label produces paired images for comparison, with explicit-label and randomized-label conditioning as reference settings. The paper includes the architecture, training procedure and qualitative examples.

Scope. These experiments use generated faces and predefined attention tasks. The recorded signals include attention and P300 components. Continuous image presentation can produce overlapping responses, which motivates the RSVP controls reported above.

Citation

@inproceedings{ma2026eegfacesem,
  title = {{EEGFaceSem}: An {EEG} Benchmark with Paired Generative Latents for Semantic Visual Modeling},
  author = {Ma, Jun and Spap{\'e}, Michiel M. and Davis III, Keith M. and Ruotsalo, Tuukka},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2026},
  url = {https://arxiv.org/abs/2611.10781}
}