A Subject-Independent Comparison of EEGNet and CSP+LDA on EEGMMIDB: Implications for Assistive Educational Technology

Nur  Aziezah; Fajar Nur Hamzah; Walidatush Sholihah

doi:10.58797/cser.040104

Authors

Nur Aziezah Software Engineering Technology Study Program, Vocational School, IPB University, Jl. Kumbang, No. 14 Bogor 16128, Indonesia
Fajar Nur Hamzah Faculty of Science, University of Sydney, Eastern Ave, Camperdown NSW 2050, Australia
Walidatush Sholihah Faculty of Science and Engineering, University of Groningen, Groningen, The Netherlands

DOI:

https://doi.org/10.58797/cser.040104

Keywords:

CSP+LDA, EEGMMIDB, EEGNet, leaveone-subject-out (LOSO), motor imagery EEG

Abstract

Motor imagery (MI) electroencephalography (EEG) classification remains difficult in cross-subject settings because signals vary substantially between individuals. This study compares a compact deep learning model (EEGNet) with a classical Common Spatial Pattern and Linear Discriminant Analysis pipeline (CSP+LDA) for hands-versus-feet MI classification. Data came from the EEG Motor Movement/Imagery Dataset (EEGMMIDB) on PhysioNet. A leave-one-subject-out (LOSO) evaluation was conducted across 109 subjects using runs 6, 10, and 14. Preprocessing applied a 7-30 Hz band-pass filter, a 1-2 s post-cue window, and subject-wise z-score normalization. Because normalization used unlabeled statistics from each held-out subject, the setting is LOSO with unsupervised test-time normalization rather than a fully inductive calibration-free protocol. EEGNet reached a mean balanced accuracy of 0.660 ± 0.136 (macro-F1 0.640 ± 0.147; Cohen’s kappa 0.312 ± 0.271), whereas CSP+LDA reached 0.635 ± 0.141 (macro-F1 0.605 ± 0.164; kappa 0.270 ± 0.281). A descriptive paired comparison showed that EEGNet scored higher for 60 of 109 subjects, CSP+LDA scored higher for 47 subjects, and 2 subjects were tied. The aggregated EEGNet confusion matrix showed higher recall for hands than for feet (0.735 vs 0.574), and balanced accuracy varied widely across subjects. These results provide a reproducible subject-independent comparison on EEGMMIDB and show that inter-subject variability remains a major barrier to robust MI decoding. The reproducible pipeline also offers a practical neuroinformatics case for teaching biosignal preprocessing, leakage-aware validation, and model comparison, while the subject-level variability shows what must be addressed before assistive educational deployment.

References

Ang, K. K., Chin, Z. Y., Wang, C., Guan, C., & Zhang, H. (2012). Filter bank common spatial pattern algorithm on BCI Competition IV datasets 2a and 2b. Frontiers in Neuroscience, 6, 39. https://doi.org/10.3389/fnins.2012.00039

Antonenko, P., Paas, F., Grabner, R., & van Gog, T. (2010). Using electroencephalography to measure cognitive load. Educational Psychology Review, 22(4), 425–438. https://doi.org/10.1007/s10648-010-9130-y

Autthasan, P., Chaisaen, R., Sudhawiyangkul, T., Rangpong, P., Kiatthaveephong, S., Dilokthanakul, N., Bhakdisongkhram, G., Phan, H., Guan, C., & Wilaiprasitporn, T. (2022). MIN2Net: End-to-end multi-task learning for subject-independent motor imagery EEG classification. IEEE Transactions on Biomedical Engineering, 69(6), 2105–2118. https://doi.org/10.1109/TBME.2021.3137184

Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., & Müller, K.-R. (2008). Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine, 25(1), 41–56. https://doi.org/10.1109/MSP.2008.4408441

Borra, D., Magosso, E., & Ravanelli, M. (2025). A protocol for trustworthy EEG decoding with neural networks. Neural Networks, 182, 106847. https://doi.org/10.1016/j.neunet.2024.106847

Bukach, C. M., Bukach, N., Reed, C. L., & Couperus, J. W. (2021). Open science as a path to education of new psychophysiologists. International Journal of Psychophysiology, 165, 76–83. https://doi.org/10.1016/j.ijpsycho.2021.04.001

Craik, A., He, Y., & Contreras-Vidal, J. L. (2019). Deep learning for electroencephalogram (EEG) classification tasks: A review. Journal of Neural Engineering, 16(3), 031001. https://doi.org/10.1088/1741-2552/ab0ab5

Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23), e215–e220. https://doi.org/10.1161/01.CIR.101.23.e215

Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C., Parkkonen, L., & Hämäläinen, M. S. (2014). MNE software for processing MEG and EEG data. NeuroImage, 86, 446–460. https://doi.org/10.1016/j.neuroimage.2013.10.027

Keutayeva, A., Fakhrutdinov, N., & Abibullaev, B. (2024). Compact convolutional transformer for subject-independent motor imagery EEG-based BCIs. Scientific Reports, 14, 25775. https://doi.org/10.1038/s41598-024-73755-4

Kwon, O.-Y., Lee, M.-H., Guan, C., & Lee, S.-W. (2020). Subject-independent brain–computer interfaces based on deep convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems, 31(10), 3839–3852. https://doi.org/10.1109/TNNLS.2019.2946869

Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018). EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 15(5), 056013. https://doi.org/10.1088/1741-2552/aace8c

Liang, S., Xuan, C., Hang, W., Lei, B., Wang, J., Qin, J., Choi, K.-S., & Zhang, Y. (2023). Domain-generalized EEG classification with category-oriented feature decorrelation and cross-view consistency learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, 3285–3296. https://doi.org/10.1109/TNSRE.2023.3300961

Liu, K., Yang, M., Yu, Z., Wang, G., & Wu, W. (2023). FBMSNet: A filter-bank multi-scale convolutional neural network for EEG-based motor imagery decoding. IEEE Transactions on Biomedical Engineering, 70(2), 436–445. https://doi.org/10.1109/TBME.2022.3193277

Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. Journal of Neural Engineering, 15(3), 031005. https://doi.org/10.1088/1741-2552/aab2f2

Niso, G., Krol, L. R., Combrisson, E., Dubarry, A. S., Elliott, M. A., François, C., Héjja-Brichard, Y., Herbst, S. K., Jerbi, K., Kovic, V., Lehongre, K., Luck, S. J., Mercier, M., Mosher, J. C., Pavlov, Y. G., Puce, A., Schettino, A., Schön, D., Sinnott-Armstrong, W., … Chaumon, M. (2022). Good scientific practice in EEG and MEG research: Progress and perspectives. NeuroImage, 257, 119056. https://doi.org/10.1016/j.neuroimage.2022.119056

Nouri, A. (2025). A scoping review of educational neurotechnology: Methods, applications, opportunities, and challenges. Review of Education, 13, e70070. https://doi.org/10.1002/rev3.70070

Pfurtscheller, G., & Lopes da Silva, F. H. (1999). Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clinical Neurophysiology, 110(11), 1842–1857. https://doi.org/10.1016/S1388-2457(99)00141-8

Roy, Y., Banville, H., Albuquerque, I., Gramfort, A., Falk, T. H., & Faubert, J. (2019). Deep learning-based electroencephalography analysis: A systematic review. Journal of Neural Engineering, 16(5), 051001. https://doi.org/10.1088/1741-2552/ab260c

Sakhavi, S., Guan, C., & Yan, S. (2018). Learning temporal information for brain–computer interface using convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems, 29(11), 5619–5629. https://doi.org/10.1109/TNNLS.2018.2789927

Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wolpaw, J. R. (2004). BCI2000: A general-purpose brain–computer interface (BCI) system. IEEE Transactions on Biomedical Engineering, 51(6), 1034–1043. https://doi.org/10.1109/TBME.2004.827072

Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., & Ball, T. (2017). Deep learning with convolutional neural networks for EEG decoding and visualization. Human Brain Mapping, 38(11), 5391–5420. https://doi.org/10.1002/hbm.23730

Song, Y., Zheng, Q., Liu, B., & Gao, X. (2023). EEG Conformer: Convolutional transformer for EEG decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, 710–719. https://doi.org/10.1109/TNSRE.2022.3230250

Wang, X., Liesaputra, V., Liu, Z., Wang, Y., & Huang, Z. (2024). An in-depth survey on deep learning-based motor imagery electroencephalogram (EEG) classification. Artificial Intelligence in Medicine, 147, 102738. https://doi.org/10.1016/j.artmed.2023.102738

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clinical Neurophysiology, 113(6), 767–791. https://doi.org/10.1016/S1388-2457(02)00057-3

Wu, D., Xu, Y., & Lu, B.-L. (2022). Transfer learning for EEG-based brain–computer interfaces: A review of progress made since 2016. IEEE Transactions on Cognitive and Developmental Systems, 14(1), 4–19. https://doi.org/10.1109/TCDS.2020.3007453

Xu, J., & Zhong, B. (2018). Review on portable EEG technology in educational research. Computers in Human Behavior, 81, 340–349. https://doi.org/10.1016/j.chb.2017.12.037

Xu, L., Xu, M., Ke, Y., An, X., Liu, S., & Ming, D. (2020). Cross-dataset variability problem in EEG decoding with deep learning. Frontiers in Human Neuroscience, 14, 103. https://doi.org/10.3389/fnhum.2020.00103

Zhang, K., Robinson, N., Lee, S.-W., & Guan, C. (2021). Adaptive transfer learning for EEG motor imagery classification with deep convolutional neural network. Neural Networks, 136, 1–10. https://doi.org/10.1016/j.neunet.2020.12.013