Psychology, Interamerican
Evidence of validity and precision of the instruments used to measure mental workload: an integrative review of the literature
Creative Commons
PDF (Español (España))

Keywords

Carga mental
propiedades psicométricas
instrumento
medición

How to Cite

Rodio-Trevisan, K. R., Moraes Cruz, R., Nunes, C., Ornellas Ariño, D., Dalagasperina, P., Trevisan, R. L., & Molina Albarracín, D. C. (2024). Evidence of validity and precision of the instruments used to measure mental workload: an integrative review of the literature. Revista Interamericana De Psicología/Interamerican Journal of Psychology, 58(1), e1894. https://doi.org/10.30849/ripijp.v58i1.1894

Abstract

La carga mental de trabajo representa el valor asignado al esfuerzo mental requerido para realizar las tareas. El objetivo de este estudio fue identificar evidencias de validez y precisión de los instrumentos utilizados para medir la carga de trabajo mental en los últimos 10 años, a través de una revisión de la literatura. Se realizaron búsquedas en las bases de datos Scopus, Web Of Science, PsycNet, PubMed y en el portal de la BVS (2012-2021), en inglés, español y portugués. Se incluyeron un total de 145 artículos y se ubicaron 16 instrumentos para medir CMT. Los resultados demuestran el uso de información sobre las propiedades psicométricas de los instrumentos que se basan en conceptos y procedimientos estadísticos que no siempre son consistentes con la verificación de la estructura interna del constructo. La validez de la estructura interna de los instrumentos fue verificada en sólo cuatro estudios, con índices de confiabilidad satisfactorios, pero aún con el uso de conceptos y metodologías de tratamiento y análisis insatisfactorios. Existen, por tanto, desafíos metodológicos, técnicos y conceptuales en los procesos de medición de la carga mental de trabajo.

https://doi.org/10.30849/ripijp.v58i1.1894
PDF (Español (España))

References

Abich I. V. J., Reinerman-Jones, L., & Matthews, G. (2017). Impact of three task demand factors on simulated intelligence, surveillance, and reconnaissance of unmanned systems. Ergonomics, 60 (6), 791-809. http://doi.org/10.1080/00140139.2016.1216171

Alberdi, A., Aztiria, A., Basarab, A., & Cook, D. J. (2018). Using smart offices to predict occupational stress. International Journal of Industrial Ergonomics, 67, 13-26. http://doi.org/10.1016/j.ergon.2018.04.005

American Educational Research Association, AERA, American Psychological Association, APA, & Natiional Council on Measurement in Education, NCME. (2014). Standards for education and psychological testing. American Educational Research Association.

Associação Brasileira de Psicologia Organizacional e do Trabalho [SBPOT]. (2020). Competências para a atuação em psicologia organizacional e do trabalho: um referencial para a formação e qualificação profissional no Brasil. UniCEUB. https://repositorio.uniceub.br/jspui/handle/prefix/14119

Atalay, K. D., Can, G. F., Erdem, S. R., & Müderrisoglu, I. H. (2016). Assessment of mental workload and academic motivation in medical students. The Journal of the Pakistan Medical Association, 66(5), 574-578. http://europepmc.org/abstract/med/27183939

Ba, Y., & Zhang, W. (2011, Julho). A review of driver mental workload in driver-vehicle-environment system. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), San Jose, CA. https://link.springer.com/content/pdf/10.1007/978-3-642-21660-2.pdf#page=143

Bara, L., Alessandra, L. M., Meloni, L., Giusino, D., & Pietrantoni, L. (2021). Assessment Methods of Usability and Cognitive Workload of Rehabilitative Exoskeletons: A Systematic Review. Applied Sciences, 11(15), 7146. https://doi.org/10.3390/app11157146

Braarud, P. Ø. (2021). Investigating the validity of subjective workload rating (NASA TLX) and subjective situation awareness rating (SART) for cognitively complex human–machine work. International Journal of Industrial Ergonomics, 86, 103233. https://doi.org/10.1016/j.ergon.2021.103233

Braarud, P. Ø. (2020). An efficient screening technique for acceptable mental workload based on the NASA Task Load Index—development and application to control room validation. International Journal of Industrial Ergonomics, 76, 102904. https://doi.org/10.1016/j.ergon.2019.102904

Bridger, R. S., & Brasher, K. (2011). Cognitive task demands, self-control demands and the mental well-being of office workers. Ergonomics, 54(9), 830-839. https://doi.org/10.1080/00140139.2011.596948

Ceballos-Vásquez, P., Paravic Klijn, T., Burgos Moreno, M., & Barriga, O. (2014). Validación de Escala Subjetiva de Carga Mental de Trabajo en funcionarios/as universitarios. Ciencia y enfermería, 20(2), 73-82. https://doi.org/10.4067/S0034988720120009008

Ceballos-Vásquez, P., Rolo-González, G., Hernández-Fernaud, E., Díaz-Cabrera, D., Paravic-Klijn, T., Burgos-Moreno, M., & Barriga, O. (2016). Validación de la Escala Subjetiva de Carga mental de trabajo (ESCAM) en profesionales de la salud de Chile. Universitas Psychologica, 15(1), 261-270. https://doi.org/10.11144/Javeriana.upsy15-1.vsmw

Charles, R. L., & Nixon, J. (2019). Measuring mental workload using psysiological measures: A systematic review. Applied ergonomics, 74, 221-232. https://doi.org/10.1016/j.apergo.2018.08.028

Dai, J., Wang, H., Yang, L., Cao, X., Wang, C., Gao, Z., Hu, W., & Wen, Z. (2021). The effects of emotional trait factors on simulated flight performance under an acute psychological stress situation. International journal of occupational safety and ergonomics, 1-8. https://doi.org/10.1080/10803548.2021.1994750

Devos, H., Gustafson, K., Ahmadnezhad, P., Liao, K., Mahnken, J. D., Brooks, W. M., & Burns, J. M. (2020). Psychometric properties of NASA-TLX and Index of Cognitive Activity as measures of cognitive workload in older adults. Brain Sciences, 10(12), 994. https://doi.org/10.3390/brainsci10120994

Digiesi, S., Manghisi, V. M., Facchini, F., Klose, E. M., Foglia, M. M., & Mummolo, C. (2020). Heart rate variability based assessment of cognitive workload in smart operators. Management and Production Engineering Review, 11. https://doi.org/10.24425/mper.2020.134932

Fallahi, M., Motamedzade, M., Heidarimoghadam, R., Soltanian, A. R., & Miyake, S. (2016). Assessment of operators’ mental workload using physiological and subjective measures in cement, city traffic and power plant control centers. Health promotion perspectives, 6(2), 96. https://doi.org/10.15171/hpp.2016.17

Ferrer, R., & Dalmau, I. (2004). Revisión del concepto de carga mental: Evaluación, consecuencias y proceso de normalización. Anuario de Psicología, 35(4), 521-545. https://dialnet.unirioja.es/servlet/articulo?codigo=1119140

Frutuoso, J.T., & Cruz, R. M. (2005). Mensuração da carga de trabalho e sua relação com a saúde do trabalhador. Revista Brasileira de Medicina do Trabalho, 3(1), 29-36. http://www.rbmt.org.br/details/166/pt-BR/work-load-evaluation-and-its-relation-with-workers--health-conditions

Galvão, T. F., Pansani, T. D. S. A., & Harrad, D. (2015). Principais itens para relatar Revisões sistemáticas e Meta-análises: A recomendação PRISMA. Epidemiologia e serviços de saúde, 24, 335-342. https://doi.org/10.5123/S1679-49742015000200017

Galy, E. (2020). A multidimensional scale of mental workload evaluation based on Individual–Workload–Activity (IWA) model: validation and relationships with job satisfaction. Tutorials in Quantitative Methods for Psychology, 16, 240-252. https://doi.org/10.20982/tqmp.16.3.p240

Ghanavati, F. K., Choobineh, A., Keshavarzi, S., Nasihatkon, A. A., & Roodbandi, A. S. J. (2019). Assessment of mental workload and its association with work ability in control room operators. La Medicina del lavoro, 110(5), 389. https://doi.org/10.23749/mdl.v110i5.8115

Guru, K. A., Shafiei, S. B., Khan, A., Hussein, A. A., Sharif, M., & Esfahani, E. T. (2015). Understanding cognitive performance during robot-assisted surgery. Urology, 86(4), 751-757. https://doi.org/10.1016/j.urology.2015.07.028

Hart, S. G., & Staveland, L. (1988). Development of the NASA task load index (TLX): Results of empirical and theoretical research. Human mental workload. Amsterdam: North-Holland. Holland MA, Ziavra NV, Bronstein AM (2004) A new paradigm to investigate the roles of head and eye movements in coordination of whole-body movements. Exp Brain Res, 154, 261-266. https://doi.org/10.1016/S0166-4115(08)62386-9

Heidarimoghadam, R., Saidnia, H., Joudaki, J., Mohammadi, Y., & Babamiri, M. (2019). Does mental workload can lead to musculoskeletal disorders in healthcare office workers? Suggest and investigate a path. Cogent Psychology, 6(1), 1664205. https://doi.org/10.1080/23311908.2019.1664205

Helton, W. S., Funke, G. J., & Knott, B. A. (2014). Measuring workload in collaborative contexts: trait versus state perspectives. Human factors, 56(2), 322-332. https://doi.org/10.1177/0018720813490727

Horat, S. K., Herrmann, F. R., Favre, G., Terzis, J., Debatisse, D., Merlo, M. C., & Missonnier, P. (2016). Assessment of mental workload: a new electrophysiological method based on intra-block averaging of ERP amplitudes. Neuropsychologia, 82, 11-17. https://doi.org/10.1016/j.neuropsychologia.2015.12.013

Li, J. Q., Zhang, H. Y., Zhang, Y., & Liu, H. T. (2018). Systematic assessment of intrinsic factors influencing visual attention performances in air traffic control via clustering algorithm and statistical inference. Plos one, 13(10), e0205334. https://doi.org/10.1371/journal.pone.0205334

Laurie-Rose, C., Frey, M., Ennis, A., & Zamary, A. (2014). Measuring perceived mental workload in children. The American journal of psychology, 127(1), 107-125. https://doi.org/0.5406/amerjpsyc.127.1.0107

Laurie-Rose, C., Curtindale, L. M., & Frey, M. (2017). Measuring Sustained Attention and Perceived Workload: A Test With Children. Human factors, 59(1), 76-90. https://doi.org/10.1177/0018720816684063

Longo, L. (2018). Experienced mental workload, perception of usability, their interaction and impact on task performance. PloS one, 13(8), e0199661. https://doi.org/10.1371/journal.pone.0199661

Lu, C., Hu, Y., Fu, Q., Governor, S., Wang, L., Li, C., Deng, L., & Xie, J. (2019). Physician mental workload scale in China: development and psychometric evaluation. BMJ open, 9(10), e030137. https://doi.org/10.1136/bmjopen-2019-030137

Marçon, B., Ngueyon Sime, W., Guillemin, F., Hubert, N., Lagrange, F., Huselstein, C., & Hubert, J. (2019). An ergonomic assessment of four different donor nephrectomy approaches for the surgeons and their assistants. Research and Reports in Urology, 261-268. https://doi.org/10.2147/rru.s220219

Matthews, G., Reinerman-Jones, L. E., Barber, D. J., & Abich I. V, J. (2015). The psychometrics of mental workload: Multiple measures are sensitive but divergent. Human Factors, 57(1), 125-143. http://doi.org/10.1177/0018720814539505

Mazur, L. M., Adams, R., Mosaly, P. R., Stiegler, M. P., Nuamah, J., Adapa, K., Chera, B., & Marks, L. B. (2020). Impact of Simulation-Based Training on Radiation Therapists’ Workload, Situation Awareness, and Performance. Advances in Radiation Oncology, 5(6), 1106-1114. https://doi.org/10.1016/j.adro.2020.09.008

McNeer, R. R., Bennett, C. L., & Dudaryk, R. (2016). Intraoperative noise increases perceived task load and fatigue in anesthesiology residents: a simulation-based study. Anesthesia & Analgesia, 122(2), 512-525. https://doi.org/10.1213/ANE.0000000000001067

Mohamed, R., Raman, M., Anderson, J., McLaughlin, K., Rostom, A., & Coderre, S. (2014). Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate-the learning curve for endoscopy training. Canadian Journal of Gastroenterology and Hepatology, 28(3), 155-160. https://doi.org/10.1155/2014.892476

Mohammadi, M., Mazloumi, A., Kazemi, Z., & Zeraati, H. (2015). Evaluation of Mental Workload among ICU Ward's Nurses. Health promotion perspectives, 5(4), 280. https://doi.org/10.15171/hpp.2015.033

Monfort, S. S., Graybeal, J. J., Harwood, A. E., McKnight, P. E., & Shaw, T. H. (2017). A single-item assessment for remaining mental resources: development and validation of the Gas Tank Questionnaire (GTQ). Theoretical Issues in Ergonomics Science, 1-23. https://doi.org/10.1080/1463922X.2017.1397228

Mouze-Amady, M., Raufaste, E., Prade, H., & Meyer, J. P. (2013). Fuzzy-TLX: using fuzzy integrals for evaluating human mental workload with NASA-Task Load index in laboratory and field studies. Ergonomics, 56(5), 752-763. https://doi.org/10.1080/00140139.2013.776702

Nasirizad Moghadam, K., Chehrzad, M. M., Reza Masouleh, S., Maleki, M., Mardani, A., Atharyan, S., & Harding, C. (2021). Nursing physical workload and mental workload in intensive care units: Are they related? Nursing Open, 8(4), 1625-1633. https://doi.org/10.1002/nop2.785

Nayar, S. K., Musto, L., Fernandes, R., & Bharathan, R. (2019). Validation of a virtual reality laparoscopic appendicectomy simulator: a novel process using cognitive task analysis. Irish Journal of Medical Science, 188(3), 963-971. https://doi.org/10.1007/s11845-018-1931-x

Nino, L., Marchak, F., & Claudio, D. (2020). Physical and mental workload interactions in a sterile processing department. International Journal of Industrial Ergonomics, 76, 102902. https://doi.org/10.1016/j.ergon.2019.102902

Nodoushan, R. J., Madadizadeh, F., Anoosheh, V. S., Boghri, F., & Chenani, K. T. (2021). Mental workload and occupational burnout among the faculty members and administrative staff of Yazd Public Health School. Journal of Education and Health Promotion, 10. https://doi.org/10.4103/jehp.jehp_1076_200

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Telzlaff, J. M., Akl., E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J.M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L.A., Thomas, J., Tricco, A. C., Welch, V.A., Whiting, P., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372(71). https://doi.org/10.1136/bmj.n71

Peruzzini, M., Tonietti, M., & Iani, C. (2019). Transdisciplinary design approach based on driver's workload monitoring. Journal of Industrial Information Integration, 15, 91-102. https://doi.org/10.1016/j.jii.2019.04.001

Piranveyseh, P., Kazemi, R., Soltanzadeh, A., & Smith, A. (2021). A field study of mental workload: conventional bus drivers versus bus rapid transit drivers. Ergonomics, 1-11. https://doi.org/10.1080/00140139.2021.1992021

Rodríguez-López, A. M., Rubio-Valdehita, S., & Díaz-Ramiro, E. M. (2021). Influence of the Covid-19 pandemic on mental workload and burnout of fashion retailing workers in Spain. International Journal of Environmental Research and Public Health, 18(3), 983. https://doi.org/10.3390/ijerph18030983

Ródio-Trevisan, K.R. (2020). Avaliação da associação entre carga mental de trabalho, fatores de risco psicossociais ocupacionais e agravos à saúde mental dos professores (Dissertação de mestrado, Universidade Federal de Santa Catarina).

Ródio-Trevisan, K. R., Cruz, R. M., Crocce-Carlotto, P. A., & Guilland, R. (2023). Construção e evidências de validade da Escala de Rastreio da Carga Mental de Trabalho. Trabalho (En) Cena, 8(Contínuo), e023021-e023021. https://doi.org/10.20873/2526-1487e023021

Rostami, F., Babaei-Pouya, A., Teimori-Boghsani, G., Jahangirimehr, A., Mehri, Z., & Feiz-Arefi, M. (2021). Mental Workload and Job Satisfaction in Healthcare Workers: The Moderating Role of Job Control. Frontiers in Public Health, 9. https://doi.org/10.3389/fpubh.2021.683388

Rubio-Valdehita, S., López-Núñez, M. I., López-Higes, R., & Díaz-Ramiro, E. M. (2017). Development of the CarMen-Q Questionnaire for mental workload assessment. Psicothema, 29(4). https://doi.org/10.7334/psicothema2017.151

Sarsangi, V., Salehiniya, H., Hannani, M., Marzaleh, M. A., Abadi, Y. S., Honarjoo, F., Dehkordi, A. S., & Derakhshanjazari, M. (2017). Assessment of workload effect on nursing occupational accidents in hospitals of Kashan, Iran. Biomedical Research and Therapy, 4(08), 1527-1540. http://old.bmrat.org/index.php/BMRAT/article/view/226

Schoenenberger, S., Moulin, P., Brangier, E., & Gilibert, D. (2014). Patients’ characteristics and healthcare providers’ perceived workload in French hospital emergency wards. International Journal of Occupational Safety and Ergonomics, 20(4), 551-559. https://doi.org/10.1080/10803548.2014.11077071

Shan, Y., Shang, J., Yan, Y., Lu, G., Hu, D., & Ye, X. (2021). Mental workload of frontline nurses aiding in the COVID‐19 pandemic: A latent profile analysis. Journal of Advanced Nursing, 77(5), 2374-2385. https://doi.org/10.1111/jan.14769

Teh, E., Jamson, S., Carsten, O., & Jamson, H. (2014). Temporal fluctuations in driving demand: The effect of traffic complexity on subjective measures of workload and driving performance. Transportation research part F: traffic psychology and behaviour, 22, 207-217. https://doi.org/10.1016/j.trf.2013.12.005

Van Hoffen, M. F. A., Heymans, M. W., Twisk, J. W. R., van Rhenen, W., & Roelen, C. A. M. (2016). Can psychosocial working conditions identify workers at risk of mental sickness absence? European Journal of Public Health, 26. https://doi.org/10.1093/eurpub/ckw164.082

Whittemore, R., & Knafl, K. (2005). The integrative review: updated methodology. Journal of advanced nursing, 52(5), 546-553. https://doi.org/10.1111 / j.1365-2648.2005.03621.x

Wilbanks, B. A., & McMullan, S. P. (2018). A review of measuring the cognitive workload of electronic health records. CIN: Computers, Informatics, Nursing, 36(12), 579-588. https://doi.org/10.1097/CIN.0000000000000469

Wu, J., Li, H., Geng, Z., Wang, Y., Wang, X., & Zhang, J. (2021). Subtypes of nurses’ mental workload and interaction patterns with fatigue and work engagement during coronavirus disease 2019 (COVID-19) outbreak: A latent class analysis. BMC nursing, 20(1), 1-9. https://doi.org/10.1186/s12912-021-00726-9

Young, M. S., Brookhuis, K. A., Wickens, C. D., & Hancock, P. A. (2015). State of science: mental workload in ergonomics. Ergonomics, 58(1), 1-17. https://doi.org/10.108/00140139.2014.956151

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2024 Karen Rayany Rodio-Trevisan, Roberto Moraes Cruz, Cyntia Nunes, Daniela Ornellas Ariño, Patricia Dalagasperina, Rafaela Luiza Trevisan, Doris Cecilia Molina Albarracín