Does e-assessment always fit digital natives? A within-subject comparison between paper- and tablet-based gambling assessments in adolescents

  1. Víctor Martínez-Loredo 1
  2. Alba González-Roz 1
  3. Eduardo García-Cueto 1
  4. Aris Grande-Gosende 1
  5. Fernández-Hermida 1
  1. 1 Universidad de Oviedo
    info

    Universidad de Oviedo

    Oviedo, España

    ROR https://ror.org/006gksa02

Aldizkaria:
Revista de Psicología Clínica con Niños y Adolescentes

ISSN: 2340-8340

Argitalpen urtea: 2021

Alea: 8

Zenbakia: 2

Orrialdeak: 17-22

Mota: Artikulua

DIALNET GOOGLE SCHOLAR lock_openDialnet editor

Beste argitalpen batzuk: Revista de Psicología Clínica con Niños y Adolescentes

Laburpena

Technological development has enabled the use of sophisticated methods for assessing multiple human behaviors. Despite the advantages of these new technologies, concerns exist regarding their equivalence with paper-based measures in epidemiological and health-related surveys. To date, literature on this topic in relation to adolescents is virtually nonexistent. This study compares respondents’ performance on the same survey using both paper- and electronic tablet-based assessment methods. A final sample of 135 adolescents (mean age 17.30 years, SD = 0.59; 56.3% males) consecutively completed two versions of the same survey on gambling behaviors and two questionnaires: The Gambling Motives Questionnaire (GMQ) and the South Oaks Gambling Screening-Revised for Adolescents (SOGS-RA). An ad-hoc questionnaire assessing participants’ satisfaction levels with each method was also used. The digital survey yielded a lifetime, past year, and past month gambling prevalence of 54.1%, 45.2%, and 27.4%, respectively. Paper-based prevalence rates were 3.7-5.2% lower (all p < .092) and there were discrepancies in gambling activities. Although the reliability of the questionnaires was high in both formats, total scores were consistently higher in the paper-based format. GMQ and SOGS-RA intraclass correlations between versions ranged from .856-.884. Unexpectedly, students preferred the paper-based survey to the e-assessment (51.5% vs. 48.5%) and also enjoyed it more (31.3% vs 26.1%). Paper- and tablet-based surveys yield different, albeit non-statistically significant, estimations of gambling behaviors even when the same participants were surveyed at one time. We recommend that consistency be routinely checked across assessment formats when adapting paper-and-pencil measures to digital formats

Erreferentzia bibliografikoak

  • Bailey, S.K.T., Neigel, A.R., Dhanani, L.Y., & Sims, V.K. (2018). Establishing measurement equivalence across computer- and paper-based tests of spatial cognition. Human Factors, 60(3), 340–350. https://doi. org/10.1177/0018720817747731
  • Beck, F., Guignard, R., & Legleye, S. (2014). Does computer survey technology improve reports on alcohol and illicit drug use in the general population? A comparison between two surveys with different data collection modes in France. PLoS ONE, 9(1). https://doi.org/10.1371/journal.pone.0085810
  • Becoña, E. (1997). Pathological gambling in Spanish children and adolescents: An emerging problem. Psychological Reports, 81(1), 275–287.
  • Boo, J., & Vispoel, W. (2012). Computer versus paper-and-pencil assessment of educational development: A comparison of psychometric features and examinee preferences. Psychological Reports, 111(2), 443–460. https://doi.org/10.2466/10.03.11.PR0.111.5.443-460
  • Fonseca-Pedrero, E., Paíno-Piñeiro, M., Lemos-Giráldez, S., Villazón-García, Ú., & Muñiz, J. (2009). Validation of the Schizotypal Personality Questionnaire—Brief Form in adolescents. Schizophrenia Research, 111(1), 53–60. https://doi.org/10.1016/j.schres.2009.03.006
  • Gnambs, T., & Kaspar, K. (2014). Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behavior Research Methods, 47(4), 1237–1259. https://doi.org/10.3758/s13428-014-0533-4
  • González-Roz, A., Fernández-Hermida, J.R., Weidberg, S., Martínez-Loredo, V., & Secades-Villa, R. (2017). Prevalence of problem gambling among adolescents: A comparison across modes of access, gambling activities, and levels of severity. Journal of Gambling Studies, 33(2), 371–382. https://doi.org/10.1007/s10899-016-9652-4
  • Grande-Gosende, A., Martínez-Loredo, V., & Fernández-Hermida, J.R. (2019). Gambling motives questionnaire validation in adolescents: Differences based on gambling severity and activities. Adicciones, 31(3), 212-220. https://doi.org/10.20882/adicciones.1012
  • Gwaltney, C.J., Shields, A.L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value in Health, 11(2), 322–333. https://doi. org/10.1111/j.1524-4733.2007.00231.x
  • Khazaal, Y., Chatton, A., Monney, G., Nallet, A., Khan, R., Zullino, D., & Etter, J.F. (2015). Internal consistency and measurement equivalence of the cannabis screening questions on the paper-and-pencil face-to-face ASSIST versus the online instrument. Substance Abuse: Treatment, Prevention, and Policy, 10(1), 2–11. https://doi.org/10.1186/s13011-015-0002-9
  • Lai, C.K.Y., Kwan, R.Y.C., Cheung, D.S.K., Wu, Y.M., & Yap, L.S.Y. (2016). A computerized cognitive assessment method in a nurse-led clinic. CIN: Computers, Informatics, Nursing, 34(12), 554–559. https://doi.org/10.1097/cin.0000000000000283
  • Livingston, M.D., Komro, K.A., & Wagenaar, A.C. (2015). The effects of survey modality on adolescents’ responses to alcohol use items. Alcoholism: Clinical and Experimental Research, 39(4), 710–715. https://doi.org/10.1111/acer.12659
  • Margolin, S.J., Driscoll, C., Toland, M.J., & Kegler, J.L. (2013). E-readers, computer screens, or paper: Does reading comprehension change across media platforms? Applied Cognitive Psychology, 27(4), 512-519. https://doi.org/10.1002/acp.2930
  • Mauz, E., von der Lippe, E., Allen, J., Schilling, R., Müters, S., Hoebel, J., Schmich, P., Wetzstein, M., Kamtsiuris, P., & Lange, C. (2018). Mixing modes in a population-based interview survey: Comparison of a sequential and a concurrent mixed-mode design for public health research. Archives of Public Health, 76(1), 1–17. https://doi.org/10.1186/s13690-017-0237-1
  • Meyer, C., Bischof, A., Westram, A., Jeske, C., de Brito, S., Glorius, S., Schön, D., Porz, S., Gürtler, D., Kastirke, N., Hayer, T., Jacobi, F., Lucht, M., Premper, V., Gilberg, R., Hess, D., Bischof, G., John, U., & Rumpf, H-J. (2015). The “Pathological Gambling and Epidemiology”(PAGE) study program: design and fieldwork. International Journal of Methods in Psychiatric Research, 24(1), 11–31. https://doi.org/10.1002/mpr.1458
  • Monteiro, S., Sibbald, D., & Coetzee, K. (2018). i-Assess: Evaluating the impact of electronic data capture for OSCE. Perspectives on Medical Education, 7(2), 110–119. https://doi.org/10.1007/s40037-018-0410-4
  • Moragas, L., Granero, R., Stinchfield, R., Fernández-Aranda, F., Fröberg, F., Aymamí, N., Gómez-Peña, M., Fagundo, A.B., Islam, M.A., Pino-Gutiérrez, A., Agüera, Z., Savvidou, L.G., Arcelus, J., Witcomb, G.L., Sauchelli, S., Menchón, J.M., & Jiménez-Murcia, S. (2015). Comparative analysis of distinct phenotypes in gambling disorder based on gambling preferences. BMC Psychiatry, 15, 1-11. https://doi.org/10.1186/s12888-015-0459-0
  • Muñiz, J. (2018). Introducción a la psicometría. Teoría Clásica y TRI. [Introduction to psychometry. Classical Theory and IRT]. Pirámide. Noyes, J.M., & Garland, K.J. (2008). Computer- vs. paper-based tasks: Are they equivalent? Ergonomics, 51(9), 1352–1375. https://doi. org/10.1080/00140130802170387
  • Penner, I.K., Kobel, M., Stöcklin, M., Weber, P., Opwis, K., & Calabrese, P. (2012). The stroop task: Comparison between the original paradigm and computerized versions in children and adults. Clinical Neuropsychologist, 26(7), 1142–1153. https://doi.org/10.1080/13854046.2012.713513
  • Piaw, C. Y. (2011). Comparisons between computer-based testing and paper-pencil testing: Testing effect, test scores, testing time and testing. The Informatics Conference, January 2011, 1–9.
  • Piaw, C. Y. (2012). Replacing paper-based testing with computer-based testing in assessment: Are we doing wrong? Procedia - Social and Behavioral Sciences, 64, 655–664. https://doi.org/10.1016/j.sbspro.2012.11.077
  • Porion, A., Aparicio, X., Megalakaki, O., Robert, A., & Baccino, T. (2016). The impact of paper-based versus computerized presentation on text comprehension and memorization. Computers in Human Behavior, 54, 569-576. https://doi.org/10.1016/j.chb.2015.08.002
  • Retnawati, H. (2015). The comparison of accuracy scores on the paper and pencil testing vs. Computer-based testing. Turkish Online Journal of Educational Technology, 14(4), 135–142.
  • Spark, S., Lewis, D., Vaisey, A., Smyth, E., Wood, A., Temple-Smith, M., Lorch, R., Guy, R., & Hocking, J. (2015). Using computer-assisted survey instruments instead of paper and pencil increased completeness of self-administered sexual behavior questionnaires. Journal of Clinical Epidemiology, 68(1), 94–101. https://doi.org/10.1016/j.jclinepi.2014.09.011
  • Stewart, S.H., & Zack, M. (2008). Development and psychometric evaluation of a three‐dimensional Gambling Motives Questionnaire. Addiction, 103(7), 1110–1117. https://doi.org/10.1111/j.1360-0443.2008.02235.x
  • Weidberg, S., González-Roz, A., Fernández-Hermida, J.R., Martínez-Loredo, V., Grande-Gonsende, A., García-Pérez, Á., & Secades-Villa, R. (2018). Gender differences among adolescent gamblers. Personality and Individual Differences, 125, 38–43. https://doi.org/10.1016/j.paid.2017.12.031
  • Winters, K.C., Stinchfield, R.D., & Fulkerson, J. (1993). Toward the development of an adolescent gambling problem severity scale. Journal of Gambling Studies, 9(1), 63–84. https://doi.org/10.1007/BF01019925
Datuen iturria: Dialnet