Towards a Solution to Create, Test and Publish Mixed Reality Experiences for Occupational Safety and Health LearningTraining-MR

  1. Miguel A. Lopez 1
  2. Sara Terron 1
  3. J. M. Lombardo 1
  4. Rubén Gonzalez-Crespo 2
  1. 1 Fundación I+D del software libre (FIDESOL), Granada (Spain)
  2. 2 Universidad Internacional de La Rioja, Logroño (Spain)
Revista:
IJIMAI

ISSN: 1989-1660

Año de publicación: 2021

Volumen: 7

Número: 2

Páginas: 212-223

Tipo: Artículo

DOI: 10.9781/IJIMAI.2021.07.003 DIALNET GOOGLE SCHOLAR

Otras publicaciones en: IJIMAI

Resumen

Artificial intelligence, Internet of Things, Human Augmentation, virtual reality, or mixed reality have been rapidly implemented in Industry 4.0, as they improve the productivity of workers. This productivity improvement can come largely from modernizing tools, improving training, and implementing safer working methods. Human Augmentation is helping to place workers in unique environments through virtual reality or mixed reality, by applying them to training actions in a totally innovative way. Science still has to overcome several technological challenges to achieve widespread application of these tools. One of them is the democratisation of these experiences, for which is essential to make them more accessible, reducing the cost of creation that is the main barrier to entry. The cost of these mixed reality experiences lies in the effort required to design and build these mixed reality training experiences. Nevertheless, the tool presented in this paper is a solution to these current limitations. A solution for designing, building and publishing experiences is presented in this paper. With the solution, content creators will be able to create their own training experiences in a semiassisted way and eventually publish them in the Cloud. Students will be able to access this training offered as a service, using Microsoft HoloLens2. In this paper, the reader will find technical details of the Training-MR, its architecture, mode of operation and communication.

€ Ver financiación

Información de financiación

The whole Fidesol team for the great work and capacity in "Interacción y Programación Inteligente para la Industria 4.0 a través de la Realidad Mixta y Fabricación Aditiva" project with code "PY18- RE-0024".

Financiadores

Referencias bibliográficas

  • A. P. Botha, “Rapidly arriving futures: Future readiness for industry 4.0,” South African Journal of Industrial Engineering, vol. 29, nov 2018, doi: 10.7166/29-3-2056.
  • J. A. Saucedo-Martínez, M. Pérez-Lara, J. A. Marmolejo-Saucedo, T. E. Salais-Fierro, P. Vasant, “Industry 4.0 framework for management and operations: a review,” Journal of Ambient Intelligence and Humanized Computing, vol. 9, pp. 789–801, jun 2017, doi: 10.1007/s12652-017-0533-1.
  • J. Sengupta, S. Ruj, S. D. Bit, “A secure fog- based architecture for Industrial internet of things and industry 4.0,” IEEE Transactions on Industrial Informatics, vol. 17, pp. 2316–2324, apr 2021, doi: 10.1109/tii.2020.2998105.
  • J. L. Ruiz-Real, J. Uribe-Toril, J. A. Torres, J. D. Pablo, “Artificial Intelligence in Business and Economics Research: Trends and Future,” Journal of Business Economics and Management, vol. 22, pp. 98–117, oct 2020, doi: 10.3846/jbem.2020.13641.
  • P. Milgram, F. Kishino, “A taxonomy of mixed reality visual displays,” IEICE Trans. Information Systems, no. 12, pp. 1321–1329, 1994.
  • M.-D. González-Zamar, E. Abad-Segura, “Implications of virtual reality in arts education: Research analysis in the context of higher education,” Education Sciences, vol. 10, p. 225, aug 2020, doi: 10.3390/educsci10090225.
  • J. Wolfartsberger, “Analyzing the potential of virtual reality for engineering design review,” Automation in Construction, vol. 104, pp. 27–37, aug 2019, doi: 10.1016/j.autcon.2019.03.018.
  • J. M. Lombardo, M. A. Lopez, M. López, M. León, F. Miron, J. Arambarri, D. Álvarez, “MOBEEZE. natural interaction technologies, virtual reality and artificial intelligence for gait disorders analysis and rehabilitation in patients with parkinson’s disease,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 5, no. 6, p. 54, 2019, doi: 10.9781/ijimai.2019.07.003.
  • Y.-C. Du, S.-C. Fan, L.-C. Yang, “The impact of multi-person virtual reality competitive learning on anatomy education: a randomized controlled study,” BMC Medical Education, vol. 20, oct 2020, doi: 10.1186/s12909-020-02155-9.
  • D. V. Joao, P. Z. Lodetti, A. B. dos Santos, M. A. I. Martins, S. de Francisci, J. F. B. Almeida, “Augmented reality application to assist in on-field activities on the electrical sector,” in 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), feb 2021, IEEE.
  • Y. Dan, Z. Shen, Y. Zhu, L. Huang, “Using mixed reality (MR) to improve on-site design experience in community planning,” Applied Sciences, vol. 11, p. 3071, mar 2021, doi: 10.3390/app11073071.
  • E. Bottani, F. Longo, L. Nicoletti, A. Padovano, G. P. C. Tancredi, L. Tebaldi, M. Vetrano, G. Vignali, “Wearable and interactive mixed reality solutions for fault diagnosis and assistance in manufacturing systems: Implementation and testing in an aseptic bottling line,” Computers in Industry, vol. 128, p. 103429, jun 2021, doi: 10.1016/j.compind.2021.103429.
  • “Employer-reported workplace injuries and illnesses 2016,” Bureau of labor statistics. U.S Departament of labor, 2016.
  • J. M. D. Delgado, L. Oyedele, P. Demian, T. Beach, “A research agenda for augmented and virtual reality in architecture, engineering and construction,” Advanced Engineering Informatics, vol. 45, p. 101122, aug 2020, doi: 10.1016/j.aei.2020.101122.
  • J. Wan, Y. Zheng, Y. Li, H. Mei, L. Lin, L. Kuang, “Oil depot safety inspection and emergency training system based on virtual reality technology,” IOP Conference Series: Materials Science and Engineering, vol. 782, p. 042018, apr 2020, doi: 10.1088/1757- 899x/782/4/042018.
  • J. S. D. Orlean G. Dela Cruz, “Virtual reality (vr): A review on its application in construction safety,” Turkish Journal of Computer and Mathematics Education, vol. 12, no. 11, pp. 3379–3393, 2021.
  • R. R. Nick Bostrom, “Ethical issues in human enhancement,” New Waves in Applied Ethics, pp. 120– 152, 2008.
  • T. Garcia, R. Sandler, “Enhancing justice?,” NanoEthics, vol. 2, pp. 277–287, nov 2008, doi: 10.1007/s11569-008- 0048-5.
  • Z. Li, “Ethical problems concerning human augmentation technology and its future aspects,” in Proceedings of the 7th International Conference on Humanities and Social Science Research (ICHSSR 2021), 2021, Atlantis Press.
  • R. Raisamo, I. Rakkolainen, P. Majaranta, K. Salminen, J. Rantala, A. Farooq, “Human augmentation: Past, present and future,” International Journal of Human- Computer Studies, vol. 131, pp. 131–143, nov 2019, doi: 10.1016/j.ijhcs.2019.05.008.
  • E. Pietrafesa, S. Iavicoli, A. Martini, R. Simeone, Polimeni, “Occupational safety and health education and training: an innovative format and experience,” in 6th International Conference on Higher Education Advances (HEAd’20), jun 2020, Universitat Politècnica de València.
  • P.-E. Boileau, “Sustainability and prevention in occupational health and safety,” Industrial Health, vol. 54, no. 4, pp. 293–295, 2016, doi: 10.2486/indhealth.54-293.
  • A. Simeone, A. Caggiano, L. Boun, R. Grant, “Cloud- based platform for intelligent healthcare monitoring and risk prevention in hazardous manufacturing contexts,” Procedia CIRP, vol. 99, pp. 50–56, 2021, doi:10.1016/j.procir.2021.03.009.
  • R. Suganya, S. Gowtham, “Individual health and safety monitoring of workers in deep underground mines using IOT,” Journal of Physics: Conference Series, vol. 1717, p. 012044, jan 2021, doi: 10.1088/1742-6596/1717/1/012044.
  • M. B. V. kumar, M. B. Jayasree, M. D. Kiruthika, “Iot based underground coalmine safety system,” Journal of Physics: Conference Series, vol. 1717, p. 012030, jan 2021, doi: 10.1088/1742-6596/1717/1/012030.
  • V. Jayasree, M. N. Kumari, “IOT based smart helmet for construction workers,” in 2020 7th International Conference on Smart Structures and Systems (ICSSS), jul 2020, IEEE.
  • I. Campero-Jurado, S. Márquez-Sánchez, J. Quintanar- Gómez, S. Rodríguez, J. M. Corchado, “Smart helmet 5.0 for industrial internet of things using artificial intelligence,” Sensors, vol. 20, p. 6241, nov 2020, doi: 10.3390/s20216241.
  • M. A. Gigante, “Virtual reality: Definitions, history and applications,” in Virtual Reality Systems, Elsevier, 1993, pp. 3–14, doi: 10.1016/b978-0-12-227748-1.50009-3.
  • S. S. Kardong-Edgren, S. L. Farra, G. Alinier, H. M. Young, “A call to unify definitions of virtual reality,” Clinical Simulation in Nursing, vol. 31, pp. 28–34, jun 2019, doi: 10.1016/j.ecns.2019.02.006.
  • F. Quigley, A. Moorhead, R. Bond, H. Zheng, T. McAloon, “A virtual reality training tool to improve weight-related communication across healthcare settings,” in Proceedings of the 31st European Conference on Cognitive Ergonomics, sep 2019, ACM.
  • E. A.-L. Lee, K. W. Wong, C. C. Fung, “How does desktop virtual reality enhance learning outcomes? a structural equation modeling approach,” Computers & Education, vol. 55, pp. 1424–1442, dec 2010, doi: 10.1016/j.compedu.2010.06.006.
  • C. A. Cohen, M. Hegarty, “Visualizing cross sections: Training spatial thinking using interactive animations and virtual objects,” Learning and Individual Differences, vol. 33, pp. 63–71, jul 2014, doi: 10.1016/j.lindif.2014.04.002.
  • N. Bouali, E. Nygren, S. S. Oyelere, J. Suhonen, V. Cavalli-Sforza, “Imikode,” in Proceedings of the 19th Koli Calling International Conference on Computing Education Research, nov 2019, ACM.
  • S. Greenwald, A. Kulik, A. Kunert, S. Beck, B. Frohlich, S. Cobb et al., Technology and applications for collaborative learning in virtual reality. 2017.
  • T. Civelek, E. Ucar, H. Ustunel, M. K. Aydın, “Effects of a haptic augmented simulation on k-12 students’ achievement and their attitudes towards physics,” EURASIA Journal of Mathematics, Science and Technology Education, vol. 10, dec 2014, doi: 10.12973/eurasia.2014.1122a.
  • C. Porter, J. Smith, E. Stagar, A. Simmons, M. Nieberding, C. Orban, J. Brown, A. Ayers, “Using virtual reality in electrostatics instruction: The impact of training,” Physical Review Physics Education Research, vol. 16, sep 2020, doi: 10.1103/physrevphyseducres.16.020119.
  • R. H., “Development of virtual reality training for fire safety education,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, pp. 5906–5912, aug 2020, doi: 10.30534/ijatcse/2020/253942020.
  • T. Pitana, H. Prastowo, A. P. Mahdali, “The development of fire safety appliances inspection training using virtual reality (VR) technology,” IOP Conference Series: Earth and Environmental Science, vol. 557, p. 012064, sep 2020, doi: 10.1088/1755- 1315/557/1/012064.
  • M. Li, Z. Sun, Z. Jiang, Z. Tan, J. Chen, “A virtual reality platform for safety training in coal mines with AI and cloud computing,” Discrete Dynamics in Nature and Society, vol. 2020, pp. 1–7, oct 2020, doi: 10.1155/2020/6243085
  • J. M. Lombardo, M. A. Lopez, V. García, M. López, R. Cañadas, S. Velasco, M. León, “PRACTICA. a virtual reality platform for specialized training oriented to improve the productivity,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 5, no. 4, p. 94, 2019, doi: 10.9781/ijimai.2018.04.007.
  • M. A. Lopez, J. M. Lombardo, R. González-Crespo, “Educon 2021-creame: human augmentation platform for thecreation of training in educational lakes inherent todangerous situations.” 2021.
  • M. Speicher, B. D. Hall, M. Nebeling, “What is mixed reality?,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, may 2019, ACM.
  • O. M. Tepper, H. L. Rudy, A. Lefkowitz, K. A. Weimer, S. M. Marks, C. S. Stern, E. S. Garfein, “Mixed reality with HoloLens,” Plastic and Reconstructive Surgery, vol. 140, pp. 1066–1070, nov 2017, doi: 10.1097/prs.0000000000003802.
  • Microsoft, “Hololens 2 techspec,” [Online]. Available: https://www.microsoft.com/en-us/d/hololens2/91pnzzznzwcp?activetab=pivot:techspecstab.
  • Microsoft, “Hololens 2 gestures for authoring and navigating in dynamics 365 guides,” [Online]. Available: https://docs.microsoft.com/en- us/dynamics365/mixed-reality/guides/authoring- gestures-hl2.
  • S. Rokhsaritalemi, A. Sadeghi-Niaraki, S.-M. Choi, “A review on mixed reality: Current trends, challenges and prospects,” Applied Sciences, vol. 10, p. 636, jan 2020, doi: 10.3390/app10020636.
  • R. G. Boboc, F. Gîrbacia, E. V. Butilă, “The application of augmented reality in the automotive industry: A systematic literature review,” Applied Sciences, vol. 10, p. 4259, jun 2020, doi: 10.3390/app10124259.
  • W. Kurschl, S. Pimminger, J. Schönböck, M. Augstein, J. Altmann, “Using mixed reality in intralogistics - are we ready yet?,” Procedia Computer Science, vol. 180, pp. 132–141, 2021, doi: 10.1016/j.procs.2021.01.136.
  • A. Kaluza, M. Juraschek, L. Büth, F. Cerdas, C. Herrmann, “Implementing mixed reality in automotive life cycle engineering: A visual analytics based approach,” Procedia CIRP, vol. 80, pp. 717–722, 2019, doi: 10.1016/j.procir.2019.01.078.
  • B. Bejczy, R. Bozyil, E. Vaičekauskas, S. B. K. Petersen, S. Bøgh, S. S. Hjorth, E. B. Hansen, “Mixed reality interface for improving mobile manipulator teleoperation in contamination critical applications,” Procedia Manufacturing, vol. 51, pp. 620–626, 2020, doi: 10.1016/j.promfg.2020.10.087.
  • R. Zhang, X. Liu, J. Shuai, L. Zheng, “Collaborative robot and mixed reality assisted microgravity assembly for large space mechanism,” Procedia Manufacturing, vol. 51, pp. 38–45, 2020, doi: 10.1016/j.promfg.2020.10.007.
  • A. Siyaev, G.-S. Jo, “Towards aircraft maintenance metaverse using speech interactions with virtual objects in mixed reality,” Sensors, vol. 21, p. 2066, mar 2021, doi: 10.3390/s21062066.
  • X. Wang, “Editorial visualization in engineering,” Visualization in Engineering, vol. 2, mar 2014, doi: 10.1186/2213-7459-2-1.
  • H. Silva, R. Resende, M. Breternitz, “Mixed reality application to support infrastructure maintenance,” in 2018 International Young Engineers Forum (YEF-ECE), may 2018, IEEE.
  • H. Eschen, T. Kötter, R. Rodeck, M. Harnisch, T. Schüppstuhl, “Augmented and virtual reality for inspection and maintenance processes in the aviation industry,” Procedia Manufacturing, vol. 19, pp. 156–163, 2018, doi: 10.1016/j.promfg.2018.01.022.
  • A. Fonnet, N. Alves, N. Sousa, M. Guevara, L. Magalhaes, “Heritage BIM integration with mixed reality for building preventive maintenance,” in 2017 24º Encontro Português de Computação Gráfica e Interação (EPCGI), oct 2017, IEEE.
  • J. Christian, H. Krieger, A. Holzinger, R. Behringer, “Virtual and mixed reality interfaces for e- training: Examples of applications in light aircraft maintenance,” in Lecture Notes in Computer Science, Springer Berlin Heidelberg, 2007, pp. 520–529.
  • F. D. Pace, F. Manuri, A. Sanna, D. Zappia, “A comparison between two different approaches for a collaborative mixed-virtual environment in industrial maintenance,” Frontiers in Robotics and AI, vol. 6, mar 2019, doi: 10.3389/frobt.2019.00018.
  • S. R. Sorko, C. Trattner, J. Komar, “Implementing AR/MR – learning factories as protected learning space to rise the acceptance for mixed and augmented reality devices in production,” Procedia Manufacturing, vol. 45, pp. 367–372, 2020, doi: 10.1016/j.promfg.2020.04.037.
  • S. Lang, M. S. S. D. Kota, D. Weigert, F. Behrendt, “Mixed reality in production and logistics: Discussing the application potentials of microsoft HoloLensTM,” Procedia Computer Science, vol. 149, pp. 118–129, 2019, doi: 10.1016/j.procs.2019.01.115.
  • L. Wunder, N. A. G. Gomez, J. E. Gonzalez, G. Mitzova- Vladinov, M. Cacchione, J. Mato, C. L. Foronda, J. A. Groom, “Fire in the operating room: Use of mixed reality simulation with nurse anesthesia students,” Informatics, vol. 7, p. 40, sep 2020, doi: 10.3390/informatics7040040.
  • H. F. Moore, M. Gheisari, “A review of virtual and mixed reality applications in construction safety literature,” Safety, vol. 5, p. 51, aug 2019, doi: 10.3390/safety5030051.
  • M. Czarski, Y. T. Ng, M. Vogt, M. Juraschek, B. Thiede, P. S. Tan, S. Thiede, C. Herrmann, “A mixed reality application for studying the improvement of HVAC systems in learning factories,” Procedia Manufacturing, vol. 45, pp. 373–378, 2020, doi: 10.1016/j.promfg.2020.04.039.
  • A. Rojo, L. Raya, A. Sanchez, “A novel mixed reality solution based on learning environment for sutures in minor surgery,” Applied Sciences, vol. 11, p. 2335, mar 2021, doi: 10.3390/app11052335.
  • K. Kounlaxay, S. K. Kim, “Design of learning media in mixed reality for lao education,” Computers, Materials & Continua, vol. 64, no. 1, pp. 161–180, 2020, doi: 10.32604/cmc.2020.09930.
  • J. T. T. B. K. Päivi Hämäläinen, “Global estimates of occupational accidents and workrelated illnesses 2017,” in World Congress on Safety and Health at Work 2017, 3-4 September 2017 Singapore, 2017.
  • J. Takala, P. Hämäläinen, K. L. Saarela, L. Y. Yun, K. Manickam, T. W. Jin, P. Heng, C. Tjong, L. G. Kheng, S. Lim, G. S. Lin, “Global estimates of the burden of injury and illness at work in 2012,” Journal of Occupational and Environmental Hygiene, vol. 11, pp. 326–337, apr 2014, doi: 10.1080/15459624.2013.863131.
  • H. Wang, M. Naghavi, C. Allen, R. M. Barber, “Global, regional, and national life expectancy, all- cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: a systematic analysis for the global burden of disease study 2015,” Lancet, vol. 388, no. 10053, pp. 1459–1544., 2016.
  • “Employer-reported workplace injury and illness - 2016,” Bureau of Labor Statistics, 2017. [Online]. Available: https://www.bls.gov/news.release/pdf/ osh.pdf.
  • D. Fan, C. J. Zhu, A. R. Timming, Y. Su, X. Huang, Y. Lu, “Using the past to map out the future of occupational health and safety research: where do we go from here?,” The International Journal of Human Resource Management, vol. 31, pp. 90–127, sep 2019, doi: 10.1080/09585192.2019.1657167.
  • A. Kramer, S. Cho, R. S. Gajendran, “12-year longitudinal study linking within-person changes in work and family transitions and workplace injury risk,” Journal of Safety Research, vol. 75, pp. 140–149, dec 2020, doi: 10.1016/j.jsr.2020.08.009.
  • K. Schwaber, “SCRUM development process,” in Business Object Design and Implementation, Springer London, 1997, pp. 117–134.
  • M. A. Lopez, M. D. Ruiz, D. Alvarez, “Designme-mr: Toolbox for the creation of learning scenes to training in occupation risk prevention with mixed reality.” June 2021.
  • L.-P. T. Kamalmeet Singh, Adrian Ianculescu, Design Patterns and Best Practices in Java. 2018.
  • M. Richards, Software Architecture Patterns. 2015.
Fuente de los datos: Dialnet