MEASURING PERFORMANCE DURING A FALLBACK PROCEDURE IN AUTONOMOUS VEHICLES
DOI:
https://doi.org/10.46585/pc.2022.1.2130Keywords:
autonomous vehicles, automation, situational awareness, fallback task, fallback performance measurement, experimentAbstract
The aim of this paper was to assess the functionality of an experiment that was developed for the DRILL project. The experiment used a computer program and a distracting task to measure a driver’s ability to take control of an autonomous vehicle (fallback performance). Switching theory and tachistoscopic traffic tests served as the foundation for this experiment. The experiment measured the subject's reaction time and the correctness of the reaction. The final sample consisted of N = 48 participants, who were self-selected. The results of the statistical analyses suggest that the experiment does successfully induce and register the switching effect. It confirmed the linear relationship between the results of the experiment and the results of the ATAVT test. Further, it analyzed the reaction time on the item level (e. g. reaction times for an item level depending on its order, the effect of the correctness of the item´s response on the reaction time). However, the small size of the task-switching effect and the unresolved method of evaluation has not yet allowed us to draw a clear conclusion about the functionality of the experiment. Possibilities for improvements to the experiment design are also presented.
Downloads
References
Allport, A., Styles, E. A. and Hsieh, S. 1994. Shifting intentional set: Exploring the dynamic control of tasks. In C. Umilta & M. Moscovitch (Eds.), Conscious and nonconscious information processing: Attention and peifonnance XV (pp. 421-452). Cambridge, MA: MIT Press.
Altmann, E. M. 2005. Repetition priming in task switching: Do the benefits dissipate? Psychonomic Bulletin & Review, 2, pp. 535-540.
Anund, A., Lindström, A., Schagen, I. V. and Linder, A. 2020. FERSI position paper: Safety through automation? Ensuring that automated and connected driving contributes to a safer transportation system. In Rethinking transport. 8th Transport Research Arena TRA 2020, April 27-30, 2020. Helsingfors, Finland (Conference canceled) (p. 5).
Aria, E., Olstam, J., and Schwietering, C. 2016. Investigation of Automated Vehicle Effects on Driver’s Behavior and Traffic Performance, 15, 761–770. <http://doi.org/10.1016/j.trpro.2016.06.063>
Bartolacci, C., Scarpelli, S., D’Atri, A., Gorgoni, M., Annarumma, L., Cloos, C., ... and De Gennaro, L. 2020. The Influence of Sleep Quality, Vigilance, and Sleepiness on driving-related Cognitive Abilities: A Comparison between Young and Older Adults. Brain Sciences, 10(6), 327.
Brynjolfsson, E. and McAfee, A. 2012. Winning the Race with Ever-Smarter Machines. MIT Sloan Management Review, 53(2), 53-60.
Caban, J., Nieoczym, A., Dudziak, A., Krajka, T. and Stopková, M. 2022. The Planning Process of Transport Tasks for Autonomous Vans—Case Study. Appllied Sciences, 12(6), pp. 2993.
California DMV. 2019. Disengagement reports. [Online]. Available at: https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/ [Accessed: 20. 1. 2021].
Carsten, O., Lai, F. C. H., Barnard, Y., Jamson, A. H. and Merat, N. 2012. Control Task Substitution in Semiautomated Driving. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(5), 747–761.
Chakrabarty, N., Gupta, M. K., Velmurugan, S. and Rikku, M. R. 2015. Assessment of Driving Behaviour and Skills of Heavy Vehicle Drivers in India. Paper ID. 451.
Cummings, M. L. and Ryan, J. 2014. POINT OF VIEW: Who Is in Charge? The Promises and Pitfalls of Driverless Cars. TR News, 292, 25-30.
De Winter, J. C., Happee, R., Martens, M. H. and Stanton, N. A. 2014. Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transportation research part F: traffic psychology and behavior, 27, pp. 196-217.
Dixit, V. V., Chand, S. and Nair, D. J. 2016. Autonomous vehicles: disengagements, accidents, and reaction times. PLoS One, 11(12), e0168054.
Dogan, E., Rahal, M. C., Deborne, R., Delhomme, P., Kemeny, A. and Perrin, J. 2017. The transition of control in a partially automated vehicle: effects of anticipation and non-driving-related task involvement. Transportation research part F: traffic psychology and behavior, 46, 205-215.
Dudziak, A., Stoma, M., Kuranc, A. and Caban, J. 2021. Assessment of Social Acceptance for Autonomous Vehicles in Southeastern Poland. Energies, 14(18), pp. 5778.
Endsley, M. R. 1995. Toward a theory of situation awareness in dynamic systems. human factors, 37, 32–64.
Endsley, M. R. 2017. From here to autonomy: lessons learned from human-automation research. Human factors, 59(1), 5-27.
Eriksson, A., Banks, V.A. and Stanton, N.A. 2017. Transition to manual: Comparing simulator with on-road control transitions. Accident Analysis & Prevention, 102, p. 227-234.
Fagnant, D., and Kockelman, K. 2015. Preparing a nation for autonomous vehicles: Opportunities, barriers and policy implications. Transportation Research Part A, 77, pp. 167-81.
Favarò, F. M., Nader, N., Eurich, S. O., Tripp, M. and Varadaraju, N. 2017. Examining accident reports involving autonomous vehicles in California. PLoS One, 12(9), e0184952.
Fox, J. and Weisberg, S. 2019. An {R} Companion to Applied Regression, Third Edition. Thousand Oaks CA: Sage. Available at: https://socialsciences.mcmaster.ca/jfox/Books/Companion/
Gessner, D. 2020, July 22. Experts say we're decades from fully autonomous cars. Here's why. [Online]. Available at: https://www.businessinsider.com/self-driving-cars-fully-autonomous-vehicles-future-prediction-timeline-2019-8 [Accessed: 6. 12. 2020].
Gold, C., Damböck, D., Lorenz, L. and Bengler, K. 2013. “Take over!” How long does it take to get the driver back into the loop?. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 57, No. 1, pp. 1938-1942). Sage CA: Los Angeles, CA: Sage Publications.
Gugerty, L. 2011. Situation awareness in driving. Handbook for driving simulation in engineering, medicine, and psychology, 1.
Guthrie, S (2020, December 4). Elon Musk 'extremely confident' Tesla will launch full-autonomous driving in 2021. [Online]. Available at: https://www.caradvice.com.au/906276/elon-musk-extremely-confident-tesla-will-launch-full-autonomous-driving-in-2021/ [Accessed: 20. 1. 2021].
Hoffmann, J., Kiesel, A. and Sebald, A. 2003. Task switches undergo/no go conditions and the decomposition of switch costs. European Journal of Cognitive Psychology, 15, 101-128. <http://doi.org/10.l080/09541440244000003>
Hollnagel, E. & Woods, D. D. 2005. Joint cognitive systems: Foundations of cognitive systems engineering. Boca Raton, FL: Taylor and Francis.
Jamson, A. H., Merat, N., Carsten, O. M. and Lai, F. C. 2013. Behavioral changes in drivers experiencing highly-automated vehicle control in varying traffic conditions. Transportation research part C: emerging technologies, 30, 116-125.
Kaber, D.B. and Endsley, M.R. 2004. The effects of level of automation and adaptive automation on human performance, situation awareness, and workload in a dynamic control task. Theoretical Issues in Ergonomic Science, 5(2), pp. 113‐153.
Kácovský, A. (2020, November 16). Waymo: Naše technologie automa-tického řízení je pokročilejší než Tesly. [Online]. Available at: https://elektrickevozy.cz/clanky/waymo-nase-technologie-automatickeho-rizeni-je-pokrocilejsi-nez-tesly [Accessed: 20. 1. 2021].
Karner, T. and Neuwirth, W. 2000. Validation of traffic psychology tests by comparing with actual driving. In International Conference on Traffic and Transport Psychology, 4-7 September. Berne Switzerland
Kass, S.J., Cole, K.S. and Stanny, C.J. 2007. Effects of distraction and experience on situation awareness and simulated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 10, pp. 321–329.
Kassambara, A. 2020. rstatix: Pipe-Friendly Framework for Basic Statistical Tests. R package version 0.6.0. [Online]. Available at: https://CRAN.R-project.org/package=rstatix [Accessed: 20. 1. 2021].
Kelly, T. 2020, November 11. Honda says will be the first to mass-produce level 3 autonomous cars. [Online]. Available at: https://www.reuters.com/article/honda-autonomous-level3-idUSKBN27R0M7 [Accessed: 6. 12. 2020].
Kockelman, K., Avery, P., Bansal, P., Boyles, S. D., Bujanovic, P., Choudhary, T., ... and Hutchinson, R. 2016. Implications of connected and automated vehicles on the safety and operations of roadway networks: a final report (No. FHWA/TX-16/0-6849-1).
Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M. and Koch, I. 2010. Control and interference in task switching—A re-view. Psychological bulletin, 136(5), 849.
Llaneras, R. E., Salinger, J. and Green, C. A. 2013. Human factors issues associated with limited ability autonomous driving systems: Drivers’ allocation of visual attention to the forward roadway.
Lee, J. and See, K. 2004. Trust in automation: designing for appropriate reliance. Human Factors, 46, pp. 50-80.
Litman, T. 2020. Autonomous Vehicle Implementation Predictions Implications for Transport Planning. Victoria Transport Policy Institute.
Louw, T., Merat, N. and Jamson, H. 2015. Engaging with Highly Automated Driving: To be or Not to be in the Loop?.
Maurer, M., Christian Gerdes, J., Lenz, B. and Winner, H. 2016. Autonomous driving: technical, legal and social aspects. Springer Nature.
Marshall, A. 2019, February 19. Elon Musk Promises a Really Truly Self-Driving Tesla in 2020. [Online]. Available at: https://www.wired.com/story/elon-musk-tesla-full-self-driving-2019-2020-promise/ [Accessed: 6. 12. 2020].
Merat, N., Seppelt, B., Louw, T., Engström, J., Lee, J. D., Johansson, E., ... and McGehee, D. 2019. The “out-of-the-loop” concept in automated driving: Proposed definition, measures, and implications. Cognition, Technology & Work, 21(1), pp. 87-98.
Merat, N., Jamson, A. H., Lai, F. C., Daly, M. and Carsten, O. M. 2014. Transition to manual: Driver behavior when resuming control from a highly automated vehicle. Transportation research part F: traffic psychology and behavior, 27, pp. 274-282.
Merat, N., Jamson, A. H., Lai, F. C. and Carsten, O. 2012. Highly automated driving, secondary task performance, and driver state. Human factors, 54(5), pp. 762-771.
Meiran, N., Chorev, Z. and Sapir, A. 2000. Component processes in task switching. Cognitive Psychology, 41, pp. 211-253. <http://doi.org/10.10061 cogp.2000.0736>
Monsell, S. 2003. Task switching. Trends in Cognitive Sciences, 7, 134-140. <http://doi.org/10. 10 I/S 1364(03)00028-7>
Monsell, S., Sumner, P. and Waters, H. 2003. Taskset reconfiguration with predictable and unpredictable task switches. Memory & Cognition, 31, 327-342.
National Highway Traffic Safety Administration (NHTSA). 2013. Preliminary statement of policy concerning automated vehicles. National Highway Traffic Safety Administration, Washington DC, National Highway Traffic Safety Administration.
Ogle, D.H., P. Wheeler, P. and Dinno, A. 2021. FSA: Fisheries Stock Analysis. R package version 0.8.32. [Online]. Available at: https://github.com/droglenc/FSA. [Accessed: 6. 2. 2021].
Parasuraman, R. and Riley, V. 1997. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2), pp. 230-253.
Parasuraman, R., Sheridan, T. B. and Wickens, C. D. 2008. Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of cognitive engineering and decision making, 2(2), pp. 140-160.
Reichardt, D. M. 1998. Using automated assistance systems-Putting the driver into focus. In IEEE Intelligent Vehicles (IV) Symposium (pp. 413-418).
Radlmayr, J., Gold, C., Lorenz, L., Farid, M. and Bengler, K. 2014. How traffic situations and non-driving-related tasks affect the take-over quality in highly automated driving. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 58, No. 1, pp. 2063-2067). Sage CA: Los Angeles, CA: Sage Publications.
Ratcliff, R. 1993. Methods for dealing with reaction time outliers. Psychological bulletin, 114(3), pp. 510.
R Core Team. 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. [Online]. Available at: https://www.r-project.org [Accessed: 20. 1. 2021].
Rogers, R. D. and Monsell, S. 1995. Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124, pp. 207-231.
Rogers, M., Zhang, Y., Kaber, D., Liang, Y. and Gangakhedkar, S. 2011. The effects of visual and cognitive distraction on driver situation awareness. In International Conference on Engineering Psychology and Cognitive Ergonomics (pp. 186-195). Springer, Berlin, Heidelberg.
Ruthruff, E., Remington, R. W. and Johnston, J. C. 2001. Switching between simple cognitive tasks: The interaction of top-down and bottom-up factors. Journal of Experimental Psychology: Human Perception and Performance, 27, 1404-1419. <http://doi.org/10.1037110096-1523.27.6.1404>
SAE J3016. 2014. Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems. Society of Automotive Engineers.
Saffarian, M., de Winter, J. and Happee, R. 2012. Automated Driving: Human-Factors Issues and Design Solutions. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), pp. 2296-2300.
Salmon, P., Stanton, N., Walker, G. and Green, D. 2006. Situation awareness measurement: A review of applicability for C4i environments. Journal of Applied Ergonomics, 37(2), pp. 225-238.
Salmon, P. M., Stanton, N. A., Walker, G. H., Jenkins, D., Ladva, D., Rafferty, L. and Young, M. 2009. Measuring Situation Awareness in complex systems: Comparison of measures study. International Journal of Industrial Ergonomics, 39(3), pp. 490-500.
Salmon, P. M., Stanton, N. A., Walker, G. H., and Jenkins, D. P. 2009. Distributed Situation Awareness: Advances in Theory, Measurement, and Application to Teamwork. Aldershot, UK: Ashgate.
Schoettle, B. and Sivak, M. 2014. Public opinion about self-driving vehicles in China, India, Japan, the US, the UK, and Australia. University of Michigan, Ann Arbor, Transportation Research Institute.
Schuhfried, G. 2011. Adaptive Tachstoscopic Traffic Perception Test: Manual. Mödling: Schuhfried GmbH
Schuhfried, G., Sommer,M., Aniţei, M. and Chraif, M. 2010. The validation methodology of a psychological assessment battery for drivers’ schools in Romania. Romanian Journal of Applied Experimental Psychology, 1(1), pp. 5-14.
Sommer, M., Herle, M., Häusler, J., Risser, R., Schützhofer, B. and Chaloupka, C. 2008. Cognitive and personality determinants of fitness to drive. Transportation research part F: traffic psychology and behaviour, 11(5), pp. 362-375.
Sommer, M., Schuhfried, G., Aniţei, M. and Chraif, M. 2010. A model of driving test battery validation. Journal of Psychology, 56, (3–4), pp. 243–250.
Sommer, M., Häusler, J. and Schuhfried, G. 2017. Non-linear methods for the identification of drivers at risk to cause accidents. Driver Behaviour and Training,2 (2), pp. 425.
Stanton, N. A. and Young, M. S. 2005. Driver behavior with adaptive cruise control. Ergonomics, 48(10), pp. 1294-1313.
Stoma, M., Dudziak, A., Caban, J. and Droździel, P. 2021. The Future of Autonomous Vehicles in the Opinion of Automotive Market Users. Energies, 14(16), pp. 4777.
Strand, N., Nilsson, J., Karlsson, I. M. and Nilsson, L. 2014. Semi-automated versus highly automated driving in critical situations caused by automation failures. Transportation research part F: traffic psychology and behavior, 27, pp. 218-228.
Straka, J. and Pelešková, J. 2020. Ročenka nehodovosti na pozemních komunikacích za rok 2019. Praha: Ředitelství služby dopravní policie Policejního prezidia České republiky.
Vahidi, A. and Eskandarian, A. 2003. Research advances in intelligent collision avoidance and adaptive cruise control. IEEE transactions on intelligent transportation systems, 4(3), pp. 143-153.
Walch, M., Lange, K., Baumann, M. and Weber, M. 2015. Autonomous driving: investigating the feasibility of car-driver handover assistance. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 11-18). ACM.
de Waard, D., van der Hulst, M., Hoedemaeker, M. and Brookhuis, K. A. 1999. Driver behavior in an emergency situation in the Automated Highway System. Transportation human factors, 1(1), pp. 67-82.
Wickham, H. and Bryan, J. 2019. readxl: Read Excel Files. R package version 1.3.1. [Online]. Available at: https://CRAN.R-project.org/package=readxl [Accessed: 5. 12. 2020].
Wojcik, R. (2019, November 21). How Self-Driving Cars Will Change Digital Marketing Forever. [Online]. Available at: https://www.digitalcurrent.com/blog/what-self-driving-cars-mean-for-the-future-of-digital-marketing/ [Accessed: 5. 12. 2020].
Young, K. L., Salmon, P. M. and Cornelissen, M. 2013. Missing links? The effects of distraction on driver situation awareness. Safety Science, 56, 36–43.
Young, M. S. and Stanton, N. A. 2007. What’s skill got to do with it? Vehicle automation and driver mental workload. Ergonomics, 50(8), 1324––1339.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Miroslava Horáková, Adam Gregorovič
This work is licensed under a Creative Commons Attribution 4.0 International License.
Accepted 2022-06-23
Published 2022-06-30
