ISO INTERNATIONAL STANDARD 39003 First edition 2023-07 Road traffic safety (RTS) Guidance on ethical considerations relating to safety for autonomous vehicles Sécurité routiere Recommandations relatives aux considérations éthiques en matiere de sécurité pour les vehicules autonomes Reference number IS0 39003:2023(E) ISO @IS02023 IS0 39003:2023(E) COPYRIGHT PROTECTED DOCUMENT IS02023 All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may the internet or an intranet, withoutpriorwrittenpermission.Permission can be requested from eitherIso at the addressbelow or Iso's member body in the country of the requester. ISO copyright office CP 401 · Ch. de Blandonnet 8 CH-1214Vernier,Geneva Phone: +41 22 749 01 11 Email: [email protected] Website: www.iso.org PublishedinSwitzerland i @ IS0 2023 - All rights reserved IS0 39003:2023(E) Contents Page Foreword .V Introduction. .vi 1 Scope. 1 2 Normative references .1 3 Terms and definitions 1 4 External factors affecting autonomous vehicle safety .2 4.1 General 4.2 The road environment .2 5 Interested parties in AV design and operations .3 5.1 General .3 5.2 Producers - Manufacturers, designers and their suppliers .3 5.3 Distribution chain - Distributors, sellers. 3 5.4 Purchasers, owners and operators 4 5.5 Government agencies and other interested parties 4 6 Governance, assessment and evaluation 4 6.1 General 4 6.2 Ethical reference for assessment .4 6.3 Additional standards. .4 6.4 General .5 6.4.1 Higher organizational level 5 6.4.2 Development organizational level 5 6.4.3 Specific development and implementation processes 6 6.4.4 Post implementation checking against ethical criteria 6.5 Conducting the assessment. 7 6.6 Expression of results and conclusion .8 7 Operationalization of ethics - discussion on values and ethics to consider .8 7.1 General. .8 7.2 Ethical framework for the design of AVs (driving action policies and ethical design) 8 7.2.1 Purpose. 9 7.2.2 Values 10 7.2.3 Principles 10 7.2.4 Methods for construction and evaluation of maxims 13 7.3 Background of maxims 13 7.3.1 Maxim design and construction 13 7.3.2 Evaluations of maxims.. 14 7.4 Driving action policies 15 7.4.1 Need versus desire driving action policy 16 7.4.2 Once on the road space 16 7.4.3 In the lane behaviour (includes braking) 17 7.4.4 Lane switching 18 7.4.5 In the presence of the other 19 7.4.6 Road/infrastructure use cases 19 7.4.7 Resoiving conflict. 20 7.4.8 Negotiations 21 7.4.9 AV unable to function as intended 21 7.4.10 Yielding to first responders and emergency response vehicles 22 7.4.11 Protecting other road users 22 7.4.12 Unavoidable collision with other road users. 23 7.4.13 Otherissues 24 8 Framework for rule construction and dealing with violations and deviations .24 8.1 General 24 @ IS0 2023 - All rights reserved ii IS0 39003:2023(E) 8.2 Framework 24 8.3 Goals .25 8.4 Primary rules. 25 8.5 Supportingrules .26 8.6 Precautionary and disabling rules (prevention) .26 8.7 Reinforcing and enabling rules (performance) .26 8'8 Counter rules. 26 8.8.1 Exceptions(preventionandperformance 26 8.8.2 Discretionary and compensatory rules. .27 8.8.3 Misconduct. .27 8.8.4 Violation .27 8.8.5 Deviations. .27 8.8.6 Breakdowns 28 8.9 Rule strategy. 28 8.10 Boundaries (for prevention of unwanted behaviour) 28 8.10.1 Margins 28 8.10.2 Barriers 28 8.10.3 Buffers. 28 8.11 Promoters (for performance) .29 8.12 Further considerations 29 9 External/internal design ..29 10 Sustainability .30 11 Review and re-evaluation following controls system updates ..31 Annex A (informative) Overview of ethical philosophy related to Av ..32 AnnexB(informative)Sustainabilityissues ..34 Annex C (informative) Responsibility and accountability in the context of AVs ..35 Annex D (informative) Action plan - Example ..41 Bibliography .43 iv IS02023-Allrightsreserved IS0 39003:2023(E) Foreword IsO (the International Organization for Standardization) is a worldwi