Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
124 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Journey to Trustworthy AI: Pursuit of Pragmatic Frameworks (2403.15457v3)

Published 19 Mar 2024 in cs.CY, cs.AI, and cs.HC

Abstract: This paper reviews Trustworthy Artificial Intelligence (TAI) and its various definitions. Considering the principles respected in any society, TAI is often characterized by a few attributes, some of which have led to confusion in regulatory or engineering contexts. We argue against using terms such as Responsible or Ethical AI as substitutes for TAI. And to help clarify any confusion, we suggest leaving them behind. Given the subjectivity and complexity inherent in TAI, developing a universal framework is deemed infeasible. Instead, we advocate for approaches centered on addressing key attributes and properties such as fairness, bias, risk, security, explainability, and reliability. We examine the ongoing regulatory landscape, with a focus on initiatives in the EU, China, and the USA. We recognize that differences in AI regulations based on geopolitical and geographical reasons pose an additional challenge for multinational companies. We identify risk as a core factor in AI regulation and TAI. For example, as outlined in the EU-AI Act, organizations must gauge the risk level of their AI products to act accordingly (or risk hefty fines). We compare modalities of TAI implementation and how multiple cross-functional teams are engaged in the overall process. Thus, a brute force approach for enacting TAI renders its efficiency and agility, moot. To address this, we introduce our framework Set-Formalize-Measure-Act (SFMA). Our solution highlights the importance of transforming TAI-aware metrics, drivers of TAI, stakeholders, and business/legal requirements into actual benchmarks or tests. Finally, over-regulation driven by panic of powerful AI models can, in fact, harm TAI too. Based on GitHub user-activity data, in 2023, AI open-source projects rose to top projects by contributor account. Enabling innovation in TAI hinges on the independent contributions of the open-source community.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (135)
  1. Civil Rights Act of 1964, 1964. Pub. L. No. 88-352, 78 Stat. 241 (1964).
  2. ACM News. Japan goes all in: Copyright doesn’t apply to ai training. https://cacm.acm.org/news/273479-japan-goes-all-in-copyright-doesnt-apply-to-ai-training/fulltext. Accessed on February 6, 2024.
  3. Assuring the machine learning lifecycle: Desiderata, methods, and challenges. ACM Computing Surveys (CSUR), 54(5):1–39, 2021.
  4. Real-world effectiveness of model year 2015–2020 advanced driver assistance systems. In 27th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration, number 23-0170, 2023.
  5. A systematic literature review of user trust in ai-enabled systems: An hci perspective. International Journal of Human–Computer Interaction, pages 1–16, 2022.
  6. Big Data’s Disparate Impact. California law review, pages 671–732, 2016.
  7. James Barrat. Our final invention: Artificial intelligence and the end of the human era. Hachette UK, 2023.
  8. BBC News. Home Office drops ‘racist’ algorithm from visa decisions. https://www.bbc.com/news/technology-53650758, August 2020. Accessed on January 30, 2024.
  9. David Beer. The social power of algorithms. In The Social Power of Algorithms, pages 1–13. Routledge, 2019.
  10. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849–15854, 2019.
  11. President Joe Biden. Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. The White House, 2023. URL: https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/.
  12. J Mark Bishop. Artificial intelligence is stupid and causal reasoning will not fix it. Frontiers in Psychology, 11:2603, 2021.
  13. A cellular platform for the development of synthetic living machines. Science Robotics, 6(52):eabf1571, 2021.
  14. Bloomberg. OpenAI’s Altman sees UAE as world’s AI regulatory testing ground. https://www.bloomberg.com/news/articles/2024-02-13/openai-s-altman-says-uae-could-be-an-ai-sandbox-for-the-world, February 2024. Accessed on 22nd February 2024.
  15. Enterprise risk management: Review, critique, and research directions. Long range planning, 48(4):265–276, 2015.
  16. Current and near-term ai as a potential existential risk factor. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 119–129, 2022.
  17. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018.
  18. Governance from the bottom up: Complexity and divergence in comparative perspective. Public Administration, 90(1):56–73, 2012.
  19. Joseph Carlsmith. Is power-seeking ai an existential risk? arXiv preprint arXiv:2206.13353, 2022.
  20. Balancing the tradeoff between regulation and innovation for artificial intelligence: An analysis of top-down command and control and bottom-up self-regulatory approaches. Available at SSRN, 2022. doi: 10.2139/ssrn.4223016.
  21. China Law Translate. Interim measures for the management of generative artificial intelligence services (translation). https://www.chinalawtranslate.com/en/generative-ai-interim/, 2023. Accessed on 2024-02-07.
  22. A survey on trust modeling. ACM Computing Surveys (CSUR), 48(2):1–40, 2015.
  23. Jessica B Cicchino. Real-world effects of general motors forward collision alert and front automatic braking systems. Arlington, VA: Insurance Institute for Highway Safety, 2018.
  24. The scored society: Due process for automated predictions. Wash. L. Rev., 89:1, 2014.
  25. A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning. Advances in Neural Information Processing Systems, 36, 2024.
  26. Predicting audit failure: Evidence from auditing enforcement releases. Spanish Journal of Finance and Accounting/Revista Española de Financiación y Contabilidad, 48(3):274–305, 2019.
  27. Aleatory or epistemic? does it matter? Structural safety, 31(2):105–112, 2009.
  28. Sea change in software development: Economic and productivity analysis of the ai-powered developer lifecycle. arXiv preprint arXiv:2306.15033, 2023.
  29. The accuracy, fairness, and limits of predicting recidivism. Science advances, 4(1):eaao5580, 2018.
  30. ETO. Exploring trustworthy AI research with the Map of Science, part 3: Leading research institutions. https://eto.tech/blog/exploring-trustworthy-ai-research-part-three/, 2023. Accessed: February 9, 2024.
  31. European Parliament Press. Artificial intelligence act: Deal on comprehensive rules for trustworthy ai. https://www.europarl.europa.eu/news/en/press-room/20231206IPR15699/artificial-intelligence-act-deal-on-comprehensive-rules-for-trustworthy-ai, December 2023. Accessed on March 3, 2024.
  32. Threats by artificial intelligence to human health and human existence. BMJ global health, 8(5), 2023.
  33. Richard W Feldmann. The cardano-tartaglia dispute. The Mathematics Teacher, 54(3):160–163, 1961.
  34. Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns. Big Data & Society, 6(1):2053951719860542, 2019.
  35. Effectiveness of low speed autonomous emergency braking in real-world rear-end crashes. Accident Analysis & Prevention, 81:24–29, 2015.
  36. Ragnar Fjelland. Why general artificial intelligence will not be realized. Humanities and Social Sciences Communications, 7(1):1–9, 2020.
  37. Jane E Fountain. The moon, the ghetto and artificial intelligence: Reducing systemic racism in computational algorithms. Government Information Quarterly, 39(2):101645, 2022.
  38. Foxglove. Home Office says it will abandon its racist visa algorithm after we sued them. https://www.foxglove.org.uk/2020/08/04/home-office-says-it-will-abandon-its-racist-visa-algorithm-after-we-sued-them/, 2020. Accessed on December 18, 2023.
  39. Ori Freiman. Making sense of the conceptual nonsense ‘trustworthy ai’. AI and Ethics, 3(4):1351–1360, 2023.
  40. The 4th industrial revolution powered by the integration of ai, blockchain, and 5g. Communications of the Association for Information Systems, 49(1):6, 2021.
  41. Re-engineering humanity. Cambridge University Press, 2018.
  42. CM. Froehle. The Evolution of an Accidental Meme. https://medium.com/@CRA1G/the-evolution-of-an-accidental-meme-ddc4e139e0e4, 2016. Accessed on December 18, 2023.
  43. Vassilis Galanos. Exploring expanding expertise: artificial intelligence as an existential threat and the role of prestigious commentators, 2014–2018. Technology Analysis & Strategic Management, 31(4):421–432, 2019.
  44. Diego Gambetta et al. Can we trust trust. Trust: Making and breaking cooperative relations, 13(2000):213–237, 2000.
  45. A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342, 2021.
  46. GitHub Octoverse. The state of open source and ai. https://github.blog/2023-11-08-the-state-of-open-source-and-ai/, November 2023. Accessed: February 25, 2024.
  47. GNU Project. What is Free Software? https://www.gnu.org/philosophy/free-sw.html, 2024. Accessed: February 9, 2024.
  48. Government of the United Kingdom. The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023. https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration/the-bletchley-declaration-by-countries-attending-the-ai-safety-summit-1-2-november-2023, 2023. Accessed on January 30, 2024.
  49. Mark Granovetter. Economic action and social structure: The problem of embeddedness. In The sociology of economic life, pages 22–45. Routledge, 2018.
  50. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pages 79–90, 2023.
  51. Artificial intelligence and decision-making: The question of accountability. 2021.
  52. David Gunning. Explainable Artificial Intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), nd Web, 2(2):1, 2017.
  53. Thilo Hagendorff. Blind spots in ai ethics. AI and Ethics, 2(4):851–867, 2022.
  54. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
  55. Holistic AI. How is Brazil Leading South America’s AI Legislation Efforts? https://www.holisticai.com/blog/brazil-ai-legislation-proposals, November 2023. Accessed: February 9, 2024.
  56. Multilevel governance: Framing the integration of top-down and bottom-up policymaking. International Journal of Public Administration, 42(7):572–582, 2019.
  57. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110:457–506, 2021.
  58. 50 years of test (un) fairness: Lessons for machine learning. In Proceedings of the conference on fairness, accountability, and transparency, pages 49–58, 2019.
  59. IEEE. Ieee releases ethically aligned design, version 2 to show “ethics in action” for the development of autonomous and intelligent systems (a/is). https://standards.ieee.org/news/ead_v2/, Accessed on 29th January 2024a.
  60. IEEE. Ieee vision & mission. https://www.ieee.org/about/vision-mission.html, Accessed on 29th January 2024b.
  61. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically aligned design: A vision for prioritizing human well-being with autonomous and intelligent systems, version 2. IEEE, 2017. URL http://standards.ieee.org/develop/indconn/ec/autonomous_systems.html.
  62. IEEE Spectrum. How NASA Designed a Helicopter That Could Fly Autonomously on Mars. https://spectrum.ieee.org/nasa-designed-perseverance-helicopter-rover-fly-autonomously-mars, 2021. Accessed: February 9, 2024.
  63. Deliberation for autonomous robots: A survey. Artificial Intelligence, 247:10–44, 2017.
  64. C Isidore. Machines are driving wall street’s wild ride, not humans. CNN Business, CNN, Feb, 6, 2018.
  65. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in ai. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 624–635, 2021.
  66. Harvey S James Jr. The trust paradox: a survey of economic inquiries into the nature of trust and trustworthiness. Journal of Economic Behavior & Organization, 47(3):291–307, 2002.
  67. The road towards 6g: A comprehensive survey. IEEE Open Journal of the Communications Society, 2:334–366, 2021.
  68. The global landscape of ai ethics guidelines. Nature machine intelligence, 1(9):389–399, 2019.
  69. Katie Collins–CNET. UK Gov agrees to redesign ‘racist’ algorithm that decides visa applications. https://www.cnet.com/tech/services-and-software/uk-gov-agrees-to-redesign-racist-algorithm-that-decides-visa-applications/, 2020. Accessed on December 18, 2023.
  70. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International conference on machine learning, pages 2564–2572. PMLR, 2018.
  71. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807, 2016.
  72. A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. Artificial Intelligence, 295:103458, 2021.
  73. Algorithmic bias: review, synthesis, and future research directions. European Journal of Information Systems, 31(3):388–409, 2022.
  74. From github to gdp: A framework for measuring open source software innovation. Research Policy, 53(3):104954, 2024.
  75. Machine learning operations (mlops): Overview, definition, and architecture. IEEE Access, 2023.
  76. Kinematic self-replication in reconfigurable organisms. Proceedings of the National Academy of Sciences, 118(49):e2112672118, 2021.
  77. Ethics from within: Google glass, the collingridge dilemma, and the mediated value of privacy. Science, Technology, & Human Values, 44(2):291–314, 2019.
  78. Artificial intelligence for long-term robot autonomy: A survey. IEEE Robotics and Automation Letters, 3(4):4023–4030, 2018.
  79. Technology, humanness, and trust: Rethinking trust in technology. Journal of the Association for Information Systems, 16(10):1, 2015.
  80. Trustworthy artificial intelligence and the european union ai act: On the conflation of trustworthiness and acceptability of risk. Regulation & Governance, 18(1):3–32, 2024.
  81. Yann LeCun. Meta AI Chief Yann LeCun Skeptical About AGI, Quantum Computing. https://www.cnbc.com/2023/12/03/meta-ai-chief-yann-lecun-skeptical-about-agi-quantum-computing.html, 2023. Accessed on December 4, 2023.
  82. David Levi-Faur. The Oxford handbook of governance. Oxford University Press, 2012.
  83. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860, 2023.
  84. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1–35, 2021.
  85. Albert J Menkveld. The economics of high-frequency trading: Taking stock. Annual Review of Financial Economics, 8:1–24, 2016.
  86. Transparency and trust in human-ai-interaction: The role of model-agnostic explanations in computer vision-based decision support. In Artificial Intelligence in HCI: First International Conference, AI-HCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings 22, pages 54–69. Springer, 2020.
  87. Devops and software quality: A systematic mapping. Computer Science Review, 38:100308, 2020.
  88. The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2):2053951716679679, 2016.
  89. Cognitive biases in software engineering: a systematic mapping study. IEEE Transactions on Software Engineering, 46(12):1318–1339, 2018.
  90. NASA. NASA Mars Helicopter. https://mars.nasa.gov/technology/helicopter/overview/#Five-Things, 2024. Accessed: February 9, 2024.
  91. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). https://doi.org/10.6028/NIST.AI.100-1, 2022. DOI: 10.6028/NIST.AI.100-1.
  92. Jon Michael Raasch-Fox News. AI gives birth to AI: Scientists say machine intelligence now capable of replicating without humans, December 2023. URL https://www.foxnews.com/science/ai-gives-birth-ai-scientists-say-machine-intelligence-capable-replicating-without-humans. Accessed on January 5, 2024.
  93. Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ecg analysis. Circulation: Arrhythmia and Electrophysiology, 13(3):e007988, 2020.
  94. OECD. The state of implementation of the OECD AI principles four years on. (3), 2023. doi: https://doi.org/https://doi.org/10.1787/835641c9-en. URL https://www.oecd-ilibrary.org/content/paper/835641c9-en.
  95. Official Website of the National Institute of Standards and Technology. About NIST. https://www.nist.gov/about-nist. Accessed on January 22, 2024.
  96. I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pages 1–13, 2018.
  97. Brad C Pardue. Printing, power, and piety: appeals to the public during the early years of the English Reformation, volume 162. Brill, 2012.
  98. Scott Patterson. Dark pools: The rise of the machine traders and the rigging of the US stock market. Currency, 2013.
  99. The fourth industrial revolution. Journal of International Affairs, 72(1):17–22, 2018.
  100. Review of artificial intelligence adversarial attack and defense technologies. Applied Sciences, 9(5):909, 2019.
  101. Guardian: A multi-tiered defense architecture for thwarting prompt injection attacks on llms. Journal of Software Engineering and Applications, 17(1):43–68, 2024.
  102. Reuters. Governments race to regulate AI tools. https://www.reuters.com/technology/governments-race-regulate-ai-tools-2023-10-13/, October 2023. Accessed on 14th February 2024.
  103. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  104. Randy Robertson. Censorship and Conflict in Seventeenth-Century England: The Subtle Art of Division. Penn State Press, 2015.
  105. Julian B Rotter. Interpersonal trust, trustworthiness, and gullibility. American psychologist, 35(1):1, 1980.
  106. The age of secrecy and unfairness in recidivism prediction. Harvard Data Science Review, 2(1):1, 2020.
  107. How do fairness definitions fare? examining public attitudes towards algorithmic definitions of fairness. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 99–106, 2019.
  108. Foundation models: A new paradigm for artificial intelligence. Business & Information Systems Engineering, pages 1–11, 2024.
  109. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE access, 5:3909–3943, 2017.
  110. Matt Sheehan. China’s ai regulations and how they get made. Carnegie Endowment for International Piece. Accessed August, 2:2003, 2023.
  111. Mathematical notions vs. human perception of fairness: A descriptive approach to fairness for machine learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2459–2468, 2019.
  112. Charlotte Stix. Artificial intelligence by any other name: a brief history of the conceptualization of “trustworthy artificial intelligence”. Discover Artificial Intelligence, 2(1):26, 2022.
  113. The top500 supercomputers project. https://www.top500.org/lists/top500/2023/11/, 11 2023. Accessed: February 25, 2024.
  114. Supreme Court of the State of Arizona. Varela v. FCA US LLC, ET AL., 2022. URL {https://www.azcourts.gov/Portals/0/OpinionFiles/Supreme/2022/CV200157PR.pdf}. No. CV-20-0157-PR.
  115. Mlops-definitions, tools and challenges. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC), pages 0453–0460. IEEE, 2022.
  116. Artificial intelligence in human resources management: Challenges and a path forward. California Management Review, 61(4):15–42, 2019.
  117. Telefonaktiebolaget LM Ericsson. Follow the journey to 6G. https://www.ericsson.com/en/6g. Accessed on March 2, 2024.
  118. Mlops: A taxonomy and a methodology. IEEE Access, 10:63606–63618, 2022.
  119. The Editors of Encyclopaedia Britannica. Scipione ferro, 2023. URL https://www.britannica.com/biography/Scipione-Ferro. Accessed 9 December 2023.
  120. Trustworthy artificial intelligence. Electronic Markets, 31:447–464, 2021.
  121. Automating judicial discretion: How algorithmic risk assessments in pretrial adjudications violate equal protection rights on the basis of race. Law & Ineq., 40:371, 2022.
  122. Who cares about trust? clusters of research on trustworthy ai. Technical report, Center for Security and Emerging Technology, 2023. URL https://doi.org/10.51593/20230014b.
  123. Pier Domenico Tortola. Clarifying multilevel governance. European Journal of Political Research, 56(2):234–250, 2017.
  124. A sandbox approach to regulating high-risk artificial intelligence applications. European Journal of Risk Regulation, 13(2):270–294, 2022.
  125. UK Government. A Pro-innovation Approach to AI Regulation. https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach/white-paper#executive-summary. Accessed on February 6, 2024.
  126. UN Human Rights Council. UN Resolution- HRC/RES/47/16 the promotion, protection and enjoyment of human rights on the internet: resolution, July 2021. URL https://undocs.org/A_HRC_RES_47_16-EN. Accessed on: March 8, 2024.
  127. UNESCO. Recommendation on the ethics of artificial intelligence, 2021. URL https://unesdoc.unesco.org/ark:/48223/pf0000380455.locale=en. Accessed on 2024-01-29.
  128. USA Today. Equifax had patch 2 months before hack and didn’t install it, security group says. https://www.usatoday.com/story/money/2017/09/14/equifax-identity-theft-hackers-apache-struts/665100001/. Accessed on: March 5, 2024.
  129. Warning, bias may occur: A proposed approach to detecting cognitive bias in interactive visual analytics. In 2017 ieee conference on visual analytics science and technology (vast), pages 104–115. IEEE, 2017.
  130. Mika Westerlund. The emergence of deepfake technology: A review. Technology innovation management review, 9(11), 2019.
  131. Fei Yang and Yu Yao. A new regulatory framework for algorithm-powered recommendation services in china. Nature Machine Intelligence, 4(10):802–803, 2022.
  132. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211, 2024.
  133. Ethical awareness of uxers in the loop: Ethical issues in the uxer-ai collaboration process from a ux perspective. In Proceedings of the 25th International Conference on Mobile Human-Computer Interaction, pages 1–6, 2023.
  134. Recent advances in quantum machine learning. Quantum Engineering, 2(1):e34, 2020.
  135. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. arXiv preprint arXiv:2302.09419, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets