Foundation Model Transparency Reports (2402.16268v1)

Abstract: Foundation models are critical digital technologies with sweeping societal impact that necessitates transparency. To codify how foundation model developers should provide transparency about the development and deployment of their models, we propose Foundation Model Transparency Reports, drawing upon the transparency reporting practices in social media. While external documentation of societal harms prompted social media transparency reports, our objective is to institutionalize transparency reporting for foundation models while the industry is still nascent. To design our reports, we identify 6 design principles given the successes and shortcomings of social media transparency reporting. To further schematize our reports, we draw upon the 100 transparency indicators from the Foundation Model Transparency Index. Given these indicators, we measure the extent to which they overlap with the transparency requirements included in six prominent government policies (e.g., the EU AI Act, the US Executive Order on Safe, Secure, and Trustworthy AI). Well-designed transparency reports could reduce compliance costs, in part due to overlapping regulatory requirements across different jurisdictions. We encourage foundation model developers to regularly publish transparency reports, building upon recommendations from the G7 and the White House.

• The paper introduces a structured framework for Foundation Model Transparency Reports that standardizes and details model disclosures.
• It leverages insights from social media transparency practices to establish six design principles addressing comprehensive reporting and compliance.
• The framework aligns with government policies to reduce compliance costs and build public trust in the responsible development of AI models.

Proposing Foundation Model Transparency Reports: A Structured Approach to Autonomous Transparency in AI Development

Introduction

The domain of AI has witnessed an unprecedented surge in interest and development of foundation models, significantly impacting various aspects of society. Despite their transformative potential, a glaring opacity within the foundation model ecosystem has raised substantial concerns. Addressing this issue head-on, this paper proposes Foundation Model Transparency Reports as a structured method to ensure comprehensive and coherent transparency from the developers of these models.

Reflections on Social Media Transparency Reports

Drawing parallels from the field of social media, where transparency reporting has evolved into a crucial mechanism to address societal harms, the paper intricately analyses the trajectory of these reports. The analysis uncovers valuable insights into the driving forces behind their emergence and evolution, highlighting the role of societal and regulatory pressures in fostering greater transparency. It elaborates on how, despite their benefits, such reports have struggled with standardization, completeness, and the precision of disclosed information, raising concerns about their effectiveness in truly fostering trust and accountability.

Design Principles for Foundation Model Transparency Reports

Navigating through the shortcomings and successes of social media transparency initiatives, the paper identifies six fundamental design principles crucial for the conceptualization of Foundation Model Transparency Reports. These principles emphasize the necessity for a structured, standardized, and methodologically clear reporting schema that is independently specified and comprehensively covers upstream resources, model properties, and downstream impacts of foundation models. The proposed design meticulously addresses the need for centralization, contextualization, and clarity in transparency reporting, aiming for a holistic depiction of foundation model ecosystems.

Aligning with Government Policies and Enhancing Compliance

The endeavor further explores the alignment of proposed transparency indicators with existing and forthcoming government policies across jurisdictions, shedding light on the considerable gap between current regulatory expectations and the detailed transparency facilitated by the proposed reports. By offering a schema that potentially reduces compliance costs and enhances regulatory alignment, the paper posits Foundation Model Transparency Reports as a strategic tool in navigating the complex regulatory landscapes governing AI development and deployment.

A Call for Robust Transparency Norms and Industry Standards

This research not only underscores the immediate need for enhanced transparency within the foundation model ecosystem but also advocates for the establishment of robust industry standards and norms that transcend mere compliance. Through a critical examination of existing practices and a forward-looking approach to transparency reporting, it sets the stage for significant shifts in how foundational models are developed, deployed, and scrutinized in the public domain.

Concluding Remarks

In summarizing, the paper positions Foundation Model Transparency Reports as a pivotal mechanism for institutionalizing transparency within the nascent foundation model industry. By drawing from historical precedents, existing practices, and a comprehensive understanding of the landscape, it charts a path toward a more transparent, accountable, and socially responsive AI future. The proposed framework not only promises to mitigate the risks associated with foundation models but also potentially fosters a culture of openness and trust, laying the groundwork for future developments in the field of generative AI.

The research concludes with a call to action for foundation model developers, urging them to embrace the practice of transparency reporting proactively. It is a clarion call for developers to align with broader societal values and regulatory expectations, ensuring that the advancement of AI technologies does not come at the cost of transparency, accountability, or societal well-being.