Co-Leads
Jennifer Tour Chayes, Mariano-Florentino Cuéllar and Li Fei-Fei co-led this scholarly working group. Institutional affiliations identify contributors; the report does not represent their institutions’ positions.
Lead Writers
Rishi Bommasani and Scott R. Singer served as lead writers.
Senior Advisors
Daniel E. Ho, Percy Liang, Dawn Song, Joseph E. Gonzalez and Jonathan Zittrain served as senior advisors.
Writing and Editing Group
The writing and editing group comprised Ruth E. Appel, Sarah Cen, A. Feder Cooper, Elena Cryst, Lindsey A. Gailmard, Ian Klaus, Meredith M. Lee, Inioluwa Deborah Raji, Anka Reuel, Drew Spence, Alexander Wan, Angelina Wang and Daniel Zhang.
How to cite
The source is the Joint California Policy Working Group on AI Frontier Models report dated June 17, 2025, cited as Bommasani and Singer et al. The working group acknowledges multidisciplinary expert review of its March draft and subsequent public feedback.
Executive Summary
This report proposes an evidence-generating approach to frontier-AI governance: developers disclose meaningful information, independent evaluators check claims, protected insiders can surface concerns, and incident reports reveal deployment harms. Policymakers then adjust obligations and thresholds as evidence develops. It offers principles rather than endorsing a particular bill, estimating a single acceptable risk level or demonstrating that one regulatory package is optimal.
Its eight connected principles are to balance benefits and material risks; use broad evidence beyond already observed harm; consider durable early design choices; align incentives through disclosure and verification; improve transparency where information is missing; protect evaluators and whistleblowers; learn from adverse events using relevant authorities; and scope interventions with adaptable thresholds. The 52-page source contains no figures, tables or displayed equations. All 145 references and the feedback questionnaire remain available.
The evidence, model examples and legal landscape summarized here are those of the June 2025 report. This is a reading edition of that document, not an update on subsequent policy or model capabilities. Labor-market change, environmental impacts and the full range of AI misuse lie outside its primary remit.
Origins of the Report
Governor Gavin Newsom requested the report in September 2024. The co-leads assembled independent academic contributors, sought expert review of a draft, and invited public feedback before the June 2025 final report. This process produces a multidisciplinary policy synthesis; it is not a representative public vote or a controlled evaluation of policy outcomes.
1. Introduction
The report concerns general-purpose foundation models and the most capable frontier subset, rather than every automated decision system. It combines technical studies, historical cases, modeling and policy analysis because a young, fast-moving technology offers incomplete empirical evidence [17]. Its central move is to make policy generate better evidence for later decisions, not merely consume existing studies [16][31].
1.1. Encouraging Innovation and Implementing Safety Guardrails
The authors seek both innovation and safeguards: public policy can support talent, entrepreneurship and beneficial applications while reducing severe risks. They acknowledge disagreement about artificial general intelligence, its timeline and possible harms. A cited 2023 survey of 1,500 Californians found 31% worried, 17% pessimistic, 27% optimistic and 11% excited [71]. These reported categories do not exhaust responses and do not estimate the probability of future harm.
1.2. The Broader Policy Landscape and California’s Potential Impact
California can shape standards through its concentration of developers, talent, customers and regulatory activity. The report situates state action alongside other US states, federal initiatives and international frameworks, emphasizing harmonization to reduce duplicate compliance. Its argument is that credible safety and quality standards can support adoption and competitiveness. It does not quantify that effect, resolve every state–federal jurisdictional question or establish that every intervention improves competition.
1.3. Foundations
The International AI Safety Report supplies a common technical starting point while documenting expert disagreement [13]. The evidence dilemma is symmetric: premature intervention may be ineffective, while waiting for conclusive evidence may leave little time to mitigate a serious risk. Disclosures, researcher access and incident reports are proposed as complementary ways to improve that decision environment.
1.3.1. Current capabilities
As characterized in the report’s 2025 evidence base, models can assist with programming, multilingual conversation, multimodal tasks and difficult academic questions, but remain unreliable in unfamiliar settings and extended autonomous projects. Benchmark scores may poorly represent real use, and human-augmentation measures lack common standards [137]. Forecasts of future progress disagree; historical increases in training resources do not determine a unique capability trajectory.
1.3.2. Current risks
Risks are grouped into deliberate misuse, unintended malfunction and systemic effects of widespread deployment. Evidence is established for harms such as fraud, non-consensual imagery, bias, unreliable content and privacy violations, while the magnitude and timing of some large-scale biological, cyber and loss-of-control risks remain disputed. The recommended comparison is marginal risk: what does a model enable beyond existing tools and access to information? [69][133]. A model capability alone does not identify the probability or scale of real-world harm.
1.4. Changes Since SB 1047 Veto in September 2024
Between the September 2024 veto and the final report, the authors identify inference-time scaling, more capable agents, changing developer safeguards and new evaluations of deception or reward hacking. They use company system cards and research demonstrations as reasons to improve scrutiny, including internal deployment. For example, the cited virology result is a question-answering benchmark [58], not a direct measurement of successful biological attacks. Controlled scheming tests and developers’ warning thresholds likewise differ from observed population-level incidents. Their relevance is evidence about possible mechanisms and evaluation weaknesses.
1.5. Report Road Map
The argument proceeds from historical evidence to transparency, post-deployment reporting and regulatory scope, followed by feedback and revisions. These mechanisms are meant to work together: disclosure without verification or a response process would leave major accountability gaps.
2. Grounding AI Policy in Evidence and Experience: Understanding the Broader Context
Historical comparisons examine when early choices lock in vulnerabilities, when privileged information is hidden, and when independent assessment changes incentives. Internet governance, consumer products and energy supply the main cases; pesticides, buildings and seat belts supply shorter positive examples. These are selected analogies used to derive principles, not proof that AI follows the same trajectory.
2.1. Evidence-Based Policymaking Beyond Observed Harms
The authors treat evidence as including disciplined predictions, simulations and historical analysis, as well as observed harms. Case studies combine contemporary documents and later scholarship, focusing on critical policy junctures and counterfactual opportunities [52][53][126]. They explicitly reject an inference that AI companies share the motives of tobacco or fossil-fuel companies. The method can suggest governance mechanisms, but differences in incentives, technical controllability and institutional context limit transfer.
2.2. Making Robust Early Design Choices and Acting on Open Policy Windows: Lessons From the Development and Evolving Governance of the Internet
Early internet protocols assumed a more trusted and limited community than the network eventually served. The report uses the Morris worm and later security costs to argue that architecture and governance can become difficult to change [33][144]. Existing courts and agencies still respond when targeted legislation is absent, although this can be reactive and fragmented. The AI implication is to consider interconnected deployments and oversight early; the analogy does not establish that the same technical failures or policy window apply to AI.
2.3. The Need for Public Transparency and Clear Standards: Lessons From Regulating Consumer Products
Consumer-product standards can support confidence and innovation, but tobacco illustrates how concealed evidence and manufactured uncertainty can obstruct informed choices and eventually produce costly litigation [24][85]. The proposed lesson is disclosure plus independent verification, with requirements calibrated to market structure so smaller firms can participate. The claimed reduction in AI developers’ future litigation exposure is a policy rationale, not a measured result or a legal immunity established by this report.
2.4. Contending Interests, Accountability, and Useful Evidence From Simulations: Lessons From Energy Policy
The energy case contrasts internal climate modeling with public communications, emphasizing the usefulness of predictive evidence and the dangers of conflicts of interest [123]. Independent access should balance public accountability with security and commercial confidentiality. The report relates this to AI safety tests, including oversight avoidance, alignment faking and reward hacking [84][55][10]. Such controlled demonstrations motivate investigation; their experimental frequencies are not real-world incident rates. Climate cost estimates illustrate decision-making under uncertainty rather than supplying numerical estimates of AI costs.
2.5. Brief Examples of Positive Outcomes
Three shorter examples illustrate how targeted safeguards and productive activity can coexist. Their favorable outcomes are used as design lessons rather than a claim that regulation always succeeds.
2.5.1. Balancing Innovation and Safety and Carrots and Sticks: Lessons From Pesticide Regulation
Pesticide governance combines safety assessment, reevaluation, enforcement and support for safer practice. The report emphasizes both incentives and penalties, including assistance that helps smaller producers adopt improvements. Its broader lesson concerns proportionality and practical implementation; it does not prescribe importing a pesticide regulatory framework wholesale into AI.
2.5.2. The Importance of Clear Standards: Lessons From Building Codes
Building codes illustrate standards that evolve with engineering knowledge while construction continues. Clear requirements can coordinate expectations about safety. Detailed physical standards are easier to specify and inspect than many frontier-model risks, so this is an example of compatibility between safeguards and activity, not a ready-made AI measurement system.
2.5.3. A False Safety-Innovation Binary: Lessons From Seat Belts
Seat belts illustrate a safety intervention that preserves the central utility of a technology despite initial opposition. The report argues against assuming a fixed safety–innovation trade-off. It also identifies limits: physical products are more geographically bounded, individuals can often reduce their own exposure, and AI may spread faster with societal effects that people cannot individually avoid.
2.6. Learning the Right Lessons for Future Policy
The synthesis favors timely, adaptable governance supported by diverse evidence and independent scrutiny. Historical experience helps frame uncertainty but does not remove it. The report supplies no comparative causal evaluation establishing the net benefit of its proposed package; outcomes depend on implementation, institutional capacity and the evolving technology.
3. Transparency
Transparency should make important information available, understandable and checkable, enabling procurement, policy design, enforcement and public scrutiny. The cited 2024 Foundation Model Transparency Index covers 14 major developers and reports average scores of 34% for training-data transparency, 31% for risk mitigations and 15% for downstream impact [19]. These are disclosure-index scores, not model safety scores or estimates of harm. Transparency is an instrument for accountability, not an end in itself.
3.1. Improving Transparency
The proposed disclosure areas are data acquisition, safety measures, security practices, pre-deployment evaluation and downstream use. For evaluations, independence, payment, access depth, testing time and publication restrictions affect what outside readers can infer. For deployment, hosts and distributors may know usage patterns that model developers cannot observe, particularly for open weights [21].
Policy must choose both the information and its audience: public disclosure permits broad scrutiny, while sensitive details may warrant restricted access. Openness of weights permits some kinds of inspection and correlates with stronger transparency in the cited index, but is not equivalent to full disclosure or guaranteed safety. Release choices create benefits for research and competition alongside misuse risks [69][118].
3.2. Improving Third-Party Risk Assessment
Independent third-party evaluation adds scale, diversity and distance from a developer’s commercial incentives. It differs from first-party testing and developer-paid second-party review. The authors propose protection for good-faith evaluators against contractual or platform penalties, drawing on cybersecurity practice [73][80]. This safe harbor concerns researchers, not blanket immunity for model developers.
Responsible disclosure needs rules for notice, access, publication and routing findings to affected parties. A vulnerability may implicate upstream data, downstream applications and multiple models, so notifying only the original developer is insufficient. Independence alone is not a quality guarantee: standards, access and evaluation science remain immature.
3.3. Protecting Whistleblowers
Whistleblower design must specify protected people, eligible concerns and secure reporting channels. Contractors may need coverage as well as employees, and dangerous departures from a company’s safety policy may deserve protection even without a clear statutory violation [95][141]. The authors call attention to uneven existing protections and procurement-related possibilities. They deliberately leave enforcement mechanisms and interaction with nondisclosure agreements undeveloped, so this is a design agenda rather than a complete operative protection regime.
3.4. Guiding Principles
The authors argue that some transparency improvements are possible without immediate losses elsewhere, while acknowledging real conflicts. Protect personal information, coordinate vulnerability disclosure, restrict access to sensitive commercial material when appropriate, use manageable reporting interfaces and reduce duplication. The report also discusses compelled-speech constraints as part of its 2025 legal analysis [25].
Disclosure must connect to analysis and action to avoid transparency washing [143]. Claims of low cost or improved competition depend on what is disclosed, who can use it and whether follow-up is funded; the report does not provide measured compliance costs or a universal disclosure template.
4. Adverse Event Reporting
Adverse event reporting is proposed as a post-deployment learning mechanism that aggregates harms, incidents and relevant near misses. Design choices are what counts as a reportable event, what information is submitted, who must or may report, and who receives individual or aggregated findings [42]. Confidentiality can encourage reporting and protect sensitive data. Existing domain-specific systems provide examples, but an AI report is a signal for investigation rather than proof that a model caused an event.
4.1. Benefits
The expected benefits are identifying unexpected harms, encouraging preventive action, coordinating responses across organizations and collecting evidence with a potentially lighter burden than continuous direct oversight [127]. These are conditional benefits of a well-run system, not cost-effectiveness estimates established here. Passive reports complement controlled studies and active monitoring; they cannot alone estimate population incidence or causal effects.
4.2. Challenges
Three implementation problems dominate: defining useful events without overwhelming the system, funding staff to investigate reports, and equipping and motivating people to report despite liability or reputational incentives. Even mandatory systems can suffer underreporting. The report cites substantial operating costs in cybersecurity, so its earlier description of comparatively limited cost should not be read as negligible cost. Shared-interest risks and confidential channels may improve reporting, but do not guarantee completeness.
4.3. Guiding Principles
The proposed starting point is a narrow, clear event definition, reviewed and expanded as evidence develops; mandatory developer reports combined with voluntary user reports; routing to agencies with relevant expertise; and use of existing authority where suitable [41][91]. Confidential records and anonymized sharing can support coordinated mitigation. Narrow criteria reduce noise but can miss emerging harm, requiring active reassessment.
The report also offers pre-market testing as a possible basis for exemption from some reporting requirements. This is an illustrative design option, not an adopted rule or evidence that testing eliminates post-deployment risk. Implementing the framework still requires concrete definitions, staffing, enforcement choices and attention to reporting fatigue.
5. Scoping
Scoping determines which entities face which obligations. A threshold compares a measurable quantity with a cutoff, but continuous risk rarely follows such a sharp boundary. The report’s examples of compute and monetary cutoffs describe past or then-current policy approaches, not a recommendation for one numerical limit. Proportional design also needs to distinguish initial developers from downstream modifiers and continually updated models.
5.1. Thresholds for Foundation Models
Four threshold families use developer properties, development costs, model capabilities or downstream impact. The right family depends on the aim: protecting small firms, assessing dangerous capability and monitoring widespread dependence are different objectives. Every candidate must be assessed for when it becomes measurable, how reliably it can be measured and whether outsiders can verify it [15][63].
Compute can be estimated before training, evaluations arrive later, and real-world impact appears after deployment; this timing constrains which obligations each can trigger. Self-designation without verification can undermine coverage. Flexible systems may combine metrics, allow expert alerts and update both values and the underlying measurement approach as technology changes.
5.2. Guiding Principles
Generic company size can miss small influential developers while capturing large firms with modest AI activity. Training compute is comparatively measurable and sometimes externally observable, but is an imperfect proxy for risk: modality, algorithmic efficiency, inference-time computation, distillation, synthetic data and deployment patterns can break a simple relationship [30][66]. The authors recommend against using training compute alone; at most it can serve as an initial screening tool.
Risk evaluations and downstream-impact measures align more directly with some safety goals but have unresolved standards and access problems. Open-model developers may not observe subsequent use. The recommendation is to build measurement and monitoring infrastructure, not to assume a validated alternative threshold is already available.
6. Summary of Feedback and Changes
The March draft received more than 60 feedback submissions. Academia, industry/private sector and civil society each contributed at least 25% of responses. Feedback was summarized without identifying individual respondents. This describes a self-selected consultation, not a representative survey or a quantified expert consensus.
6.1. Clarifications on Report Content
Respondents asked for clearer definitions, a sharper distinction between evidence-based and evidence-generating policy, updated technical evidence, and a more explicit remit. Other comments focused on California’s role, regulatory fragmentation, incentives for safety and international competitiveness.
6.2. Critique of Substantive Content
Substantive critiques questioned the historical analogies, the treatment of voluntary industry safety practices and the lack of policy-failure cases. They asked how disclosures produce accountability, how independent evaluation can work without mature standards, and how reporting definitions and costs would be implemented. Comments agreed thresholds should adapt but disagreed about the value of compute as a proxy. The authors summarize critiques selectively and do not adopt all of them; including a criticism is not an endorsement.
6.3. Future Directions for Further Inquiry
Unresolved directions include application-specific governance, labor effects and education, workforce preparation, public research and compute resources, open-versus-closed model governance and government technical capacity. These remain partly outside the chosen scope. In particular, weak regulatory capacity could make proposed safeguards ineffective or counterproductive, a practical limitation that information disclosure alone cannot solve.
6.4. Changes to the Report
The final report expands discussion of state–federal coordination, new risk evidence, incentives and internal deployment. It strengthens treatment of openness, whistleblowers, protected research and legal constraints; favors initially narrower incident reporting; and more directly criticizes compute-only thresholds. These revisions explain the final framing but do not supply a full implementation plan or new empirical policy evaluation.
Appendix A Feedback Form
The appendix records the feedback form used for the March draft. The respondent fields and all five substantive questions below preserve what contributors were asked; they do not constitute new requests to the reader.
A.1. Preamble Text
The invitation explained the independent scholarly status of the working group and invited feedback by April 8, 2025 for consideration in the expected June final report. It promised not to publish individual responses or attribute feedback to individuals, instead aggregating input.
A.2. Information about Respondent
Original respondent fields:
- Email Address (Required)
- First Name (Optional)
- Last Name (Optional)
- Please check the box if you would like to opt in for receiving email updates. [Checkbox] (Optional)
- Which category best describes you? [Select one] (Optional)
- Academia
- Government/Public Sector
- NGO/Civil Society
- Industry/Business/Private Sector
- Other [free response field provided]
- Title (Optional)
- School/Company/Organization/Agency Name (Optional)
A.3. Form Questions
Original feedback questions:
- The structure of the Draft Report includes sections about Context, Transparency and Third-Party Risk Assessment, Adverse Event Reporting, and Scoping. At a high level, what might you find valuable? What types of questions are you most interested in, and how might you use the report in your work? (Optional)
- From your perspective and experience, what key factors do you see affecting California’s path forward in AI governance? Please feel free to provide specific feedback referring to the sections of the draft report. (Optional)
- Numerous frontier AI governance-focused groups have been working on frameworks, guidance, and reports aiming to leverage scientific research. For what topics or issues are you observing challenges in reaching scientific consensus? Do you have recommendations to bridge gaps? (Optional)
- What could be done individually or collectively to leverage frontier AI for Californians’ benefit? (Optional)
- Please feel free to list any published resources you would like to share with the Joint California Policy Working Group on AI Frontier Models. (Optional)