← all ideas

AI Safety Trust Signal

The idea

The idea is that for you to trust an AI company's safety-first decisions, they need to publicly acknowledge and accept financial losses due to these decisions. The argument is that if a company like Google, for instance, had to publicly walk away from a lucrative AI Pentagon deal due to employee concerns, it would be a clear signal of their commitment to safety. Another example is if the founder of an AI project like OpenAI were to delay a major release until it passed all safety checks, even if it meant foregoing a significant amount of revenue. This idea holds that the only true indication of a company's commitment to safety is if they are willing to make a costly but safe decision and publicly admit to it.

Why it works

• The argument is that publicly losing money on a safety-first decision demonstrates a company's commitment to safety, making it easier to trust their claims.
• The example of Google publicly walking away from the AI Pentagon deal in 2018 due to employee concerns offers a clear signal of prioritizing safety over profit.
• If an AI company were to publicly declare they would delay a potentially harmful product release until all safety checks are met, it would be a strong indicator of their commitment to safety.
• The idea is that foregone revenue serves as an indisputable signal of a company's priorities, making such a declaration more credible.
• One view holds that if an AI company could demonstrate they have lost significant revenue due to a safety-first decision, it would be a convincing signal of their commitment to safety.

The playbook

To build trust in AI companies, follow these steps:

1. Evaluate Their Financial Health: If an AI company publicly loses money due to safety concerns, it signals their commitment to safety. This is a tangible signal that you can observe and verify. For instance, Google publicly walked away from a Pentagon deal in 2018 when their employees pushed back hard on the project’s safety. By doing so, they demonstrated a genuine commitment to safety over profit, which is a strong indicator of their reliability.

1. Monitor Their Safety Processes: Look for clear, publicly acknowledged safety check marks and delays in product releases. If an AI company takes their time to ensure every aspect of their product is safe before releasing it to the public, it shows they are prioritizing safety over speed. For example, if the founder of a company like OpenAI were to delay a voice AI release until it passed all safety check marks, it would be a clear sign that they value safety.

1. Follow Their Public Disclosures: Keep track of how companies like OpenAI respond to public concerns and criticisms. If they address these concerns openly and transparently, it indicates they are willing to take responsibility for their actions and are committed to building a trustworthy reputation. For instance, if the founder of OpenAI were to acknowledge their mistakes publicly and express their commitment to safety, it would build trust.

1. Engage with Their Community: If an AI company actively engages with their community, including sharing updates on their safety measures and listening to feedback, it demonstrates their commitment to openness and transparency. This could involve contributions to open-source projects, participation in public forums, or direct communication with users and developers.

1. Support and Advocate for Safety Advocates: Finally, support and advocate for individuals and organizations that advocate for AI safety and ethics. By doing so, you help amplify the message that safety is a priority in AI development. This could mean following their work, sharing their content, or even contributing to their efforts. By doing so, you reinforce the importance of safety in AI development.

Where people get it wrong

The argument is that people often overlook the severity of potential failure modes in AI systems. These include unintended consequences, where an AI system behaves in unforeseen ways that lead to harm. Another failure mode is algorithmic bias, where AI systems perpetuate or even exacerbate existing biases in society. The third is the risk of an AI system becoming a tool for misuse, such as hacking or creating harmful content.

Instead, people should demand transparency and accountability from AI companies. This includes being clear about potential unintended consequences, how the system is tested for fairness, and how safeguards are put in place to prevent misuse. Companies should be open about their processes and be willing to share data and results from their AI models. Demonstrating willingness to lose money over safety concerns is a strong signal, as mentioned in the source material.

Do this next

To build trust in an AI company's safety-first decisions, start by publicly losing money on a critical safety decision. Here are four concrete steps you can take today:

1. Conduct a Safety Audit and Publicly Announce Any Financial Losses: Review your company’s AI projects and identify areas where safety could be compromised. If you find a critical issue, announce the potential loss and any resulting financial impact publicly.

1. Deliver a Delayed Product Release: If your safety team recommends delaying a product release, be transparent about the reasons and the financial impact of this delay. This shows that safety is prioritized over immediate profits.

1. Create an Open Source Safety Checklist: Develop a comprehensive safety checklist for your AI products. Publish this checklist on your company’s website or GitHub and encourage others to contribute improvements. This demonstrates your commitment to safety and invites community feedback.

1. Engage with Your Community on Safety Issues: Start a forum or social media group dedicated to discussing AI safety issues. Encourage open dialogue and share information about your company’s safety measures and any challenges you face in ensuring safety.

By taking these actions, you can help build a more trustworthy reputation for your AI company, showing that safety is a top priority, even if it means temporarily diverting financial resources.