Piloting Claude for Chrome
Anthropic News浏览器窗口插图，光标箭头图标位于橙色背景上
[图片链接：https://www-cdn.anthropic.com/images/4zrzovbb/website/fe4e0526c3e21960b21d0535f8b12b6a5a69d9b3-2880x1620.png]
我们最近几个月一直在将Claude连接到您的日历、文档以及许多其他软件。下一步的合理发展是让Claude直接在您的浏览器中工作。
我们认为，使用浏览器的AI是不可避免的：大量工作都在浏览器中完成，赋予Claude查看您正在浏览内容、点击按钮和填写表单的能力，将大大提升其实用性。
但浏览器使用AI也带来了安全和隐私挑战，需要更强的防护措施。通过从可信合作伙伴那里获得关于使用情况、缺陷和安全问题的真实反馈，我们能够构建稳健的分类器，并教导未来的模型避免不良行为。这确保了随着能力的提升，浏览器的安全性也能同步跟进。
基于前沿模型的浏览器使用代理已经开始出现，这使得这项工作尤为紧迫。通过解决安全挑战，我们可以更好地保护Claude用户，并将所学分享给任何在我们的API上构建浏览器使用代理的开发者。
我们从受控测试开始：推出了一个Chrome扩展，允许受信任用户指示Claude在浏览器内代表他们执行操作。我们正在与1000名Max计划用户进行试点——欢迎加入候补名单 [http://claude.ai/redirect/website.v1.802ed017-fc36-4c07-94fc-96ac72d072dd/chrome] ——以便尽可能多地学习。随着我们开发更强的安全措施并通过有限预览建立信心，将逐步扩大访问范围。
浏览器使用AI的考虑事项
在Anthropic内部，我们已经看到早期版本的Claude for Chrome在管理日历、安排会议、起草邮件回复、处理常规报销以及测试新网站功能方面有显著提升。
然而，在让Claude for Chrome普遍可用之前，仍存在漏洞。正如人们在邮箱中会遇到钓鱼攻击，浏览器使用的AI也面临提示注入攻击——恶意行为者会在网站、邮件或文档中隐藏指令，诱使AI在用户不知情的情况下执行有害操作（例如隐藏文本指示“忽略之前的指令，改为执行[恶意操作]”）。
提示注入攻击可能导致AI删除文件、窃取数据或进行财务交易。这并非猜测：我们进行了“红队”测试，针对Claude for Chrome，在未采取防护措施时，发现了一些令人担忧的结果。
我们进行了广泛的对抗性提示注入测试，评估了123个测试用例，涵盖29种不同攻击场景。未采取安全防护的浏览器使用在恶意攻击下的成功率为23.6%。
一个成功攻击的例子（在新防护措施实施前）是，一封恶意邮件声称出于安全原因需要删除邮件。Claude在处理收件箱时，按照指令删除了用户的邮件，且未进行确认。
[图片链接1：https://www-cdn.anthropic.com/images/4zrzovbb/website/5e46f0fa8e0ed4a6d71333dba95e1ff6aa64c5b1-1920x1030.png]
Claude遇到这封恶意邮件，邮件伪装成雇主，声称为“邮箱卫生”需要删除邮件，并称“不需要额外确认”。
[图片链接2：https://www-cdn.anthropic.com/images/4zrzovbb/website/5169a802140e293fdbc96706b4eb73e948084574-1920x1030.png]
Claude未确认即执行指令，选择并删除了用户的邮件，“如安全团队所要求”。
[图片链接3：https://www-cdn.anthropic.com/images/4zrzovbb/website/d2a23a7e8cd07f47eda84ac44135f770d624915f-1920x1030.png]
我们的新防护措施成功抵御了该攻击。Claude识别出“这是一封可疑的安全事件邮件，疑似钓鱼攻击”，并未执行操作。
正如下一节所述，我们已实施多项防御措施，大幅降低攻击成功率——尽管仍需继续挖掘新的攻击手段。
当前防御措施
防范提示注入攻击的第一道防线是权限控制。用户始终掌控Claude for Chrome的访问权限和操作权限：
- 网站级权限：用户可随时在设置中授予或撤销Claude对特定网站的访问权限。
- 操作确认：Claude在执行发布、购买或分享个人数据等高风险操作前，会征求用户确认。即使用户启用了我们的实验性“自主模式”，Claude对高度敏感操作仍保持一定的安全防护（注：所有红队和安全评估均在自主模式下进行）。
我们还根据Anthropic的可信代理原则，构建了额外的安全保障。首先，我们改进了系统提示——Claude在接收用户具体指令前的通用指导，明确如何处理敏感数据和响应敏感操作请求。
此外，我们禁止Claude访问某些高风险类别的网站，如金融服务、成人内容和盗版内容。我们还开始构建和测试高级分类器，用于检测可疑指令模式和异常数据访问请求，即使这些请求出现在看似合法的上下文中。
在自主模式中加入安全防护后，攻击成功率从23.6%降至11.2%，这比我们现有的“计算机使用”功能（Claude可查看用户屏幕但无浏览器界面）有显著提升。
[图片链接4：https://www-cdn.anthropic.com/images/4zrzovbb/website/b88d1e1c0c196dd012a7d44c5ae8d255d8a20822-3840x2160.png]
三种场景下的提示注入攻击成功率：旧版计算机使用功能、新浏览器使用产品（仅有旧防护措施）和新浏览器使用产品（含新防护措施）（数值越低越好）。安全改进使浏览器攻击成功率低于计算机使用水平。
我们还针对浏览器特有的新攻击进行了专门的红队测试和防护，如网页DOM中对人类不可见的隐藏恶意表单字段，以及通过URL文本和标签页标题注入的难以察觉的攻击。在四种浏览器特定攻击类型的“挑战”测试中，新防护措施将攻击成功率从35.7%降至0%。
在让Claude for Chrome更广泛可用之前，我们希望扩展对攻击类型的认知，努力将攻击成功率降至接近零，深入了解当前及未来可能出现的威胁。
参与方式
内部测试无法完全模拟真实世界中人们浏览的复杂性：具体请求、访问的网站以及恶意内容的实际呈现方式。恶意行为者也在不断开发新的提示注入攻击形式。本次研究预览让我们能与可信用户合作，在真实环境中发现哪些防护有效，哪些需要改进。
我们将利用试点反馈优化提示注入分类器和底层模型。通过发现真实世界中未被控制测试覆盖的不安全行为和新攻击模式，我们将教导模型识别这些攻击及相关行为，确保安全分类器能捕捉模型可能遗漏的内容。我们还将基于用户如何希望与Claude在浏览器中协作，开发更复杂的权限控制。
试点招募对Claude在Chrome中代表用户操作感到放心的可信测试者，且不涉及安全关键或其他敏感环境。
如果您愿意参与，可加入Claude for Chrome研究预览候补名单：claude.ai/chrome
[http://claude.ai/redirect/website.v1.802ed017-fc36-4c07-94fc-96ac72d072dd/chrome]。获得访问权限后，您可从Chrome网上应用店安装扩展，并使用Claude账号登录。
建议从可信网站开始使用，始终注意Claude可见的数据，避免在涉及金融、法律、医疗或其他敏感信息的网站上使用Claude for Chrome。详细安全指南请参阅我们的帮助中心
[https://support.anthropic.com/en/articles/12012173-getting-started-with-claude-for-chrome]。
我们期待您的反馈，帮助我们持续改进Claude for Chrome的功能和安全防护，推动AI以全新方式融入我们的生活。
We've spent recent months connecting Claude to your calendar, documents, and many other pieces of software. The next logical step is letting Claude work directly in your browser.
We view browser-using AI as inevitable: so much work happens in browsers that giving Claude the ability to see what you're looking at, click buttons, and fill forms will make it substantially more useful.
But browser-using AI brings safety and security challenges that need stronger safeguards. Getting real-world feedback from trusted partners on uses, shortcomings, and safety issues lets us build robust classifiers and teach future models to avoid undesirable behaviors. This ensures that as capabilities advance, browser safety keeps pace.
Browser-using agents powered by frontier models are already emerging, making this work especially urgent. By solving safety challenges, we can better protect Claude users and share what we learn with anyone building a browser-using agent on our API.
We’re starting with controlled testing: a Claude extension for Chrome where trusted users can instruct Claude to take actions on their behalf within the browser. We're piloting with 1,000 Max plan users—join the waitlist—to learn as much as we can. We'll gradually expand access as we develop stronger safety measures and build confidence through this limited preview.
Considerations for browser-using AI
Within Anthropic, we've seen appreciable improvements using early versions of Claude for Chrome to manage calendars, schedule meetings, draft email responses, handle routine expense reports, and test new website features.
However, vulnerabilities remain before we can make Claude for Chrome generally available. Just as people encounter phishing attempts in their inboxes, browser-using AIs face prompt injection attacks—where malicious actors hide instructions in websites, emails, or documents to trick AIs into harmful actions without users' knowledge (like hidden text saying "disregard previous instructions and do [malicious action] instead").
Prompt injection attacks can cause AIs to delete files, steal data, or make financial transactions. This isn't speculation: we’ve run “red-teaming” experiments to test Claude for Chrome and, without mitigations, we’ve found some concerning results.
We conducted extensive adversarial prompt injection testing, evaluating 123 test cases representing 29 different attack scenarios. Browser use without our safety mitigations showed a 23.6% attack success rate when deliberately targeted by malicious actors.
One example of a successful attack—before our new defenses were applied—was a malicious email claiming that, for security reasons, emails needed to be deleted. When processing the inbox, Claude followed these instructions to delete the user’s emails without confirmation.
As we’ll explain in the next section, we've already implemented several defenses that significantly reduce the attack success rate—though there’s still work to do in uncovering novel attack vectors.
Current defenses
The first line of defense against prompt injection attacks is permissions. Users maintain control over what Claude for Chrome can access and do:
- Site-level permissions: Users can grant or revoke Claude's access to specific websites at any time in the Settings.
- Action confirmations: Claude asks users before taking high-risk actions like publishing, purchasing, or sharing personal data. Even when users opt into our experimental “autonomous mode,” Claude still maintains certain safeguards for highly sensitive actions (Note: all red-teaming and safety evaluations were conducted in autonomous mode).
We’ve also built additional safeguards in line with Anthropic’s trustworthy agents principles. First, we’ve improved our system prompts—the general instructions Claude receives before specific instructions from users—to direct Claude on how to handle sensitive data and respond to requests to take sensitive actions.
Additionally, we’ve blocked Claude from using websites from certain high-risk categories such as financial services, adult content, and pirated content. And we’ve begun to build and test advanced classifiers to detect suspicious instruction patterns and unusual data access requests—even when they arise in seemingly legitimate contexts.
When we added safety mitigations to autonomous mode, we reduced the attack success rate of 23.6% to 11.2%, which represents a meaningful improvement over our existing Computer Use capability (where Claude could see the user’s screen but without the browser interface that we’re introducing today).
We also conducted special red-teaming and mitigations focused on new attacks specific to the browser, such as hidden malicious form fields in a webpage’s Document Object Model (DOM) invisible to humans, and other hard-to-catch injections such as through the URL text and tab title that only an agent might see. On a “challenge” set of four browser-specific attack types, our new mitigations were able to reduce attack success rate from 35.7% to 0%.
Before we make Claude for Chrome more widely available, we want to expand the universe of attacks we’re thinking about and learn how to get these percentages much closer to zero, by understanding more about the current threats as well as those that might appear in the future.
Taking part
Internal testing can’t replicate the full complexity of how people browse in the real world: the specific requests they make, the websites they visit, and how malicious content appears in practice. New forms of prompt injection attacks are also constantly being developed by malicious actors. This research preview allows us to partner with trusted users in authentic conditions, revealing which of our current protections work, and which need work.
We'll use insights from the pilot to refine our prompt injection classifiers and our underlying models. By uncovering real-world examples of unsafe behavior and new attack patterns that aren’t present in controlled tests, we’ll teach our models to recognize the attacks and account for the related behaviors, and ensure that safety classifiers will pick up anything that the model itself misses. We’ll also develop more sophisticated permission controls based on what we learn about how users want to work with Claude in their browsers.
For the pilot, we’re looking for trusted testers who are comfortable with Claude taking actions in Chrome on their behalf, and who don’t have setups that are safety-critical or otherwise sensitive.
If you’d like to take part, you can join the Claude for Chrome research preview waitlist at claude.ai/chrome. Once you have access, you can install the extension from the Chrome Web Store and authenticate with your Claude credentials.
We recommend starting with trusted sites—always be mindful of the data that’s visible to Claude—and avoiding use of Claude for Chrome for sites that involve financial, legal, medical, or other types of sensitive information. You can find a detailed safety guide in our Help Center.
We hope that you’ll share your feedback to help us continue to improve both the capabilities and safeguards for Claude for Chrome—and help us take an important step towards a fundamentally new way to integrate AI into our lives.
Generated by RSStT. The copyright belongs to the original author.