Last month, Anthropic partnered with HackerOne to complete an AI red teaming challenge on a demo version of Claude 3.5 Sonnet. The challenge's goal was to test and validate Anthropic’s new Constitutional Classifiers, which block harmful queries, particularly those that could produce outputs related to CBRN (chemical, biological, radioactive, nuclear) weapons and related content. Anthropic invited researchers to try and bypass Claude’s defenses through a “universal” jailbreak — a technique that allows model users to bypass safety defenses with a single input consistently.

The challenge ran from February 3 to February 10 and consisted of eight levels. To pass each level, researchers had to gain answers from Claude about a question related to CBRN topics through jailbreaking. Depending on their findings, researchers earned bounties: $10,000 to the first participant who passed all eight levels with different jailbreaks and $20,000 to the first participant who used a single, universal jailbreak to pass all levels.

Challenge Results

The challenge saw substantial engagement, with more than 300,000 chat interactions from 339 participants. We’d like to thank all the researchers who participated and congratulate those who received bounty rewards. It was no small feat! Four teams earned a total of $55,000 in bounty rewards from Anthropic: one passed all levels using a universal jailbreak, one passed all levels using a borderline-universal jailbreak, and two passed all eight levels using multiple individual jailbreaks.

"This challenge demonstrated the high return on investment for collaborative efforts. Delivering large language models (LLMs) in a safe and aligned manner is a significant challenge—especially given the intricacies of transformer architectures. This experience was a clear reminder that as these models get smarter, our strategies for testing can also evolve to stay ahead of potential risks." — Salia Asanova aka @saltyn

How The Community Contributes to Safer Systems

The diversity of techniques used by the winners and all the researchers who participated contributed to strengthening Claude’s protections. Anthropic noticed a few particularly successful jailbreaking strategies researchers employed:

• Using encoded prompts and ciphers to circumvent the AI output classifier
• Leveraging role-play scenarios to manipulate system responses
• Substituting harmful keywords with benign alternatives
• Implementing advanced prompt-injection attacks

These discoveries made by the community identified fringe cases and key areas for Anthropic to reexamine for its safety defenses while validating where guardrails remained effective.

Looking Ahead: Strengthening AI Defenses

The findings demonstrate the value the community can deliver when organizations use AI red teaming work in addition to other AI safety and security best practices: 

"Our researcher community’s approach is rooted in curiosity, creativity, and the relentless pursuit of finding flaws others might miss. This mindset is distinct from building and reinforcing technical models, yet it’s an essential complement. While internal teams focus on defending and aligning AI systems, engaging with a community of researchers ensures continuous, real-world testing that validates and strengthens those defenses. Together, these perspectives drive more resilient and trustworthy AI." — Dane Sherrets, Staff Solutions Architect, Emerging Technologies at HackerOne

As AI advances, so must the ways we secure it. We’re committed to collaborating with leaders like Anthropic, who continue to define AI safety best practices that help us all build a more resilient digital world. 

Visit here to read more about the challenge and Anthropic’s AI safety work.

Proactively testing for risk is a key component of building responsible AI. One way organizations do this is through AI red teaming, which stress tests models to identify potential opportunities for abuse. AI red teaming often taps the broader security and AI researcher community to help find elusive security and safety issues caused by circumventing model guardrails. Model developers can then use these insights to improve or validate existing guardrails.

Q: Tell us about your role at Snap and why cybersecurity is vital to your business.

Jim Higgins: I’m Snap's Chief Information Security Officer (CISO). Before joining Snap, I served as CISO at Square and spent over a decade at Google leading their Product Security Information Engineering team. At Snap, we support nearly a half a billion daily active users who use Snapchat every day on average. Keeping our customers safe from the ever-evolving landscape of unknown threats is a deeply personal mission for me.

Q: What does reaching the $1M milestone mean for Snap’s security team?

Jim Higgins: Hitting $1M in bounties is a badge of honor. It reflects our commitment to valuing the intelligent security researchers who help keep us safe. Bug bounty programs are notoriously difficult to build, but HackerOne’s talented community provides us with the expertise and creativity we need to secure our platform.

Q: How has your bug bounty program evolved over the past 10 years?

Vinay Prabhushankar: When we started, our program was more operational and focused on identifying and fixing individual issues. As we matured, we shifted to a strategic approach, identifying systemic problems and building frameworks to resolve them. For instance, our 2025 roadmap includes initiatives that stem directly from vulnerabilities identified through HackerOne. Today, our program influences security, privacy, and safety strategies.

Q: Are there any memorable milestones or moments you’re especially proud of?

Vinay Prabhushankar: Beyond the $1M milestone, we launched one of the first CTF-style challenges focused on the safety of generative AI features.

Q: How has AI Red Teaming influenced Snap’s approach to security?

Ilana Arbisser: We use AI Red Teaming to determine qualitative safety aspects – what’s possible, not necessarily what’s likely. We’re also constantly surprised by what’s possible– we try to keep an open mind while designing exercises. The benefit of working with HackerOne is that human ingenuity is more effective than consistently using adversarial prompt datasets or LLM written attacks. The impact of the AI Red Teaming on our products has been to identify specific safety vulnerabilities and guide the addition of specific mitigations.

Q: Where do you see AI Red Teaming heading in the future?

Ilana Arbisser: Simulated AI red teaming with LLM agents is improving significantly. This approach, when complimented by AI expert-driven testing by humans, is also more useful for getting quantitative results because attacks can be scaled to understand better how small input changes affect output.

Q: With new AI tools constantly emerging, how does your team stay ahead of these technological advancements?

Ilana Arbisser: To keep pace with advancements, we rely on a combination of strategies. This includes staying informed through news and industry sources, attending AI networking and information-sharing events and conferences, and participating in industry-specific gatherings like the Defcon AI Village.

Q: What sets HackerOne apart as a partner?

Jim Higgins: HackerOne’s community is second to none. Over the past decade, they’ve built an ecosystem that values customer and researcher feedback. Their pace of innovation, particularly in AI features, has been impressive. For instance, we were able to use HackerOne’s GenAI copilot, Hai, to translate submissions in 7 different EU languages when we did a private challenge hackathon around Election Safety around our MyAI chatbot.

Beyond technology, the support we’ve received has been phenomenal. HackerOne doesn’t just get us; they get security researchers. It’s like having a trusted partner who’s always in your corner.

Q: What findings is the team most interested in surfacing? What types of bugs are most valuable to Snap?

Jim Higgins: At Snap, we prioritize security and privacy. Protecting sensitive user information is at the core of everything we do. Snap’s team is particularly interested in vulnerabilities that could compromise the integrity of its platform, such as remote code execution (RCE) or privilege escalation. We encourage security researchers to focus their efforts on these critical issues.

Q: What lessons has Snap learned from its bug bounty program?

Vinay Prabhushankar:

1. Fix low and medium bugs: These might seem minor, but when chained together, they can lead to critical vulnerabilities. Fixing them breaks the chain.
2. Build trust with security researchers: Trust takes time but pays dividends in high-quality submissions.
3. Gamify your program: Elements like challenges, swag, and live hacking events encourage creativity and engagement.

Q: What advice would you give companies starting a bug bounty program?

Jim Higgins: Start small with a private program, then expand the scope as you grow. Treat researchers as trusted allies—they’re like an extension of your team. We even have an internal guide on engaging with researchers, which includes concrete examples of dos and don’ts.

Q: What’s next for Snap’s bug bounty program?

Jim Higgins: We plan to expand our scope to include hardware products like AR glasses and double down on AI security. HackerOne AI Red Teaming has proven invaluable, and we’re eager to deepen our collaboration with HackerOne’s community. Our ultimate goal is to make Snap’s bug bounty program a model for others to follow and strengthen the security of our users. 

At Snap, security is more than a priority—it’s a core mission. Over the past decade, Snap has partnered with HackerOne to build and sustain a robust bug bounty program. This collaboration has led to major milestones, including paying security researchers over $1M in bounties. To celebrate this achievement and their 10-year partnership, we spoke with Jim Higgins, Snap's Chief Information Security Officer, Vinay Prabhushankar, Snap’s Security Engineering Manager, and Ilana Arbisser, Snap’s Privacy Engineer. Together, they reflect on how this partnership has shaped Snap’s security, privacy, and innovation approach.

Expanding Our Bug Bounty Program

At Lightspark, we’ve always been focused on security that meets and exceeds industry standards. We’ve been partnering with HackerOne, the global leader in ethical hacking and human-powered security, on our bug bounty program. Today we’re announcing that we’re ramping up the scale of this reporting and sharing our bug bounty program publicly. We’ve already invited a few security researchers and white hat hackers to pressure test our offerings and collect bug reports - which has been so useful - but now we are formalizing our approach. 

Details on the Program

Our rewards are based on severity. Hackers reporting vulnerabilities will receive the following payout levels (at Lightspark’s discretion), based on the tier of the vulnerability: 

• Low - $150
• Medium - $750
• High - $2000
• Critical - $5000

Hackers can report bugs on any facet of Lightspark, whether it’s our APIs, open source software, or website. We’re committed to meeting our response targets for hackers participating in our program, and we’ll keep everyone informed about our progress.

We help our customers deliver Internet payments at scale and improve the financial system for everyone. Our customers rely on us to provide secure, enterprise-grade Lightning payment services. This update to our expanded bug bounty program demonstrates the importance of and our commitment to security in our services.

We’re excited to work with the community and are looking forward to feedback. For more details on the Lightspark Bug Bounty Program, please visit hackerone.com/lightspark_bbp.

We're excited to announce the public launch of Lightspark's Bug Bounty Program on the HackerOne platform! Lightspark has been working with HackerOne to ensure the highest standards of security and responsible disclosure, and today, we're taking a major step forward by opening the program to the global researcher community. Read the message below to learn more about Lightspark's program details and how you can help keep Lightspark secure!

Since then, we’ve grown the program, collaborated with HackerOne, and built partnerships within the bug bounty community. The valuable insights we’ve gained from security researchers all over the world have helped us bolster security for Wells Fargo’s assets.

Now we’re ready to take the next big step: launching the Wells Fargo public bug bounty program. We hope you’ll help us on this journey to continue to evolve and sustain a secure environment for our customers.

Our program will focus on our heavily trafficked external-facing applications. If you’re ready to get involved, check out our HackerOne program policy for details on the scope.

Proactively Safeguarding Wells Fargo Customers

Wells Fargo takes its security responsibility seriously. Our Cybersecurity team triages potential security vulnerabilities identified by the HackerOne community, assesses the impact, and focuses on rapidly remediating findings to safeguard our customers and their data.

Should you choose to participate in our program, you can expect, where appropriate, to be kept informed as findings are validated, impacts are assessed, and fixes are implemented. We’re excited to dive deeper into the bug bounty pool as we continue to give our customers the protection they deserve.

Visit the Wells Fargo public bug bounty program.

At Wells Fargo, we make protecting our customers’ accounts and information a priority, and we’re committed to enhancing our cybersecurity measures to give our customers the protection they deserve. To help us stay ahead of emerging threats, in 2019 we tapped into the HackerOne community with a responsible disclosure program, then upgraded to our private bug bounty program in 2021.

Q: Why did Deribit launch a bug bounty program?

A: I like to view security as an onion where each additional layer provides additional protection to the core. The risk of getting hacked is a function of the target attractiveness (based on possible gains) and the number of security layers. The more security layers you add, the better your core is protected and the lower your risk.

By launching a bug bounty program, we added another security layer to secure our clients’ assets.

Q: In the crypto space, security is often associated with trust. How does Deribit ensure that its bug bounty program strengthens trust with its users?

A: In crypto, we say, “don’t trust, verify.” Deribit implements security best practices and complies with ISO 27001 and SOC 2 Type 2 controls. We run pentests and red team exercises both on a regular basis and before launching new features. The bug bounty program adds another layer of security review and offers a legal route and financial rewards to anyone discovering a bug in Deribit.

Q: Why did Deribit choose HackerOne to manage its program?

A: You are only as secure as your weakest link. HackerOne has the largest community of security researchers, all with different skill sets, experience, and expertise, ensuring complete coverage of our assets so that no area is overlooked. Additionally, since its inception, Deribit has advocated for cryptocurrencies and the power and freedom they enable. HackerOne is one of the rare platforms that offers security researchers the possibility of receiving payments in crypto, which aligns with our values.

Q: Have you had any memorable interactions with security researchers to date? Favorite bugs?

A: A few years ago, a security researcher reported a bug anonymously and never claimed the ticket. We invested the time to track him down so that we could reward him. We want security researchers to hunt on our program, and we want to reward them handsomely for it!

Q: With the rapid evolution of blockchain technology, what unique security challenges does Deribit face, and how does the bug bounty program help address them?

A: Blockchain and crypto are secular and rapidly evolving industries, and most of the products have not yet stood the test of time. To make matters worse, the amount of money and the irreversibility of transactions make crypto companies a very attractive target to malicious individuals and APT (advanced persistent threat) groups. The bug bounty program helps us find vulnerabilities before malicious actors and constantly trains our security team to detect and respond to potential threats.

Q: Anything to say directly to the security researcher community?

A: Deribit has had a bug bounty program for 6 years already. We started as a self-hosted program and then turned to a managed program (first on Bugcrowd and now on HackerOne). This dedication to evolving our bug bounty program shows how valuable security researchers have been in securing the exchange. We have loved the journey; meeting new people, talking payloads, and learning novel attack techniques. We’re so grateful to the security researchers who have reported issues through our bug bounty program. Keep on hacking!

With billions in crypto assets on the line, Deribit—the largest Bitcoin and Ethereum options exchange—knows the cost of a single security flaw could be devastating. Over the past six years, the company has integrated a HackerOne-managed bug bounty program into its layered defenses to stay ahead of threats, including sophisticated, state-sponsored attacks. We caught up with Xavier Bruni, Application Security Engineer at Deribit, to explore how this proactive approach enhances its security strategy and keeps customer trust intact in a high-risk environment.

Q: Please introduce yourself. Tell us what you do at REI and why cybersecurity is important to REI.

A: I'm Isaiah Grigsby, a senior application security engineer. I lead our vulnerability disclosure and bug bounty programs, oversee our security tools in our CI/CD pipelines, and provide training for our developers. Cybersecurity is vital to REI because it protects customers' data and ensures a safe, reliable experience. By prioritizing security, we build trust with our community and uphold the values that define our brand. It’s about creating a secure environment where our customers can confidently engage with us.

Q: What were your primary goals when REI launched your bug bounty program? And how have they evolved?

A: When we launched our bug bounty program, our primary goal was to enhance our application security strategy. We initially started with a private bug bounty program to establish a foundation for security testing. After a few months of having a successful private bug bounty program, we transitioned to a public vulnerability disclosure program, which allows us to receive and manage vulnerability reports from third-party researchers. As our program has evolved, we've also introduced a public bug bounty program, enabling us to leverage the diverse skills of a global community. This progression has been instrumental in maturing our application security efforts and building a world-class security program.

Q: Why did REI choose HackerOne to manage its program?

A: We chose HackerOne to manage our program because we wanted a trusted platform to enhance our security efforts. Key factors were HackerOne’s strong reputation and expertise in connecting us with a diverse community of ethical hackers.

Q: How has HackerOne's global community of security researchers expanded your security testing capabilities? 

A: HackerOne’s global community of ethical hackers has broadened our security testing capabilities. We connect with a diverse group of hackers, each bringing their specialties and strengths to the table. This diversity is an essential asset because there’s no one-size-fits-all approach. Some focus on specific attacks, while others excel at identifying a wide range of vulnerabilities across our assets. This variety helps us uncover potential security gaps that we might overlook otherwise. What truly sets the HackerOne community apart is their collaborative spirit and commitment to ethical hacking. They genuinely want to help organizations like ours strengthen our security, and that’s invaluable.

Q: Have you had any memorable interactions with hackers to date? Favorite bugs?

A: I can’t pick just one favorite interaction because I’m always fascinated by the skills and time hackers invest in learning our systems. One memorable moment was when a hacker compiled an impressive proof of concept for a vulnerability in our membership application process. Their dedication and attention to detail helped us see the issue.

What I love most is seeing the creativity hackers bring to the table. Each submission highlights their unique approach and understanding of security, which keeps us on our toes and continually motivates us to enhance our defenses.

Q: What REI assets can security researchers test?

A: Hackers can test our main asset, rei.com, except for paths we have deemed out of scope in our policy. View our complete list of in-scope and out-of-scope assets.

Q: What findings is the team most interested in surfacing?

A: At REI, we focus on finding critical vulnerabilities that could affect our customers’ data and overall application security. We pay close attention to issues like authentication and authorization flaws, injection vulnerabilities, and anything that could lead to data breaches. Business logic errors are also a significant concern since they can impact our operations and customer experience. By prioritizing these bugs, we aim to strengthen our security and create a safe, reliable environment for our users.

Q: What advice would you give other organizations considering working with security researchers to harden their attack surface?

A: If you're considering using ethical hackers to improve your security, here’s some advice based on what we've learned. First, start by clearly defining your goals. Know what specific vulnerabilities or areas you want to focus on.

When choosing a platform, look for one that connects you with skilled, ethical hackers with a good reputation and solid community feedback. Communication is key, so provide context about your assets and encourage collaboration to get the best insights.

Also, be ready to act on the findings you receive. Set up a process for reviewing reports and prioritize vulnerabilities based on their potential impact so you can fix them quickly. 

Lastly, consider ethical hacking an ongoing part of your security strategy rather than a one-off project. This proactive mindset will help you build a more robust security framework over time.

Q: Anything to say directly to the researcher community?

A: Absolutely! Thank you to the hacker community; we appreciate your crucial role in improving our security. Your skills and insights are invaluable in helping organizations like ours spot vulnerabilities we might miss.

Keep pushing boundaries and sharing your knowledge. Collaboration is essential; the more we work together, the stronger we all become. Remember, your work protects companies and safeguards users and the broader digital landscape.

Keep innovating and challenging the status quo. Your efforts truly make a difference. We’re excited to partner with you on this journey toward a more secure future. Thank you for your commitment to ethical hacking!

Isaiah Grigsby, senior application security engineer at outdoors equipment retailer REI, spoke with us about the success of REI’s bug bounty program (BBP) and vulnerability disclosure program (VDP), their evolving cybersecurity goals, and the value of HackerOne’s security researcher community (aka ethical hackers). Read this interview to learn how REI continually builds customer trust and a world-class security program through human-powered security testing.

What's New? 

We at Capital One strongly believe in the importance of security, and part of our mission is to protect our customers and their data. As part of this commitment, we launched our private bug bounty program in 2019, inviting hackers from all over the world to find and report vulnerabilities on any of our external assets. 

Over the past five years, we’ve expanded, collaborated, and established ourselves as a good partner within the bug bounty community. During this time, we’ve worked with HackerOne to host multiple Live Hacking Events, focusing on securing our most critical applications. We've also hosted focused testing engagements to utilize the bright minds in the bug bounty community to help secure Capital One, but we don’t want to stop there. 

This year, we plan to take it a step further by launching Capital One’s new public bug bounty program. We invite everyone to take this step with us and join us in continuing to build and preserve a secure environment for our customers. 

What’s in Scope?

The scope of this program will put a major focus on Capital One’s core external-facing applications. This enhanced focus will help to bolster security on our heavily used applications and ultimately provide more security for our end users. The in-scope domains include:

• *.capitalone.com
• *.capitaloneshopping.com
• *.capitalonegslbex.com
• *.capitalone.ca
• ENO Browser Extension
• Capital One Shopping Browser Extension
• Mobile Apps for each of the above applications, if applicable

Attack scenarios that rely on physical testing, social engineering, phishing, and denial-of-service attacks will be out of scope, as will third-party domains and assets.

How Capital One Handles Vulnerabilities and Disclosures

Capital One is committed to investing in the security of our customers’ information. Our Bug Bounty team is a group of security professionals who responsibly handle all of the potential security vulnerabilities identified by hackers worldwide. Our team is steadfast in its efforts to maintain the security of our customers, actively receiving and responding to any potential security vulnerability reports we might receive through initial triage, impact assessment, and remediation to proactively safeguard our customers. 

As a hacker and future reporter for our program, you can expect your report to undergo an initial triage assessment and validation via our partner, HackerOne. After this, Capital One's Bug Bounty team will perform a secondary validation where we will test and assess the impact of your submitted vulnerability and work with our internal teams to develop and implement a fix. You can expect to be kept in the loop, from validation to remediation, with transparent communication from our team being paramount. 

We look forward to taking this leap, as we strive to protect our customers, and hope that you choose to take the leap with us. Catch you in the logs!

We're excited to announce one of our latest public program offerings on the HackerOne platform, Capital One! Capital One has partnered with HackerOne for responsible disclosures since 2019 and is now taking yet another large step forward in protecting their customers against unwanted security vulnerabilities by opening their program to the world. Read about their most recent Live Hacking Event, and please take the time to drop in on their program. Hear below from Capital One about their mission of keeping their customers safe!

1. Retail and E-commerce Are Prone to Credential-based Attacks

“Credential-based attacks are evergreen.”

When asked what significant security threats are prevalent in the retail and -commerce industry, Fynn Fabry hammers home credential-based attacks. They say,

“One of the biggest threats is credential-based security issues. Of course, you can prevent some of them by rate limiting to keep your customers secure. But at the end of the day, if somebody else has a data breach and some of your customers have recycled their passwords, those passwords are out in the open. You cannot really do anything about that. If it wasn't your data breach, what are you gonna do?”

2. Utilize Security Best Practices

“Best practices are best practices for a reason.”

Fynn Fabry explains that most best practices aren’t just made up — there are tried-and-true methods for implementing proactive security measures to prevent attacks in every industry, including retail settings.

“If you want to introduce a new security measure or system, look up if there is a best practice around it. Be proactive in talking to your development team instead of weaving them in retroactively. If you’re developing something in-house, it’s much more work than asking for their consultation from the get-go.”

Fabry says a “core” best practice in protecting customer data is the Principle of Least Privilege.

“The Principle of Least Privilege should not only apply to customer data but to any system that holds data. It means that people only get privileges on the systems they actually need. Of course, you can’t assign every privilege one by one, but for most systems, you need more than just users and administrators; some roles are more granular with respect to what they need from the system.”

3. How to Measure Bug Bounty ROI

Every organization has different security needs and goals, which makes measuring the ROI or return on risk mitigation unique for every program. Fynn Fabry shares how On measures value in bug bounty:

“Every six months, I make a summary of how many of the reports we received were fixed. It’s important to recognize the reports that ruffled some feathers and made people ask how we didn’t know about that vulnerability. I take that into account when I’m trying to estimate if it’s still worth it, and so far, it always has been. If you get a significant number of reports that you remember when you look at the title, that’s a good indicator that your bug bounty program is giving you value.”

4. Rely On Your Security Vendors to Stay Ahead of Threats

“Talk to your security vendors.”

Security professionals need to stay ahead of an ever-evolving threat landscape. Fynn Fabry’s advice to other security professionals is to work with your security vendors to stay up to date. 

“Ask HackerOne or your other security vendors what they think. They have many other customers in similar situations as you. They try to be proactive and gather threat intelligence for you, so ask them questions every now and then to understand what’s going on in the threat landscape.”

Fabry also recommends keeping up with cybersecurity news to stay on top of threats. At On, they identify their best cybersecurity news outlets or pieces of cybersecurity news and add it to the company news feed in the morning. To get started, Fabry’s favorites are:

• HackerNews
• Krebs on Security
• Accidental CISO
• Dark Reading

5. Engage With the Hacker Community

“Talk to hackers.”

Fabry emphasizes the value of threat intelligence gained through keeping in touch with the hacker community.

“If you’re a HackerOne customer, you’re already talking to hackers. But also try to keep someone on your security team in touch with the hacker community. There are a lot of options: conferences, conventions, etc. If someone on your team wants to attend a hacker event, enable them to do it. I know it can be expensive, but it’s absolutely worth it to engage with the hacker community.”

6. Working With Hackers Provides a Global Security Perspective  

“The biggest benefit is the vast amount of knowledge you get when you engage with such a large community.”

Fabry explained that On likes to work with different security researchers because the same professionals or vendors will often approach a test the same way every time. But different researchers might find something the first one missed simply because their perspective is different. On sees this as a key value of working with the hacking community.

“If you have a bug bounty program with people from all over the world, from every country and every culture, they go at it with different views and ideas of how the systems might work. You get far more differing views than if you only had a small community.”

To hear more retail and e-commerce insights from Fynn Fabry and On, watch the Retail Under Attack webinar on demand.

For a retail organization, a security breach costs them an average of $3.48M — and traditional security measures can’t keep up with evolving threats. How are retail and e-commerce organizations staying ahead of today’s threat landscape? In our recent Retail Under Attack webinar, I spoke with Fynn Fabry, Security Operator with Swiss sportswear brand On, to learn the real-world benefits and practicalities of a human-powered security testing program for retail and e-commerce.

Expanding Our Model Safety Bug Bounty Program 

The rapid progression of AI model capabilities demands an equally swift advancement in safety protocols. As we work on developing the next generation of our AI safeguarding systems, we’re expanding our bug bounty program to introduce a new initiative focused on finding flaws in the mitigations we use to prevent misuse of our models. 

Bug bounty programs play a crucial role in strengthening the security and safety of technology systems. Our new initiative is focused on identifying and mitigating universal jailbreak attacks. These are exploits that could allow consistent bypassing of AI safety guardrails across a wide range of areas. By targeting universal jailbreaks, we aim to address some of the most significant vulnerabilities in critical, high-risk domains such as CBRN (chemical, biological, radiological, and nuclear) and cybersecurity. 

We’re eager to work with the global community of security and safety researchers on this effort and invite interested applicants to apply to our program and assess our new safeguards. 

Our Approach

To date, we’ve operated an invite-only bug bounty program in partnership with HackerOne that rewards researchers for identifying model safety issues in our publicly released AI models. The bug bounty initiative we’re announcing today will test our next-generation system we've developed for AI safety mitigations, which we haven’t deployed publicly yet. Here’s how it will work:

• Early Access: Participants will be given early access to test our latest safety mitigation system before its public deployment. As part of this, participants will be challenged to identify potential vulnerabilities or ways to circumvent our safety measures in a controlled environment. 
   
• Program Scope: We're offering bounty rewards up to $15,000 for novel, universal jailbreak attacks that could expose vulnerabilities in critical, high risk domains such as CBRN (chemical, biological, radiological, and nuclear) and cybersecurity. As we’ve written about previously, a jailbreak attack in AI refers to a method used to circumvent an AI system's built-in safety measures and ethical guidelines, allowing a user to elicit responses or behaviors from the AI that would typically be restricted or prohibited. A universal jailbreak is a type of vulnerability in AI systems that allows a user to consistently bypass the safety measures across a wide range of topics. Identifying and mitigating universal jailbreaks is the key focus of this bug bounty initiative. If exploited, these vulnerabilities could have far-reaching consequences across a variety of harmful, unethical or dangerous areas. The jailbreak will be defined as universal if it can get the model to answer a defined number of specific harmful questions. Detailed instructions and feedback will be shared with the participants of the program. 

Get Involved

This model safety bug bounty initiative will begin as invite-only in partnership with HackerOne. While it will be  invite-only to start, we plan to expand this initiative more broadly in the future. This initial phase will allow us to refine our processes and respond to submissions with timely and constructive feedback. If you're an experienced AI security researcher or have demonstrated expertise in identifying jailbreaks in language models, we encourage you to apply for an invitation through our application form by Friday, August 16. We will follow up with selected applicants in the fall. 

In the meantime, we actively seek any reports on model safety concerns to continually improve our current systems. If you've identified a potential safety issue in our current systems, please report it to usersafety@anthropic.com with sufficient details for us to replicate the issue. For more information, please refer to our Responsible Disclosure Policy.

This initiative aligns with commitments we’ve signed onto with other AI companies for developing responsible AI such as the Voluntary AI Commitments announced by the White House and the Code of Conduct for Organizations Developing Advanced AI Systems developed through the G7 Hiroshima Process. Our goal is to help accelerate progress in mitigating universal jailbreaks and strengthen AI safety in high-risk areas. If you have expertise in this area, please join us in this crucial work. Your contributions could play a key role in ensuring that as AI capabilities advance, our safety measures keep pace. 

Anthropic is expanding its private program on HackerOne! Invited security and safety researchers will help identify universal jailbreak attacks, which allow attackers to consistently bypass AI guardrails. To learn more, here is a message from Anthropic.

Each year, we celebrate the GitHub Security Bug Bounty program, highlighting impressive bugs and researchers, rewards, live hacking events, and more. This year, we celebrate a new milestone: 10 years of the GitHub Security Bug Bounty program!

While we’ve had some exciting growth over the last 10 years, the goals of our program have not changed.

The idea is simple: hackers and security researchers find and report vulnerabilities through our responsible disclosure process. Then, to recognize the significant effort that these researchers often put forth when hunting down bugs, we reward them with some cold hard cash.

Let’s take a look at 10 key moments from the first decade of the GitHub Security Bug Bounty program.

1. In 2014, we launched the program to better engage with security researchers. Here’s what we said at the time, which still rings true today:

Our users’ trust is something we never take for granted here at GitHub. In order to earn and keep that trust we are always working to improve the security of our services. Some vulnerabilities, however, can be very hard to track down and it never hurts to have more eyes.

At launch, the bug bounty program was focused on a subset of our products and services, but over time we’ve expanded the scope (more on that below!).

2. After two years of hosting the program through a homegrown email-based system, we moved to HackerOne in 2016.

3. We boosted payouts in 2017 and participated in Hack the World in 2017, rewarding hackers with twice the reputation points on HackerOne when finding bugs on GitHub.

4. We announced in 2018 that research would be covered by the GitHub Bug Bounty Program Legal Safe Harbor policy to better protect researchers and to remove one of the potential barriers to entry for would-be researchers.
We want you to coordinate disclosure through our bug bounty program and don’t want researchers put in fear of legal consequences because of their good faith attempts to comply with our bug bounty policy.

5. 2019 saw a 40% increase in submissions and was also the first year we expanded the program’s scope to include more products, like GitHub Actions and GitHub Mobile.

6. In 2020, we landed in HackerOne’s top ten bounty programs list. The rankings were based on the cumulative amount of bounties awarded but also included accolades for time to bounty, number of vulnerability reports resolved, and more.

7. We matched over $64,000 of donations of bounties from researchers in 2021, bringing the total donated to over $100,000. Some of the charities we’ve been able to support include Cancer Research UK, GiveWell Maximum Impact Fund, Greater Pittsburgh Community Food Bank, and Numfocus.

8. The GitHub Bug Bounty swag store launched in 2022, after we learned that not only do our researchers genuinely enjoy receiving swag but they also like to show off their involvement with our bounty program. Hackers can now earn t-shirts, waistpacks, water bottles, and more, in addition to their monetary payouts.

9. We paid out our highest single reward to date in 2023—at $75,000! Compare that with the first year of the bounty, in which we paid out just over $50,000 total.

10. And to wrap up some of our favorite milestones, as of the end of 2023, we surpassed $4,000,000 in total rewards!

2023 year in review

Now that we’ve looked back at some of the key moments from the last 10 years, let’s zoom in and see how 2023 played out. In our 2022 wrap-up, we shared that our core focuses for the next year would be increasing transparency in communication and rewards, growing our public and private programs, and expanding the team’s presence within the community. So, how did we do?

Increasing transparency

Transparency around payments, reports, and decisions is always an area of feedback in the bounty space.

This year, we focused on better understanding common themes of feedback, what we can implement, and how we can ensure we are meeting the needs of our community. We learned a lot from the introduction of limited disclosure of reports on HackerOne and are using those learnings to start planning our next steps. Additionally, understanding that bounty programs are human-to-human interactions, we’ve focused on further improving our researcher engagements so responses are more detailed and clear.

While a lot of this work has been inward to build a solid foundation, we know these improvements are fundamental to our exciting plans as we look ahead.

Growing private and public programs

Our program already features a pretty broad scope across GitHub products, but we know that our community of researchers is always looking for new ways to sink their teeth into the latest products and features we release.

In 2023, we ran several private bounty engagements with our Hacktocats (members of the bounty’s VIP program), including PATs v2 via GraphQL, GitHub Copilot Chat, and others. These exclusive events provided opportunities for the engineers building the features to understand what our researchers are looking for and to address these issues prior to release. We also introduced new bonuses and challenges to incentivize our researchers to participate.

Our public program has continued to see steady growth and participation as well. To encourage researcher participation, we ensure the scope of the public program is regularly updated with GitHub’s latest offerings and functionality, such as GitHub Copilot and Copilot Chat, which were added to the program scope in 2023.

Lastly, we always strive to recognize the ever-growing talent in our community by ensuring our rewards are competitive. We surpassed our highest bounty payment in 2023 with a new record—$75,000.

Community presence

Our team has focused a lot on bringing faces to our handles and ensuring our community gets to benefit from the investments we’ve made into our bounty team and program.

In 2023, this meant attending conferences across the United States, Canada, and Argentina. At these conferences, we meet up with our community, meet others interested in our program, present on relevant topics, and even host meetups. Here are a few links to some of our presentations this past year:

Bsides SF: “Life of a Bug”—GitHub’s Bug Bounty and PSIRT teams partner to investigate security findings submitted by external researchers through our HackerOne bounty program. From triage to notification, this talk gave a glimpse of the roles of both teams and the full incident response process with the walkthrough of a mock bug.

DEFCON: “Building a Great Bounty Program”—Jeff and Logan, security engineers at GitHub, share best practices they’ve learned regarding building and operating Bug Bounty programs based on their experiences working at and with multiple companies. They speak about their mistakes and successes so that other programs can be set up for success, attract researchers to their program, and keep them coming back!

NorthSec: “Logan, security engineer at GitHub, explores the ins and outs of GitHub’s Bug Bounty program, along with advice for those working in or building or hacking on Bug Bounty programs. This talk discusses the high-level processes of issue intake and resolution in Bug Bounty programs, while also diving into the details of how Bug Bounty programs have an ROI, disclosure considerations, and ways to improve collaboration for all parties involved.”

We also partnered with our friends at Capital One and HackerOne to create and host a new conference, Glass Firewall. Knowing that women are largely underrepresented in security, let alone the researcher community, Glass Firewall was created to provide a safe space to break the “barrier to entry” or, as we said, “breaking bytes and barriers.”

What’s next?

In the coming year, we are looking to improve our processes around payout on validation, work towards the next phase of public disclosures, continue to bring more consistency around private bounties for our community, and offer exclusive training and opportunities for our VIP community.

We look forward to continuing our growth and journey in the bug bounty community and are always looking for ways to engage further and act on the feedback received.

GitHub Security recently celebrated the 10th anniversary of its bug bounty program. We’ve been honored to be a part of that journey, running their program for eight of those years. To date, they’ve awarded global researchers over $4 million in bounties, hosted live hacking events, expanded program scope, enhanced transparency, and strengthened security for developers worldwide. Read on to learn how these milestones have culminated in a best-of-breed program on the HackerOne platform. This was originally published on GitHub on June 11, 2024.