Cyber threats don’t wait, and neither should your security strategy. Organizations across Singapore are facing growing regulatory demands and increasingly sophisticated cyber risks. The best defense? A proactive approach that uncovers vulnerabilities before attackers do.

That’s why we’re excited to announce that HackerOne is now officially licensed to provide penetration testing services in Singapore. With this new certification from the Cybersecurity Services Regulation Office, we can now bring our modern, scalable Pentest as a Service (PTaaS) solution to businesses across the region—helping you strengthen security, meet compliance requirements, and stay ahead of cyber threats.

Unlike traditional pentesting providers, we don’t just hand you a static report and walk away. Our agile, expert-driven approach gives you real-time collaboration, faster results, and deeper insights—so you can turn security gaps into strengths before attackers exploit them.

Ready to rethink penetration testing? Here’s what this means for you.

Why This Matters for Organizations in Singapore

Cybersecurity threats are increasing in complexity, and regulatory requirements are becoming stricter. Organizations in Singapore—particularly those handling sensitive data—should conduct penetration testing in line with laws, standards, and frameworks like:

• Monetary Authority of Singapore (MAS) TRM Guidelines
• Personal Data Protection Act (PDPA)
• PCI DSS
• NIST Cybersecurity Framework
• Cybersecurity Act of Singapore
• ISO 27001, SOC 2, and other international security standards

With our newly-approved penetration testing services, businesses can now proactively identify vulnerabilities, strengthen security postures, and align with local and global regulations.

Modern, Scalable Pentesting for APAC

HackerOne’s Pentest as a Service (PTaaS) model modernizes the traditional penetration testing process, offering a faster, more flexible, and outcome-driven approach. Instead of rigid, slow-moving engagements, our platform allows you to:

• Launch pentests in days, not weeks
• Access a vetted global community of security experts with deep industry knowledge
• Collaborate in real-time to address findings and strengthen security
• Meet compliance mandates while focusing on meaningful risk reduction

Unlike traditional consultancy-based pentests, HackerOne PTaaS integrates seamlessly into your security workflow, ensuring continuous security improvement rather than a one-time report.

What Sets HackerOne’s Pentesting Apart?

HackerOne delivers elite penetration testing services backed by industry-leading expertise and technology. Our approach is designed for speed, accuracy, and business-aligned security outcomes.

• Speed: Start your pentest in 4-7 business days
• Vetted Experts: 75% of our testers have 5+ years of experience
• High-Impact Results: 19% of findings are critical or high severity, twice the industry average
• AI-Powered Insights: Our AI Copilot (Hai) helps interpret complex reports and suggests remediation steps
• Seamless Integrations: Works with Jira, GitHub, ServiceNow, Slack, and more for streamlined remediation

With a licensed and highly specialized security testing team, HackerOne ensures that your organization stays ahead of attackers, meets compliance requirements, and builds a more resilient security posture.

Next Steps: How to Get Started

Now that HackerOne is a licensed penetration testing provider in Singapore, organizations in the region can start securing their systems with our expert-led pentesting services.

Interested in pentesting? Contact us today to discuss your security needs.
Want to learn more? Explore our Pentest Solution Brief for detailed insights into our methodology and coverage areas. 

Meet HackerOne Recommendations: a built-in intelligence layer that continuously refines your security program, delivering personalized insights and your program's historical performance.

Eliminate Guesswork With Contextual, High-value Suggestions

With HackerOne Recommendations, you don’t need to manually sift through reports or guess which actions will impact your programs most. This automated intelligence layer continuously evaluates your security program’s performance and delivers personalized, high-value recommendations—right inside your HackerOne Home Page.

Recommendations aren’t just a generic list of tasks—they are risk-driven, context-aware, and backed by real attack intelligence based on HackerOne’s comprehensive database, which comprises over 500,000 valid vulnerabilities reported across industries.

Every month, HackerOne assesses 20 trigger conditions within your program, with a continually growing set of factors that enhance its intelligence over time. As data expands, so does the system’s ability to surface even more precise, high-impact suggestions, designed to:

• Optimize vulnerability response times by identifying bottlenecks and delays in triage workflows
• Maximize hacker engagement by analyzing payout structures, report resolution speed, and incentive alignment
• Reduce critical security gaps by identifying trends in missed, delayed, or incorrectly prioritized vulnerabilities
• Benchmark your program’s efficiency against industry peers and top performers

How HackerOne Recommendations Work 

HackerOne Recommendations are updated at the first of each month, delivering clear, actionable improvements tailored to your security program. Each recommendation includes:

• A defined action plan with specific steps to improve your program
• Supporting data and metrics to justify and quantify the impact
• Guidance on implementation, whether through direct action or with assistance from your HackerOne Account Manager or Customer Success Manager

Accessing Recommendations

Recommendations are available in the Recommendations section of your HackerOne Home Page, providing an at-a-glance view of key security improvement opportunities.

• Take Action – Select a recommendation to view detailed insights, context, and next steps.
• Review All – See a consolidated list of all active recommendations for your program.

Expanded View for In-depth Analysis

Each recommendation includes a structured breakdown for clarity and ease of implementation:

• Left-hand pane – View all recommendations applicable to your program.
• Right-hand pane – See detailed insights, including supporting data and suggested actions.
• Actionable steps – Choose specific actions to address security gaps.

Customization and Feedback

• Dismiss if not relevant – Click the Dismiss button up top to remove a recommendation from your view for 90 days.
• Provide feedback – Use thumbs-up/down ratings on individual recommendations to refine future recommendations and ensure relevance.

Enhance Program Performance With Data-driven Intelligence

HackerOne Recommendations is now available to all Bounty customers at no additional cost. Built on real-world security data, it eliminates guesswork by delivering actionable, high-impact insights—not generic alerts.

Start leveraging the industry’s most comprehensive vulnerability dataset to drive measurable security improvements. Start using HackerOne Recommendations today by connecting with our experts or exploring the HackerOne Platform. 

Security teams deal with an overwhelming volume of reports, alerts, and vulnerability data—but without the right prioritization, it's easy to waste time on low-impact issues while critical risks go unnoticed. Static benchmarks don't adapt to real-world threats, and manual analysis is too slow to keep up with the evolving attack landscape.

What if your security program could self-optimize: analyze trends, identify weak points, and proactively propose actionable steps to strengthen defenses?

To ensure that AI is more secure and trustworthy, the EO calls on companies who develop AI and other companies in critical infrastructure that use AI to rely on “red-teaming”: testing to find flaws and vulnerabilities. The EO also requires broad disclosures of some of these red-team test results.

Testing AI systems isn’t necessarily new. Back in 2021, HackerOne organized a public algorithmic bias review with Twitter as part of the AI Village at DEF CON 29. The review encouraged members of the AI and security communities to identify bias in Twitter’s image-cropping algorithms. The results of the engagement brought to light various confirmed biases, informing improvements to make the algorithms more equitable.

In this blog post, we'll delve into the emerging playbook developed by HackerOne, focusing on the collaboration between ethical hackers and AI safety to fortify these systems. Bug bounty programs have proven effective at finding security vulnerabilities, but AI safety requires a new approach. According to recent findings published in the 7th Annual Hacker Powered Security Report, 55% of hackers say that GenAI tools themselves will become a major target for them in the coming years, and 61% said they plan to use and develop hacking tools using GenAI to find more vulnerabilities. 

“Every properly designed AI application has a unique safety threat model and should implement some safety parameters or guard rails to protect against adverse outcomes. The protections you care most about are going to vary based on the use case for the application and the intended audience. But how easily are those guard rails bypassed? That is what you find out with AI red teaming.”
— Dane Sherrets, Senior Solutions Architect, HackerOne

HackerOne's Approach to AI Red Teaming

HackerOne partners with leading technology firms to evaluate their AI deployments for safety issues. The ethical hackers selected for our early AI Red Teaming exceeded all expectations. Drawing from these experiences, we're eager to share the insights gleaned, which have shaped our evolving playbook for AI safety red teaming.

Our approach builds upon the powerful bug bounty model, which HackerOne has successfully offered for over a decade, but with several modifications necessary for optimal AI Safety engagement.

• Team Composition: A meticulously selected and, more importantly, diverse team is the backbone of an effective assessment. Emphasizing diversity in background, experience, and skill sets is pivotal for ensuring a safe AI. A blend of curiosity-driven thinkers, individuals with varied experiences, and those skilled in production LLM prompt behavior has yielded the best results.
• Collaboration and Size: Collaboration among AI Red Teaming members holds unparalleled significance, often exceeding that of traditional security testing. A team size ranging from 15-25 testers has been found to strike the right balance for effective engagements, bringing in diverse and global perspectives.
• Duration: Because AI technology is evolving so quickly, we’ve found that engagements between 15 and 60 days work best to assess specific aspects of AI Safety. However, in at least a handful of cases, a continuous engagement without a defined end date was adopted. This method of continuous AI red teaming pairs well with an existing bug bounty program.
• Context and Scope: Unlike traditional security testing, AI Red Teamers cannot approach a model blindly. Establishing both broad context and specific scope in collaboration with customers is crucial to determining the AI's purpose, deployment environment, existing safety features, and limitations.
• Private vs. Public: While most AI Red Teams operate in private due to the sensitivity of safety issues, there are instances where public engagement, such as Twitter's algorithmic bias bounty challenge, has yielded significant success.
• Incentive Model: Tailoring the incentive model is a critical aspect of the AI safety playbook. A hybrid economic model that includes both fixed-fee participation rewards in conjunction with rewards for achieving specific safety outcomes (akin to bounties) has proven most effective.
• Empathy and Consent: As many safety considerations may involve encountering harmful and offensive content, it is important to seek explicit participation consent from adults (18+ years of age), offer regular support for mental health, and encourage breaks between assessments.

“It’s important to underscore that different AI models or deployments will have drastically different threat models. An AI text-to-image generator deployed on a social media network will have a different threat model than an AI chabot in a medical context. Early on in these conversations we define what the threat model is based on the use case, the regulatory environment, architecture, and other factors.”
— Dane Sherrets, Senior Solutions Architect, HackerOne

In the HackerOne community, over 750 active hackers specialize in prompt hacking and other AI security and safety testing. To date, 90+ of those hackers have participated in HackerOne's AI Red Teaming engagements. In a single recent engagement, a team of 18 quickly identified 26 valid findings within the initial 24 hours and accumulated over 100 valid findings in the two-week engagement. In one notable example, one of the challenges put forth to the team was bypassing significant protections built to prevent the generation of images containing a Swastika. A particularly creative hacker on the AI Red Team was able to swiftly bypass these protections, and thanks to their findings, the model is now far more resilient against this type of abuse.

As AI continues to shape our future, the ethical hacker community, in collaboration with platforms like HackerOne, is committed to ensuring its safe integration. Our AI Red Teams stand ready to assist enterprises in navigating the complexities of deploying AI models responsibly, ensuring that their potential for positive impact is maximized while guarding against unintended consequences.

“In my opinion, the best way to secure AI is also through the use of crowdsourcing. By engaging hackers through AI red teaming engagements, I believe we can obtain a better understanding of the rapidly changing nature of AI security and AI Safety. This will result in reduced risk in implementing these exciting new technologies and allow us to capitalize on all of the benefits.”
— Josh Donlan, Senior Solutions Engineer, HackerOne

By using the expertise of ethical hackers and adapting the bug bounty model to address AI safety, HackerOne's playbook is a proactive approach to fortifying AI while mitigating potential risks. For technology and security leaders venturing into AI integration, we look forward to partnering with you to explore how HackerOne and ethical hackers can contribute to your AI safety journey. To learn more about how to implement AI Red Teaming for your organization, download the AI Red Teaming solution brief or contact our experts at HackerOne.

As AI is adopted by every industry and becomes an integral part of enterprise solutions, ensuring its safety and security is critical. In fact, the Biden Administration recently released an Executive Order (EO) that aims to shape the safe, secure, and trustworthy development of AI. This follows action taken in California and by the Leaders of the Group of Seven (“G7”) to address AI. 

Did you know we’ve teamed up with our friends at PortSwigger to offer free 90-day licenses for Burp Suite Professional?

Burp Suite is the premier offensive hacking solution, and  when new hackers reach at least a 500 reputation on HackerOne and have a positive signal, they are eligible for 3-months free of Burp Suite Professional.

With Burp Suite, you can scan for vulnerabilities, intercept browser traffic, automate custom attacks, and more. 

It’s clear that hackers love Burp and HackerOne:

“Burp Suite is pretty much all I use.” - Mark Litchfield 

“It’s the best tool out there, simply put. I use it all the time.” - Arne Swinnen 

“To be effective as a bug hunter, you need the right tools to optimize and backup your vulnerability research. Using Burp Suite means contributing to a quality approach, from research to reporting of your finds on HackerOne.” - Baptiste Moine

“I have reported many vulnerabilities on HackerOne, most of them were found with the help of Burp Suite.” - Shawar Khan

“Burp Suite has helped me to find many bugs. The Proxy and Repeater are key features and I really like the new Collaborator Client the DNS resolution is awesome! Definitely, an important tool when doing Bug bounty programs at HackerOne platform.” - Francisco Correa

You can check out all the details including an FAQ.    

Happy hacking!

Regulatory Landscape and Business Imperatives

Testing AI systems for alignment with security, safety, trustworthiness, and fairness is more than just a best practice — it is becoming a regulatory and business imperative. This practice — known as AI red teaming — helps organizations lay the foundation for trust in AI now to help avoid security and alignment failures in the future that may result in liability, reputational damage, or harm to users. 

Most recently, the European Union reached agreement on the AI Act, which sets several requirements for trust and security for AI. For some higher-risk AI systems, this includes adversarial testing, assessing and mitigating risks, cyber incident reporting, and other security safeguards.

The EU’s AI Act comes on the heels of U.S. federal guidance, such as the recent Executive Order on safe and trustworthy AI, as well as Federal Trade Commission (FTC) guidance.  These frameworks identify AI red teaming and ongoing testing as key safeguards to help ensure security and alignment. Proposed state regulations, such as those by the California Privacy Protection Agency, further emphasize the expectation that automated decision-making systems will be evaluated for validity, reliability, and fairness. In addition, the Group of Seven (G7) leaders issued statements supporting an international code of conduct for organizations developing advanced AI systems that emphasized “diverse internal and independent external testing measures.” 

At the heart of these government actions is a view that testing AI systems will better protect consumers’ privacy and reduce the risk of bias. At the same time, many private sector organizations recognize the importance of in-house testing to ensure their AI systems align with ethical norms and regulatory requirements. This approach allows organizations to fortify their systems against potential threats and align with regulatory guidelines. Private companies also utilize external AI red teaming services such as those offered by HackerOne to complement their in-house risk management efforts. This dual approach, combining internal expertise with external collaboration, showcases a commitment to fostering secure, trustworthy, and ethically aligned AI systems in the private sector.

As regulatory requirements and business imperatives surrounding AI testing become more prevalent, organizations must seamlessly integrate AI red teaming and alignment testing into their risk management and software development practices. This strategic integration is crucial for fostering a culture of responsible AI development and ensuring that AI technologies meet security and ethical expectations.

Strengthening AI Security and Reducing Bias with HackerOne

Organizations deploying AI should consider leveraging the hacker community to help secure and test AI systems for trustworthiness. Our approach to AI Red Teaming builds upon the powerful bug bounty model, optimized for AI safety engagement.

HackerOne’s bug bounty programs offer a cost-effective approach to strengthening the security of AI systems, identifying and resolving vulnerabilities before they are exploited. Simultaneously, algorithmic bias reviews help address the critical need to reduce biases and undesirable outputs in AI algorithms, aligning technology with ethical principles and societal values. 

In a rapidly evolving technological landscape, HackerOne is a steadfast partner for organizations committed to securing and aligning their AI systems with ethical norms. Our AI red teaming services not only provide powerful testing mechanisms but also empower organizations to build trust in their AI deployments. As the demand for secure and ethical AI grows, HackerOne remains dedicated to facilitating a future where technology enhances our lives while upholding security and trust. To learn more about how to strengthen your AI security with AI Red Teaming, contact the team at HackerOne.

Artificial Intelligence (AI) is poised to usher in transformative changes that can supercharge economic productivity and improve daily life. However, this immense potential comes with responsibilities to ensure that AI systems are not only secure but also align with legal requirements, expectations for trustworthiness, and prevention of bias and discrimination. That is why HackerOne offers robust AI Red Teaming services that help organizations bolster the security, fairness, and reliability of their AI deployments.

HackerOne's AI can already be used to:

1. Help automate vulnerability detection, using Nuclei, for example

2. Provide a summary of a hacker's history across many vulnerabilities

3. Provide remediation advice, including suggested code fixes

The Power of Large Language Models (LLMs)

Language is at the heart of hacking. Hackers communicate security vulnerabilities as text. Collaboration between customers, hackers, and HackerOne security analysts is text for the most part as well. Before AI, HackerOne used two parallel strategies to understand vulnerability data: feature extraction (machine learning) and creating structure where there wasn’t any (normalization). Both of these helped us build rich reporting, analytics, and intelligence.

And now Large Language Models (LLMs) give us a powerful third strategy: leveraging fine-tuning, prompt engineering, and techniques such as Retrieval-Augmented Generation (RAG) to simplify many typical machine learning tasks. Text generation, text summarization, feature and text extraction, and even text classification have become table stakes. LLMs enable us and everyone on HackerOne to increase the efficiency of existing processes significantly, and in the future it will scale the detection of security vulnerabilities, support better prioritization, and accomplish faster remediation.

HackerOne’s Approach and Principles for Responsible AI

We've been around groundbreaking technology long enough to know that there are always unintended consequences, and that everything can be hacked. We have carefully reviewed these risks in consultation with numerous customers, hackers, and other experts. Today we're ready to share those principles for further discussion.

Foundation in Large Language Models (LLMs)

At the core of our AI technology lies a foundation of state-of-the-art LLMs. These powerful models serve as the basis for how our AI interacts with the world. What sets us apart is the proprietary insight we build on top of these models, trained from real-world vulnerability information, and tailored to the specific use cases people on HackerOne engage in. By combining the strengths of foundation LLMs with our specialized knowledge and vulnerability information, we create a potent tool for discovering, triaging, validating, and remediating vulnerabilities at scale.

Data Security and Confidentiality

Security and confidentiality are embedded in our approach. We understand that customer and hacker vulnerability information is highly sensitive and must remain under their control. We do not leverage any multi-tenant or public LLMs. At no point do AI prompts or private vulnerability information leave HackerOne infrastructure or undergo transmission to any third parties.

Tailored Interactions

One size does not fit all in the world of security. We address the risk of unintended data leakage by ensuring that our AI models are tailored specifically to each customer. We do not use your private data to train our models. Rather, our approach lets you make your private data available to the model at inference time with techniques such as Retrieval-Augmented Generation (RAG). This ensures your data remains secure, confidential, and private to you and your interactions only.

Human Agency

Finally, we have instilled a governing principle requiring the deployment of AI with strong human-in-the-loop oversight. We believe in human-AI collaboration, where technology serves as a copilot, enhancing the capabilities of security analysts and hackers. Technology is a tool, not a replacement for the invaluable human expertise.

And, as with all technology we develop, AI is within the scope for our bug bounty program.

What’s Next

Far too often throughout history, emerging technologies are developed with trust, safety, and security as afterthoughts. We are changing the status quo. We are committed to enhancing security through safe, secure, and confidential AI, while tightly coupled with strong human oversight. Our goal is to provide people with the tools they need to achieve security outcomes beyond what has been possible today—without compromise.

We have started rolling out our models to customers and security analysts already. Over the next few months, we will expand this to everyone, including hackers. We're beyond excited to start sharing with you more details on the specific use cases we're focused on enhancing with AI.

Welcome to the future of hacking!

By Jobert Abma, Co-founder and Principal Software Engineer and Alex Rice, Co-founder and CTO

Generative Artificial Intelligence (GenAI) is ushering in a new era of how humans leverage technology. At HackerOne, we are combining human intelligence with artificial intelligence at scale to improve the efficiency of people and unlock entirely new capabilities. This blog will go over our approach and our principles to ensure model safety.

In an era where data breaches and cyberattacks dominate headlines, a new and unconventional approach to cybersecurity has emerged, challenging traditional notions of protection. Ethical hacking, also known as hacking for good, is rapidly gaining prominence as organizations seek innovative strategies to safeguard their digital assets. This approach involves companies hiring skilled hackers to intentionally breach their systems, identify vulnerabilities, and fortify defenses.

On a recent episode of Amazon’s “Conversations With Leaders,” Marten Mickos, CEO of HackerOne, sat down to discuss the evolving landscape of cybersecurity, the challenges organizations face, and the innovative strategies employed to build robust security cultures. 

Marten believes the essence of hacking for good lies in harnessing external hackers to identify vulnerabilities in web systems and mobile apps, enabling companies to rectify these issues before malicious actors exploit them. This “good force against bad force” approach promotes a proactive stance in enhancing security.

Ethical hacking represents a paradigm shift in cybersecurity philosophy. Organizations embrace proactive and collaborative tactics instead of relying solely on reactive measures to counteract threats. By welcoming skilled hackers into their ranks, they aim to detect weaknesses before malicious actors can exploit them.

Ethical hackers, often called “white hat”, operate with integrity and a robust code of conduct. Their mission is to expose security vulnerabilities and potential entry points within an organization’s digital infrastructure. Unlike malicious hackers, ethical hackers use their skills for constructive purposes, ultimately enhancing the security posture of the organizations they engage with.

Challenges are associated with hiring and retaining skilled security professionals in this industry. According to Marten, the solution is to create an environment where employees find meaning, autonomy, and opportunities for growth. A culture that nurtures career development and offers purposeful work can attract and retain top talent.

The Hacker Community: A Vast Pool of Expertise

A critical element that sets ethical hacking apart is its emphasis on collaboration. Ethical hackers often form communities that share knowledge, techniques, and best practices. These communities foster a supportive environment that encourages continuous learning and skill development. Organizations benefit not only from individual ethical hackers’ expertise but also from the collective knowledge of the broader community.

Companies like HackerOne have capitalized on this collaborative model, acting as intermediaries between organizations and ethical hackers. Organizations can post bug bounties through their platform, rewarding hackers who successfully identify vulnerabilities. This approach incentivizes hackers to participate in uncovering weaknesses, creating a win-win scenario for both parties.

With many potential security measures available, organizations need help prioritizing their actions effectively. Marten recommends adopting a risk-based approach focusing on essential actions aligned with business objectives.

Fostering a Positive Security Culture

While ethical hacking might sound counterintuitive, its value is increasingly evident. Data breaches and cyberattacks can result in significant financial losses, reputational damage, and legal ramifications. By investing in ethical hacking, organizations take proactive steps to prevent these scenarios. Identifying vulnerabilities before they are exploited can save companies millions of dollars in recovery costs and potential fines.

Marten draws parallels between cybersecurity and the airline industry’s safety practices. There is an emphasis on fostering a blameless culture, where mistakes are treated as learning opportunities rather than causes for retribution. This promotes open communication and rapid issue resolution.

Marten believes that the need to transform security from a roadblock to an enabler of business growth is critical for hacking for a good approach to be successful. By promoting a positive view of security, organizations can encourage employees to participate in security initiatives actively. CEOs should set the tone by highlighting security’s role in enabling business success.

Cybersecurity’s asymmetric nature demands a different approach than the standard business practices used in most organizations. Collaboration with external hackers allows organizations to tap into an immense pool of expertise that can help identify vulnerabilities quickly. This method provides flexibility and rapid access to diverse skills, ensuring a well-rounded security posture.

A Future of Enhanced Cybersecurity

As the hacking for good industry gains momentum, it reshapes how organizations approach cybersecurity. The emphasis on collaboration, transparency, and a proactive defense departs from the traditional reactive model. Ethical hacking is a testament to the power of harnessing skilled individuals for the greater good — using their expertise to strengthen digital fortifications, safeguard sensitive data, and propel the cybersecurity industry into a new era of resilience.

In an increasingly interconnected world, ethical hackers are emerging as unsung heroes, leveraging their talents to prevent data breaches and protect the digital foundations of modern society. As organizations continue to navigate the complex realm of cybersecurity, ethical hacking stands as a beacon of innovation and a testament to the remarkable potential of technology when used for positive and transformative purposes.

To hear the full “Conversations with Leaders” episode, click here.

Originally published on the Amazon Web Services Conversations With Leaders podcast blog.

Future Risk Predictions

In a recent presentation at Black Hat 2023, HackerOne Founder, Michiel Prins, and hacker, Joseph Thacker aka @rez0, discussed some of the most impactful risk predictions related to Generative AI and LLMs, including:

• Increased risk of preventable breaches 
• Loss of revenue and brand reputation
• Increased cost of regulatory compliance
• Diminished competitiveness
• Reduced ROI on development investments

Hacker Herman Satkauskas also points out that, while AI has lowered the barrier to entry for ethical hackers, “malicious attackers will also realize they have the tools at their disposal to conduct cybercrime.”

The Top Generative AI and LLM Risks According to Hackers

According to hacker Gavin Klondike, “We’ve almost forgotten the last 30 years of cybersecurity lessons in developing some of this software.” The haste of GAI adoption has clouded many organizations’ judgment when it comes to the security of artificial intelligence. Security researcher Katie Paxton-Fear aka @InsiderPhD, believes, “this is a great opportunity to take a step back and bake some security in as this is developing and not bolting on security 10 years later.”

Prompt Injections

The OWASP Top 10 for LLM defines prompt injection as a vulnerability during which an attacker manipulates the operation of a trusted LLM through crafted inputs, either directly or indirectly. Paxton-Fear warns about prompt injection, saying:

“As we see the technology mature and grow in complexity, there will be more ways to break it. We’re already seeing vulnerabilities specific to AI systems, such as prompt injection or getting the AI model to recall training data or poison the data. We need AI and human intelligence to overcome these security challenges.”

Thacker uses this example to help understand the power of prompt injection:

“If an attacker uses prompt injection to take control of the context for the LLM function call, they can exfiltrate data by calling the web browser feature and moving the data that are exfiltrated to the attacker’s side. Or, an attacker could email a prompt injection payload to an LLM tasked with reading and replying to emails.”

Ethical hacker, Roni Carta aka @arsene_lupin, points out that if developers are using ChatGPT to help install prompt packages on their computers, they can run into trouble when asking it to find libraries. Carta says, “ChatGPT hallucinates library names, which threat actors can then take advantage of by reverse-engineering the fake libraries.”

According to Thacker, “The jury is out on whether or not it’s solvable, but personally, I think it is.” He says the mitigation depends on the implementation and deployment of the prompt injection and, “of course, by testing.”

Agent Access Control

“LLMs are as good as their data,” says Thacker. “The most useful data is often private data.”

According to Thacker, this creates an extremely difficult problem in the form of agent access control. Access control issues are very common vulnerabilities found through the HackerOne platform every day. Where access control goes particularly wrong regarding AI agents is the mixing of data. Thacker says AI agents have a tendency to mix second-order data access with privileged actions, exposing the most sensitive information to potentially be exploited by bad actors.

The Evolution of the Hacker in the Age of Generative AI

Naturally, as new vulnerabilities emerge from the rapid adoption of Generative AI and LLMs, the role of the hacker is also evolving. During a panel featuring security experts from Zoom and Salesforce, hacker Tom Anthony predicted the change in how hackers approach processes with AI:

“At a recent Live Hacking Event with Zoom, there were easter eggs for hackers to find — and the hacker who solved them used LLMs to crack it. Hackers are able to use AI to speed up their processes by, for example, rapidly extending the word lists when trying to brute force systems.” 

He also senses a distinct difference for hackers using automation, claiming AI will significantly uplevel the reading of source code. Anthony says, “Anywhere that companies are exposing source code, there will be systems reading, analyzing, and reporting in an automated fashion.”

Hacker Jonathan Bouman uses ChatGPT to help hack technologies he’s less confident with. 

“I can hack web applications but not break new coding languages, which was the challenge at one Live Hacking Event. I copied and pasted all the documentation provided (removing all references to the company), gave it all the structures, and asked it ‘Where would you start?’ It took a few prompts to ensure it wasn’t hallucinating, and it did provide a few low-level bugs. Because I was in a room with 50 ethical hackers, I was able to share my findings with a wider team, and we escalated two of those bugs into critical vulnerabilities. I couldn't have done it without ChatGPT, but I couldn’t have made the impact I did without the hacking community.”

There are even new tools for the education of hacking LLMs — and therefore for identifying the vulnerabilities created by them. Anthony uses “an online game for prompt injection where you work through levels, tricking the GPT model to give you secrets. It’s all developing so quickly.”

How AI Shows the Value of Bug Bounty

It’s no secret that security leaders are faced with the challenging task of articulating the value of their security programs to stakeholders and board members. And one of the most tricky pieces of showcasing that value is comparing how much a bug bounty costs against how much that bug would cost in the hands of a cybercriminal. 

Our hacker community is using AI to prove that value. According to Satkauskas,

“I tried an experiment where I load up the security finding to ChatGPT and ask it ‘How much would this vulnerability cost a company if it was in the wrong hands?’ ChatGPT can provide a ballpark estimate, meaning it’s far easier to make a case for the impact of that finding in your report.”

According to the 7th Annual Hacker Powered Security Report, the average bug bounty across industries is $1,000 and $3,700 for high and critical vulnerabilities. When you consider the potential financial impact of GenAI-facilitated loss data, you can start to estimate the real value of ethical hackers experiements in GenAI to secure your organization.

Use the Power of Hackers for Secure Generative AI

Even the most sophisticated security programs are unable to catch every vulnerability. HackerOne is committed to helping organizations secure their GAI and LLMs and to staying at the forefront of security trends and challenges. With HackerOne, organizations can:

• Secure the use of GenAI and LLMs with community-driven AI Red Teaming
• Conduct continuous adversarial testing through Bug Bounty
• Perform targeted hacker-based testing with Challenge
• Assess an entire application with Pentest or Code Security Audit

Contact us today to learn more about how we can help take a secure approach to Generative AI.

Generative AI has undergone incredibly fast adoption, with fresh launches of the latest large language model (LLM) coming every day. As with any new technology, however, we often don’t understand the risk implications before rushing to build it into our applications. 

Ethical hackers understand the ins and outs of the security issues inherent in Generative AI, and they’ve been exploring the common mistakes made by organizations rushing to leverage the technology. The 7th Annual Hacker Powered Security Report reveals that 53% of ethical hackers are using GenAI in some way, and 66% are using or intend to use GenAI to write better reports. Who better to learn from when it comes to preventing and managing risks than the hackers who know how to exploit them?

We’ve spoken with several experienced hackers in the space to get their perspectives on the most important considerations for Generative AI and cybersecurity.

HackerOne CEO, Marten Mickos, emailed the following note to employees on August 2, 2023.

H1 Team,

I have made the painful and necessary decision to undertake a restructuring and we will reduce the size of our team by up to approximately 12%. This comes as disappointing news, as we've all built strong connections with our fellow Hackeronies. These actions are necessary to be successful long-term. However, I understand how difficult this news is and the impact this will have on all team members, and I take responsibility for the changes we are sharing today.

How are we handling departures? 
If you are impacted, we aim to notify you as soon as we can while complying with each region's applicable rules and regulations. Impacted employees in the U.S. and Canada will receive a meeting invitation in the next 15 minutes. 

It is expected that the reorganization will also affect employees in the U.K., the Netherlands, and other countries. After completing the relevant consultation proceedings, we will be able to inform all people involved with more detail. These processes will take longer.

We are offering severance packages to impacted employees that include cash compensation and non-cash benefits. Please monitor your email for country, role-specific information, and other details. 

What do we see in our business?
HackerOne, like many tech companies, has been navigating the global economic situation and the resulting shifts in our market. Our strategic plan includes investment in new product families and expanding the capabilities of our platform to expand into the enterprise. A little over a year ago, I authorized the continued hiring of new employees to fulfill this strategy. 

However, we did not anticipate the degree to which the overall economic situation is affecting us, with smaller companies running out of money and larger ones taking longer to make purchasing decisions. The new products we brought to market didn’t perform the way we wanted them to. Our bets on hiring and new products proved to be too big, and we must now restructure our teams to be successful in the future. 

We’ve designed this reduction in force as a one-time event. We don’t claim to have perfect visibility into our future financial performance or the macroeconomic climate, but we unequivocally wanted to take a single action and move forward with confidence.

How do we move forward?
HackerOne remains a category leader. We are one of the most important contributors to cybersecurity worldwide. We plan to be better, stronger, and faster when an improved business climate ultimately emerges. 

My goal is to make every Hackeronie proud, no matter when and for how long they worked at the company. We win as a team, so we also lose as a team. This decision marks a loss. Those remaining in the company are tasked with creating a bigger win over time. I remain as committed and confident as ever.

You will have many questions for me and the leadership team now. An honest question will receive an honest answer. I asked all department leaders to communicate with your team about the impact on workload and meet to discuss next steps. 

The decision is painful to me because we hired each one of you to pursue our strategy and build an outstanding business. We have a clear mission and a strong company culture. Each one of you is a devoted team member, investing your time and mind into the success of HackerOne. Now as we must say goodbye to some of you, I want to express my deep appreciation of your unwavering commitment to the company.

Marten

Any inquiries, please send to press@hackerone.com.

Offensive AI Will Outpace Defensive AI

In the short term, and possibly indefinitely, we will see offensive or malicious AI applications outpace defensive ones that use AI for stronger security. This is not a new phenomenon for those familiar with the offense vs. defense cat-and-mouse game that defines cybersecurity. While GAI offers tremendous opportunities to advance defensive use cases, cybercrime rings and malicious attackers will not let this opportunity pass either and will level up their weaponry, potentially asymmetrically to defensive efforts, meaning there isn’t an equal match between the two. 

It’s highly possible that the commoditization of GAI will mean the end of Cross-Site Scripting (XSS) and other current common vulnerabilities. Some of the top 10 most common vulnerabilities — like XSS or SQL Injection — are still far too common, despite industry advancements in Static Application Security Testing (SAST), web browser protections, and secure development frameworks. GAI has the opportunity to finally deliver the change we all want to see in this area.

However, while advances in Generative AI may eradicate some vulnerability types, others will explode in effectiveness. Attacks like social engineering via deep fakes will be more convincing and fruitful than ever. GAI lowers the barrier to entry, and phishing is getting even more convincing. 

Have you ever received a text from a random number claiming to be your CEO, asking you to buy 500 gift cards? While you’re unlikely to fall for that trick, how would it differ if that phone call came from your CEO’s phone number? It sounded exactly like them and even responded to your questions in real-time. Check out this 60 Minutes segment with hacker, Rachel Tobac, to see it unravel live. 

The strategy of security through obscurity will also be impossible with the advance of GAI. HackerOne research shows that 64% of security professionals claim their organization maintains a culture of security through obscurity. If your security strategy still depends on secrecy instead of transparency, you need to get ready for it to end. The seemingly magical ability of GAI to sift through enormous datasets and distill what truly matters, combined with advances in Open Source Intelligence (OSINT) and hacker reconnaissance, will render security through obscurity obsolete.

Attack Surfaces Will Grow Exponentially

Our second prediction is that we will see an outsized explosion in new attack surfaces. Defenders have long followed the principle of attack surface reduction, a term coined by Microsoft, but the rapid commoditization of Generative AI is going to reverse some of our progress. 

Software is eating the world, Marc Andreessen famously wrote in 2011. He wasn’t wrong — code increases exponentially every year. Now it is increasingly (or even entirely) written with the help of Generative AI. The ability to generate code with GAI dramatically lowers the bar of who can be a software engineer, resulting in more and more code being shipped by people that do not fully comprehend the technical implications of the software they develop, let alone oversee the security implications.

Additionally, GAI requires vast amounts of data. It is no surprise that the models that continue to impress us with human levels of intelligence happen to be the largest models out there. In a GAI-ubiquitous future, organizations and commercial businesses will hoard more and more data, beyond what we now think is possible. Therefore, the sheer scale and impact of data breaches will grow out of control. Attackers will be more motivated than ever to get their hands on data. The dark web price of data “per kilogram” will increase. 

Attack surface growth doesn’t stop there: many businesses have rapidly implemented features and capabilities powered by generative AI in the past months. As with any emerging technology, developers may not be fully aware of the ways their implementation can be exploited or abused. Novel attacks against applications powered by GAI will emerge as a new threat that defenders have to worry about. A promising project in this area is the OWASP Top 10 for Large Language Models (LLMs). (LLMs are the technology fueling the breakthrough in Generative AI that we’re all witnessing right now.)

What Does Defense Look Like In A Future Dominated By Generative AI?

Even with the potential for increased risk, there is hope. Ethical hackers are ready to secure applications and workloads powered by Generative AI. Hackers are characterized by their curiosity and creativity; they are consistently at the forefront of emerging technologies, finding ways to make that technology do the impossible. As with any new technology, it is hard for most people, especially optimists, to appreciate the risks that may surface — and this is where hackers come in. Before GAI, the emerging technology trend was blockchain. Hackers found unthinkable ways to exploit the technology. GAI will be no different, with hackers quickly investigating the technology and looking to trigger unthinkable scenarios — all so you can develop stronger defenses.

There are three tangible ways in which HackerOne can help you prepare your defenses for a not-too-distant future where Generative AI is truly ubiquitous:

• HackerOne Bounty: Continuous adversarial testing with the world’s largest hacker community will identify vulnerabilities of any kind in your attack surface, including potential flaws stemming from poor GAI implementation. If you already run a bug bounty program with us, contact your Customer Success Manager (CSM) to see if running a campaign focused on your GAI implementations can help deliver more secure products.
• HackerOne Challenge: Conduct scoped and time-bound adversarial testing with a curated group of expert hackers. A challenge is ideal for testing a pre-release product or feature that leverages generative AI for the first time. 
• HackerOne Security Advisory Services: Work with our Security Advisory team to understand how your threat model will evolve by bringing Generative AI into your attack surface, and ensure your HackerOne programs are firing on all cylinders to catch these flaws.

Want to hear more? I’ll be speaking on this topic at Black Hat on Thursday, August 10 at Booth #2640, or request a meeting. Check out the Black Hat event page for details. 

Generative Artificial Intelligence (GAI) is popping up in all manner of software every day. It's a trend we're seeing unfold right now, characterized by a firehose of daily announcements of new AI-powered products and capabilities. Many businesses, including HackerOne customers like Snapchat, Instacart, CrowdStrike, Salesforce, and many others, have all announced AI-powered features and user experiences. GAI capabilities will soon be table stakes for any software company as their customers will simply expect it. Those who do not take advantage of this technological evolution will decline into irrelevancy and be replaced by better and more productive alternatives. For example, users will expect to just talk directly to their reports and dashboards instead of figuring out yet another query language. 

A world where Generative AI is ubiquitous will soon be here. What does that mean for security? We have two main predictions.