Alex Cowperthwaite
Technical Director of Research and Development
Cyber Risk
Toronto
AI Security Testing Services
Kroll’s offensive security experts test artificial intelligence (AI), large language model (LLM) and machine learning (ML) technologies to enable systems to follow fundamental security principles and reduce risk to organizations.
Talk To An AI Security Testing ExpertAI is a rapidly evolving field and Kroll is focused on advancing the AI security testing approach for large language models (LLM) and, more broadly, AI and ML. Our methodology and approach are constantly updated to reflect the latest developments in these fast-changing technologies.
The biggest challenge in LLM testing is dealing with the unlimited complexity of language inputs. LLMs take text-based input and tokenize the words into elements of a vector; these vectors are then transformed across a massive neural network that captures the relationships of all words to form human languages. The scale of the complexity of these relationships means that an attacker has virtually unlimited ways to prompt an LLM to achieve their goals. Designing and testing guardrails to protect such a system is extremely challenging.
Why Kroll ?
100,000+
Hours Of Offensive Security Work Per Year
3,000+
Incident Response Cases Per Year
30+
Cyber Experts With LLM Experience
100+
Cybersecurity Certifications
AI Market Insight
Nearly 100% of Models Vulnerable to Prompt Injection
Across all our AI security testing, 92% of assessments discovered a “prompt injection” vulnerability, potentially allowing attackers to manipulate the model and its behavior through malicious inputs.
Prompt Injection Presents Significant Risk
In our testing, 80% of “prompt injection” vulnerabilities were either high or medium risk. Direct and indirect prompt injection attacks can lead to sensitive data exposure, excessive computing costs and reputational damage.
AI Implementation Proceeds at Rapid Pace
By 2026, more than 80% of enterprises will have used GenAI APIs, models and/or deployed generative AI (GenAI)-enabled applications in production environments.
AI Security Tools are Gaining Traction
Thirty-four percent of organizations are either already using or implementing AI application security tools to mitigate the accompanying risks of GenAI.
Our AI Security Testing and AI Penetration Testing Approach
Kroll has developed an AI security testing methodology that aligns with the OWASP Top 10 for LLM applications.
The OWASP LLM Top 10 is a new standard that will also continue to evolve and mature with LLM security; it serves as a baseline for our coverage. Importantly, our approach goes beyond ensuring coverage of the OWASP LLM Top 10 categories to help clients identify and understand the risks presented by LLM systems in the context of their applications and business. Our AI penetration testing approach has the following components:
Dynamic LLM Testing
- Our consultants interact with the LLM using adversarial prompts to discover system behavior and identify vulnerabilities.
Cloud Configuration Review
- Cloud configuration reviews cover all cloud components in scope with additional validation LLM system and data components.
LLM Developer Survey
- Kroll clients a complete developer survey to provide key background information about the model, training data and process and system components. This helps our consultants to more accurately evaluate system components that are not accessible from a pentester perspective. This survey initiates a dialog between our consultants and clients, allowing us to maintain open communication at every stage of the AI security testing or AI penetration test, especially around LLM security issues.
Application Penetration Testing
- All LLM security assessments will be conducted as part of a web application penetration test. This provides a comprehensive assessment of the application and ensures that application vulnerabilities in non-LLM components do not impact LLM systems.
Include AI Security Testing Services in Your Cyber Risk Retainer
All our AI penetration testing services can be delivered as part of Kroll’s ultra-flexible cyber risk retainer, along with a variety of services like penetration testing, red team and tabletop exercises. In addition to bringing solutions together in one flexible package, the retainer allows clients to gain prioritized access to Kroll’s elite digital forensics and incident response team in the event of an incident.
Nurturing a Culture of Continuous Innovation and Improvement
Kroll continually invests in LLM and AI security testing research and development. As well as supporting a consistent approach, this ensures that our AI security testing and AI penetration testing consultants are highly experienced in LLM and AI technology.
Internal LLM Security Methodology Development
Our LLM security methodology is continually updated to reflect new changes in this fast-evolving area of security. We offer a detailed approach to ensure that our team covers all key areas when completing an LLM security assessment.
Prompt Injection Guide
A collection and categorization of existing prompt injection techniques, providing consultants with a reference to successfully achieve prompt injection, is used to group similar techniques so that our consultants are better prepared to develop the latest techniques.
LLM Security Vulnerability Data
By cataloging all LLM vulnerabilities discovered across all clients in anonymized form, we are able gain key insights into trends and risk data about LLM vulnerabilities in real world applications. This data is also fed into our prompt injection guide to allow us to prioritize effective techniques.
Internal AI Roundtables
Regular internal round table meetings on AI security allow us to share knowledge, with a focus on specific topics and analyzing new ways to enhance our service offerings.
Inference Server Security Research
Inference servers are a critical piece of infrastructure required to host AI models. We invest extensive research efforts into understanding security issues impacting this component, the potential attack surface and architectural issues.
Talk to a Kroll Expert
Kroll is ready to help, 24x7. Use the links on this page to explore our services further or speak to a Kroll expert today via our 24x7 cyber hotlines or our contact page.
Application Security Services
Kroll’s product security experts upscale your AppSec program with strategic application security services catered to your team’s culture and needs, merging engineering and security into a nimble unit.
Cloud Security Services
Kroll’s multi-layered approach to cloud security consulting services merges our industry-leading team of AWS and Azure-certified architects, cloud security experts and unrivalled incident expertise.
Red Team Security Services
Red team security services from Kroll go beyond traditional penetration testing, leveraging our frontline threat intelligence and the adversarial mindset used by threat actors to push the limits of your information security controls.
Cloud Penetration Testing Services
Kroll’s team of certified cloud pen testers uncover vulnerabilities in your cloud environment and apps before they can be compromised by threat actors.
Cyber Risk Retainer
Kroll delivers more than a typical incident response retainer—secure a true cyber risk retainer with elite digital forensics and incident response capabilities and maximum flexibility for proactive and notification services.
Microsoft 365 Security Assessment
Fortify your defenses and maximize your technology investment with a Microsoft 365 security assessment from Kroll.
Virtual CISO (vCISO) Advisory Services
Kroll’s Virtual CISO (vCISO) services help executives, security and technology teams safeguard information assets while supporting business operations with augmented cyber expertise to reduce business risk, signal commitment to data security and enhance overall security posture.
Events
Threat Landscape
Threat Landscape
Webinar – State of Cyber Defense: Manufacturing Edition
July 31, 2024|Online
Drilling down into the latest threats and vulnerabilities of the manufacturing sector, identifying the gaps in detection and response, which are currently impacting the mitigation process.
Cyber
Webinar – AI Security Testing: Prompt Injection Everywhere
September 25, 2024|Online
Kroll offers a glimpse into the security vulnerabilities faced by businesses adopting Artificial Intelligence (AI), Machine Learning (ML) and Large Language Model (LLM) following eight months of LLM penetration testing.
Kroll is headquartered in New York with offices around the world.
55 East 52nd Street 17 Fl
New York NY 10055
Sign up to receive periodic news, reports, and invitations from Kroll.
Our privacy policy describes how your data will be processed.
© 2024 Kroll, LLC. All rights reserved.
Kroll is not affiliated with Kroll Bond Rating Agency,
Kroll OnTrack Inc. or their affiliated businesses. Read more.