Category 22 · 33 resources

Ai Security / Agent Security / Llm Testing

Security testing for AI systems themselves: red-team LLMs, scan agents for vulnerabilities, and benchmark prompt-injection and jailbreak resistance.

awesome-ai-security-tools GitHub

https://github.com/scadastrangelove/awesome-ai-security-tools/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

awesome-ai-security GitHub

https://github.com/ottosulin/awesome-ai-security/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

garak GitHub

https://github.com/NVIDIA/garak/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

PyRIT GitHub

https://github.com/microsoft/PyRIT/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

promptfoo GitHub

https://github.com/promptfoo/promptfoo/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

promptfoo Web

https://www.promptfoo.dev/

Promptfoo

How to use it

LLM evaluation and red-teaming CLI — test prompts systematically before shipping.

Open resource →

giskard-oss GitHub

https://github.com/Giskard-AI/giskard-oss/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

agent-scan GitHub

https://github.com/snyk/agent-scan/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

NeMo-Guardrails GitHub

https://github.com/NVIDIA/NeMo-Guardrails/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

llm-guard GitHub

https://github.com/protectai/llm-guard/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

inspect_ai GitHub

https://github.com/UKGovernmentBEIS/inspect_ai/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

inspect_evals GitHub

https://github.com/UKGovernmentBEIS/inspect_evals/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

control-arena GitHub

https://github.com/UKGovernmentBEIS/control-arena/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

ai-scanner GitHub

https://github.com/0din-ai/ai-scanner/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

pentestgpt GitHub

https://github.com/greydgl/pentestgpt/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

strix GitHub

https://github.com/usestrix/strix/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

strix Web

https://www.strix.ai/

Strix

How to use it

AI-powered penetration testing agents for your own applications.

Open resource →

shannon GitHub

https://github.com/KeygraphHQ/shannon/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

keygraph Web

https://keygraph.io/open-source/

Shannon/Keygraph

How to use it

Open-source AI pentesting tools from Keygraph.

Open resource →

CAI GitHub

https://github.com/aliasrobotics/CAI/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

AI-Infra-Guard GitHub

https://github.com/Tencent/AI-Infra-Guard/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

hexstrike-ai GitHub

https://github.com/0x4m4/hexstrike-ai/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

pentagi GitHub

https://github.com/vxcontrol/pentagi/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

pentagi Web

https://pentagi.com/

PentAGI

How to use it

Fully autonomous AI pentesting system — point at your authorized scope.

Open resource →

BugTraceAI-CLI GitHub

https://github.com/BugTraceAI/BugTraceAI-CLI/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

bugtraceai Web

https://bugtraceai.com/

BugTraceAI

How to use it

AI bug-hunting CLI for web vulnerabilities.

Open resource →

pipelock GitHub

https://github.com/luckyPipewrench/pipelock/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

pipelab Web

https://pipelab.org/

PipeLock

How to use it

Egress-control tooling to constrain what AI agents can reach on the network.

Open resource →

agent-egress-bench GitHub

https://github.com/luckyPipewrench/agent-egress-bench/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

Pentest-Swarm-AI GitHub

https://github.com/Armur-Ai/Pentest-Swarm-AI/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

pentest-ai GitHub

https://github.com/0xsteph/pentest-ai/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

ironclaw GitHub

https://github.com/nearai/ironclaw/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

nono GitHub

https://github.com/always-further/nono/

Open-source code repository

How to use it

Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.

Open resource →

Frequently asked questions

Do I need to install anything for these 22 tools?

Most entries here are GitHub projects — clone the repository and follow the README's install steps (pip, npm, go or cargo). A Kali Linux VM gives you many of them pre-installed.

Is it legal to use these resources?

Public-record lookups and defensive/research use are generally lawful, but rules vary by country and tool. Only test systems you own or have written authorization to test, respect each site's terms of service, and never use personal data unlawfully.

Where should a beginner start in this category?

Start with the web-based tools at the top of the list — they need no setup. Read each card's “How to use it” panel, run one real query, and only then move to installable tools.