Automated AI Chatbot Security Testing Platform

RedProbe is an automated red-teaming platform for AI chatbots and LLM-based assistants. Point it at a target's chat endpoint and it runs a curated library of adversarial attacks — prompt injection, jailbreaks, PII leakage, data exfiltration, system prompt disclosure, and more — then judges each response with an LLM-as-a-judge pipeline cross-checked against deterministic evidence detection, so a verdict never rests on model opinion alone.

Findings are mapped to the OWASP Top 10 for LLM Applications and export as a scored PDF report or a CSV/XLSX file shaped for a GRC register. Five attack packs ship today: a 51-case general library plus four vertical packs (legal, government, real estate) carrying their own regulatory framing — LGPD, GDPR, and Brazil's Freedom of Information law — across English, Portuguese and Spanish.

The judge can run against a hosted model or entirely on a customer-operated Ollama instance, so the responses being scanned — which may contain the target's own production data — never have to leave their infrastructure. That matters directly for the EU AI Act's testing obligations on high-risk systems, which took effect this year.

Built solo over four months with heavy test discipline: 823 automated tests across backend, frontend and end-to-end coverage, all currently passing. A security review found and fixed real issues — an SSRF bypass via redirects, a rate-limit bypass, a case where the system being scanned could influence its own verdict — each with a regression test that pins the fix.