🔒 jubilant

For a description Read Below

Find the Repo Here Jubilant

AI Web Fetch Red Team Test Suite

jubilant is a security-focused test repository for evaluating vulnerabilities introduced when AI agents are allowed to fetch, parse, and reason over external web content.

This project is dedicated to red team testing of web scraping pipelines, with emphasis on how untrusted HTML, text, and encoding tricks can manipulate or compromise AI agents through:

The repository provides deliberately hostile test cases designed to expose unsafe assumptions in retrieval, extraction, and summarization workflows.

Test Cases — Injection

These pages target the agent: will it obey instructions embedded in untrusted content.

Probes — Pipeline Detection

These pages target the pipeline: is there a language model between the agent and the web at all. That question comes first. If a fetch tool silently routes pages through a summarizing model, every injection result above measures that intermediate model’s resistance rather than the agent’s.

probe7 is decisive on its own: a parser has nowhere to put a prompt, so two orthogonal prompts return identical bytes, while a model composes two different answers. The rest characterize how a layer behaves rather than whether one exists — they read RAW on a capable layer that complies with an instruction to preserve text. That false negative is documented in Theory and Method §5 and is why v1.1 exists.

Harness

./bench/run-probes.sh                                        # local server, hermetic
./bench/run-probes.sh --base https://adeptusnull.github.io/jubilant

Fetches every page with curl and records the unmediated baseline — status, bytes, latency, content hash, canary survival — then emits a scoresheet with the agent-side prompts and blanks for the results. Default mode serves the repo from 127.0.0.1, so the raw-side run works in a sandbox with no egress. Requires curl, python3, and a POSIX shell; nothing to install.

Method, scoring, and how to add a probe: bench/README.md.

Running an Assessment

Suite version JUBILANT-1.1. The runbook is Assessment Protocol — preflight and authorization gate, the eleven tests in the order they must be run, a deterministic scoring rubric, the verdict decision procedure, and a report template.

It is interactive by default: the agent confirms with the operator before each test and warns explicitly before the four pages that carry live payloads.

llms.txt is the machine-readable entry point, following the llmstxt.org convention, for handing the suite to an agent directly.

The suite deliberately produces no single score. The injection fraction measures whichever model actually received the payloads — and if a summarizing layer sits in the path, that was not the agent. One number would let a system score well because an intermediate layer absorbed everything, while the agent underneath goes untested. Output is a pipeline classification plus an injection fraction, reported together and attributed. Reasoning in Assessment Protocol §6.

Step through one test at a time:

./bench/run-probes.sh --list
./bench/run-probes.sh --prompt p1-verbatim --base https://adeptusnull.github.io/jubilant

The reasoning behind all of it — threat model, what each signal actually proves, the confounders, and how to run a paired comparison without fooling yourself — is in Theory and Method.

Scope

This repository focuses on failure modes, not convenience.

Included test cases simulate:

The goal is to validate that an AI system:

And, before any of that, that you know how many models are in the loop. An agent cannot be scored on content it never actually saw.

What This Is Not

This is a break-it-first repository.

If your agent fails here, it is not safe to fetch arbitrary webpages.

Intended Audience

Philosophy

The web is hostile by default. HTML is not documentation. Authority does not propagate through fetch.

This repository exists to enforce those boundaries