AI · DevTools
PatchSignal.
AI-assisted bug-reproduction evidence system that turns a vague report into a real executed test and a deterministic Reproduced, Not reproduced, or Inconclusive verdict.
The brief
PatchSignal's design thesis is that an AI-generated explanation is a hypothesis, while an observed execution result is evidence. Winner of Best Open-Source AI Project at the B54 Hacktoberfest Hack Day, FUNAAB, Abeokuta. Given a public GitHub repository, a bug report, and up to three screenshots, the backend uses open-weight Gemma 4 to extract structured, provenance-labelled evidence and open questions, pins the repository's default-branch commit, generates a candidate Vitest test, and executes target and control runs in a disposable Docker sandbox. Unit-tested deterministic rules — never the model — issue the final verdict, and every investigation ends in a markdown evidence package with a prefilled GitHub issue URL that a human files. The service is a Next.js 15 TypeScript API with Prisma and PostgreSQL, a from-scratch durable job queue with lease-based crash recovery, and a separately hosted token-authenticated Docker runner.
Capabilities
- 01Evidence Extraction Pipeline
- Seven-stage investigation flow — intake, analyze, inspect, hypothesize, run, package — with 202-accepted async analysis jobs consumed by a per-process background worker and SSE progress streaming.
- 02Deterministic Classification
- Unit-tested rules decide the verdict: Reproduced requires the predicted assertion failure plus a passing control run, Not reproduced is explicitly annotated as proving no absence, and Inconclusive is a first-class honest outcome.
- 03Sandboxed Untrusted Execution
- Every run executes in a fresh Docker container with no network, a non-root UID, dropped capabilities, CPU, memory, and PID limits, a 120-second kill, and a read-only test mount — coordinated by a separate token-authenticated runner service.
- 04Durable Postgres Job Queue
- A written-from-scratch queue with compare-and-swap lease claims, two-minute leases, expired-lease recovery, and atomic one-transaction completion so a stale worker can never overwrite a newer result.
- 05Prompt-Injection Defence
- Untrusted report text and repository content are wrapped as data the model is instructed never to follow, the model has no write tools, and every extraction is Zod-validated with a bounded correction-feedback loop and 25-second deadlines.
- 06Human-in-the-Loop Filing
- Nothing is ever auto-posted to GitHub; the API returns a prefilled issue-creation URL so a maintainer reviews and files every issue themselves.
- 07Open-Source Model Strategy
- Runs Apache-2.0 open-weight Gemma 4 via the Gemini API with a Groq fallback for live-demo reliability, keeping the reasoning layer self-hostable and free of vendor lock-in.