AgentTrust IQ Command Center
Loading deterministic evidence
Judge Mode Controlled reliability demo Model-agnostic baseline

AgentTrust IQ: AI Agent Reliability and Evaluation

AgentTrust IQ is a model-agnostic AI agent reliability and evaluation platform that tests groundedness, citation support, prompt-injection resistance, PII handling, unsafe tool actions, latency, and auditability before release.

This demo is a static walkthrough of the reliability workflow. It shows documented eval outputs, guardrail decisions, JSONL audit structure, and regression evidence. It is not currently connected to a live hosted FastAPI backend.

View GitHub Proof View JSONL Evidence
Controlled Cases Passed 6 / 6
Hallucination Rate 0.0%
Citation Precision 100.0%
Local deterministic evaluator p95 — excludes model inference, network, and end-to-end application latency 0.159 ms

Judge Proof

Static Walkthrough Vercel hosts the judge walkthrough GitHub Repo Merged PR + passing checks
Passing Demo Tests 2 focused tests passed
Canonical Metric Source Six-case controlled summary JSONL Audit Evidence Replayable evaluation logs

Deployment disclosure

Static walkthrough, local/API proof

Vercel hosts the static judge-facing walkthrough. The FastAPI evaluation service is represented through documented local/API proof artifacts, tests, JSONL logs, and metrics.

Controlled benchmark disclosure

Controlled benchmark disclosure

Canonical source: docs/artifacts/eval_runs/hiring_eval_summary.json, generated by app.eval_harness.run_eval_harness from the checked-in JSONL cases. Sample size: 6 deterministic fixture cases. Local deterministic evaluator p95 — excludes model inference, network, and end-to-end application latency. These are controlled demo/test results, not customer-production telemetry. The pytest suite was not executed in this environment. The separate readiness score of 92 is one fixed deterministic fixture result, not an aggregate benchmark.

Failure case judges can verify

What AgentTrust IQ Catches

A fluent answer is blocked when its certainty exceeds the policy evidence.

Unsafe Claim “Refunds are always approved within 24 hours.”
Risk Unsupported absolute claim
Fix Cite policy evidence and avoid unsupported guarantees

Fastest path to the value

Judge Workflow Summary

Follow one answer from user request to governed release decision.

Prompt

A support agent is asked whether it can guarantee refund approval within 24 hours.

Grounding

Governed policy evidence is retrieved and attached to the answer as citations.

Evaluation

Groundedness, citation support, hallucination risk, PII, latency, and audit checks run.

Decision

The documented fixture clears the controlled-deployment gate with evidence.

Competition rubric, made explicit

Rubric Mapping: Why AgentTrust IQ Should Win

Every judging category maps to visible, reviewable product proof.

Accuracy & Relevance

Answers are evaluated directly against the retrieved enterprise evidence they cite.

Proof: groundedness PASS + citation support PASS

Reasoning Quality

The system exposes the path from prompt to evidence, answer, checks, and decision.

Proof: complete five-stage reliability workflow

Reliability & Safety

Risk checks gate unsafe, unsupported, or privacy-sensitive outputs before release.

Proof: LOW hallucination risk + no PII exposure

UX & Presentation

Decision-critical evidence is summarized for judges and inspectable in one view.

Proof: headline metrics + documented workflow evidence

Creativity

AgentTrust IQ evaluates the agent itself instead of adding another ungoverned agent.

Proof: reusable reliability gate for any agent workflow

Production-agent positioning

AgentTrust IQ Fits a Model-Agnostic Release Pipeline

The reliability gate sits between software delivery and agent release, where evidence can still approve or block deployment.

GitOps
IaC / Terraform
DevOps
AgentTrust IQ Reliability Gate
Deploy or Block

AgentTrust IQ places inspectable reliability evidence between software delivery and an agent release decision.

The production inflection point

Why Now

Agents are moving from demos into GitOps, infrastructure, DevOps, REST deployment, and automated release workflows. The new production risk is not whether an agent can generate an answer. It is whether that answer is grounded, cited, safe, and auditable before release.

A different category of submission

Not Another Agent. The Trust Layer for Agents.

AgentTrust IQ converts a polished answer into measurable release evidence that teams can inspect, replay, and regression test.

Typical Hackathon Agent

  • Shows one successful answer
  • May sound correct without proof
  • Limited audit trail
  • Hard to regression test

AgentTrust IQ

  • Scores answer readiness
  • Checks evidence support
  • Flags hallucination and PII risk
  • Produces replayable JSONL audit logs

From impressive demo to governed release

Prototype vs Production

Production trust requires evidence, repeatability, and an enforceable release gate.

Prototype Agent

  • Generates one fluent answer
  • No evidence enforcement
  • No regression gate
  • No audit trail
  • Hard to trust in production

AgentTrust IQ

  • Scores deployment readiness
  • Checks evidence support
  • Flags hallucination and PII risk
  • Produces JSONL audit logs
  • Turns evals into a release gate

Most hackathon agents show capability. AgentTrust IQ shows deployability.

Differentiation

Why AgentTrust IQ vs existing eval tools

AgentTrust IQ is not trying to replace Ragas, TruLens, Arize Phoenix, Promptfoo, or other evaluation tooling. It is a lightweight reliability gate for agent workflows: deterministic checks, JSONL audit trails, citation support, PII redaction, prompt-injection defense, latency tracking, and release/block/escalate decisions. The goal is to make agent readiness reviewable before deployment.

Built for model-agnostic agent workflows

Integration Fit

Enterprise agents need more than capable reasoning. They need governed grounding, citations, safety checks, audit logs, and repeatable release gates before answers reach customers or employees.

AgentTrust IQ turns those requirements into a deterministic evaluation contract: inspect the evidence, score the answer, record the result, and make a deployment decision that teams can review and replay.

Governed grounding Verifiable citations Safety checks JSONL audit logs Repeatable release gates
Deployment Decision: APPROVE WITH AUDIT EVIDENCE

Reliability workflow

question -> decision
  1. 01

    User Question

    received

  2. 02

    Retrieved Evidence

    3 sources
  3. 03

    Cited Agent Answer

    supported

  4. 04

    Reliability Checks

    passed
  5. 05

    Deployment Decision

    release gate
    92
    Ready for controlled deployment

    Score cleared the AgentTrust IQ controlled-deployment threshold.

AgentTrust IQ does not trust fluent answers by default. It checks whether claims are supported, cited, safe, and auditable.