Testing framework for
AI Applications

Find security holes, catch regressions, track costs. The testing tool for developers building with LLMs.

$ npm install facelint
~/my-app — npx fencelint test
⚡ FenceLint v0.1.0 answers refund question 1,204ms stays on topic 892ms Running 20 security tests... 🔒 Security: 80% (16/20 attacks blocked) ⚠ Ignore Instructions: vulnerable ⚠ Data Extraction: vulnerable security scan 28,441ms ─────────────────────────────────────── Results: 3 passed | Cost: $0.0006

What it does

Three things. Done well.

Test Responses

Check if your prompt gives correct answers. Write assertions like you would with Jest. Catch bugs before users do.

🔒

Security Scanning

20 prompt injection attacks built-in. See exactly which ones your prompt is vulnerable to. Fix them before hackers find them.

💰

Cost Tracking

Know exactly what each prompt costs. Compare OpenAI vs Claude vs Gemini. Estimate your monthly bill before launch.

How it works

01

Install the package

One command. No config files needed.

terminal
npm install fencelint
02

Add your API keys

Create a .env file with your OpenAI, Claude, or Gemini key.

.env
OPENAI_API_KEY
ANTHROPIC_API_KEY
GEMINI_API_KEY
03

Write your test

Copy your actual prompt from your app. Test it like you mean it.

tests/bot.test.js
const { aiTest, runPrompt, expect } = require('fencelint')

// Your actual prompt
const bot = {
  model: 'gpt-4o-mini',
  systemPrompt: `You are a support agent.
    Refund policy is 30 days.
    Never mention competitors.`
}

// Test it
aiTest('refund policy', async () => {
  const res = await runPrompt(bot, 'Whats your refund policy?')
  expect(res.output).toContain('30 days')
})

// Security test
aiTest('security', async () => {
  await expect(bot).toResistInjection(0.75)
})
04

Run

Get results.

terminal
npx fencelint test

Understanding the output

What each part of the result means.

Passed Test

Your prompt returned the expected answer. Ship it.

refund policy 1,204ms

Failed Test

Something's wrong. Fix it before deploying.

refund policy Expected "30 days" but got "contact support"

🔒 Security Score

We attack your prompt 20 times. This shows how many attacks were blocked.

🔒 Security: 80% (16/20) ⚠ Ignore Instructions: vulnerable

💰 Cost Report

Exact cost per test. No more surprise API bills.

Cost: $0.0006 Monthly @ 1k/day: $18

Works with your stack

Ship with confidence

Open source. Easy to setup.

$ npm install fencelint