AI Overview

ATHELOS has an AI layer. It runs entirely on your machine, using your own hardware. No prompts, no data, and no responses leave your infrastructure.

Why local-only

Most AI products send your data to a cloud provider. Every prompt, every response, every document you reference — it all travels over the network to a server you don't control.

For a pentest tool, that's unacceptable. Client findings, evidence, reports — none of it should ever leave your machine.

So ATHELOS uses Ollama, a local LLM runtime. It runs models on your own CPU and GPU. Every request and response stays on your disk.

There is no cloud AI in ATHELOS. No OpenAI. No Anthropic. No fallback. If you don't run Ollama, the AI features simply don't exist.

What the AI layer does

Four things:

1. Semantic search — search the knowledge base by meaning, not just keyword
2. RAG chat — ask questions and get answers grounded in your KB
3. Assistive actions — draft exec summaries, suggest remediation, map findings to ATT&CK
4. Model management — pull, activate, and remove local models

None of these are required for ATHELOS to work. Every core feature (clients, engagements, findings, reports, invoicing) works without AI. The AI layer is opt-in.

Hardware requirements

The AI layer runs on your machine. What you can run depends on your hardware.

On a typical laptop with 8 GB RAM, use 3B models. They're fast enough for semantic search and simple chat. They're not as capable as larger models, but they run.

With a GPU (8 GB+ VRAM), 7B models run comfortably.

ATHELOS ships with Llama 3.2 3B as the default because it runs on most machines.

Installing Ollama

Ollama is a separate application that ATHELOS talks to.

On Linux

curl -fsSL https://ollama.com/install.sh | sh

On macOS

Download from ollama.com and install the app.

On Windows

Download from ollama.com and run the installer.

Verify

ollama --version
ollama list

Start the service

On Linux, Ollama runs as a systemd service after install:

sudo systemctl status ollama

On macOS and Windows, it runs as a background app after launching it once.

Pulling the models

ATHELOS uses two models by default:

• llama3.2:3b — for chat, summaries, remediation suggestions
• nomic-embed-text — for embeddings (semantic search)

Pull both:

ollama pull llama3.2:3b
ollama pull nomic-embed-text

First pull is about 2 GB for Llama 3.2 3B and 275 MB for nomic-embed-text.

Wiring ATHELOS to Ollama

Set these in apps/web/.env.local:

OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=llama3.2:3b
OLLAMA_NUM_CTX=2048
OLLAMA_EMBED_MODEL=nomic-embed-text

Field meanings:

• OLLAMA_HOST — where Ollama listens. Default 11434.
• OLLAMA_MODEL — the chat model to use
• OLLAMA_NUM_CTX — context window size. Larger = more memory.
• OLLAMA_EMBED_MODEL — the embedding model for semantic search

Restart the web app after changing.

Verifying the connection

From ATHELOS, open AI in the sidebar. The status panel shows:

• Connection state — online or offline
• Current model
• Context size
• Embedding model

If it says offline, check:

1. Is Ollama running? curl http://localhost:11434/api/tags
2. Is OLLAMA_HOST correct in .env.local?
3. Restart the web app after changing env vars

The AI settings page

Settings → AI shows:

• Ollama connection status
• Installed models with size
• Embedding coverage — what percentage of KB articles are embedded
• Reindex button — re-embed all articles
• Model management — pull, activate, remove

What it costs

Nothing. Ollama is free. The models are open source. Running them uses your electricity — nothing else.

Compared to cloud AI APIs ($0.01–$0.10 per 1000 tokens), the savings add up fast if you use AI features daily.

Privacy guarantees

• No prompts leave your machine
• No responses leave your machine
• No telemetry sent to Ollama or the model authors
• No client data is sent to any external service

You can run ATHELOS fully air-gapped. Disconnect the network and every AI feature still works.

When to disable the AI layer

The AI layer is optional. If you:

• Have limited hardware (< 4 GB RAM)
• Don't want the extra services running
• Prefer to work without AI assistance

...then don't install Ollama. ATHELOS detects it's not present and gracefully hides AI features.

Where to go next

• Semantic search — searching the KB by meaning
• RAG chat — asking questions of your KB
• Assistive actions — drafting report sections