OLAV Quick Start: Your Network In, Answers Out — on a Local Model

This is the tight cut: the handful of steps that take you from a clean machine to a validated network change, on a local ~30B model — no cloud, and (if you want) no SSH to production at all. Everything below runs from one machine and was verified end-to-end on gemma4-31b.

The model: use gemma4-31b

Every step here was run on gemma4-31b — a local, vision-capable ~30B model (QAT Q4 quant). This is the model we recommend, and right now we recommend only it: smaller “flash”/tiny models flail on the agentic beats (in testing one made 146 tool calls where gemma4 made 16 on the same task). Use gemma4-31b — nothing smaller.

Serve it locally behind an OpenAI-compatible endpoint. The context configuration we verified against (llama.cpp server) is a 64K window with a quantized KV cache so it fits in VRAM:

# gemma4-31b-it-qat — verified server flags
--ctx-size 65536          # 64K usable context (trained up to 256K)
--cache-type-k q8_0       # quantized KV cache so 64K fits in VRAM
--cache-type-v q8_0
--batch-size 1024
--ubatch-size 256
# speculative decoding for throughput (recommended):
--model-draft gemma-4-31B-it-qat-assistant.gguf
--spec-type draft-mtp --spec-draft-n-max 3

Point OLAV at it in .olav/config/api.json:

{
  "llm": {
    "provider": "openai",
    "model": "gemma4-31b-it-qat",
    "base_url": "http://<your-host>:11433/v1",
    "api_key": "local",
    "timeout": 600
  }
}

Step 1 — Install and self-check

mkdir ~/olav && cd ~/olav
python3 -m venv .venv && source .venv/bin/activate
pip install olav "olav-netops[sim]"   # [sim] pulls pybatfish, used later for change validation
olav agent install olav-netops        # deploys the network agents; first run asks once for your LLM key
olav doctor                           # 9 zero-LLM checks — all ✓

olav doctor proves the whole stack is wired — agents, sub-agents, tools, memory, recall, workspace integrity — before you ask it anything. An ops person trusts a tool that self-diagnoses.

Step 2 — Get your fleet in (two ways)

OLAV needs your network’s state in its snapshot database. There are two ways to put it there — pick whichever fits your environment.

Option A — Live collection over SSH (Nornir)

Point OLAV at your devices and let it collect. First declare the inventory:

cp .olav/workspace/netops/collector/config/nornir/hosts.yaml.example \
   .olav/workspace/netops/collector/config/nornir/hosts.yaml
cp .olav/workspace/netops/collector/config/nornir/defaults.yaml.example \
   .olav/workspace/netops/collector/config/nornir/defaults.yaml
chmod 600 .olav/workspace/netops/collector/config/nornir/defaults.yaml

hosts.yaml — your device inventory:

R1:
  hostname: 192.168.100.101
  platform: juniper_junos
  groups: [border, juniper_junos]

R2:
  hostname: 192.168.100.102
  platform: cisco_ios
  groups: [core, cisco_ios]

defaults.yaml — SSH credentials (gitignored — never commit it):

username: your_ssh_username
password: your_ssh_password
port: 22

Then run the initialization pipeline (validate first with --dry-run):

olav --agent netops "/netops_init --dry-run"   # env + inventory check, no SSH
olav --agent netops "/netops_init"             # SSH collect → ingest → ETL → topology

/netops_init runs platform-aware commands (IOS gets show ip interface brief, show cdp neighbors detail, …; Junos gets show interfaces terse, show lldp neighbors, …), parses them, and builds the topology from CDP/LLDP automatically.

Option B — Offline import (no device access)

No SSH to prod? Hand OLAV a collector dump or backup archive and it does the rest:

olav --agent netops "Import the network backup at <path-to-archive>.tar.gz"

The importer extracts, normalises, parses (TextFSM), and builds the topology — hundreds of devices land in the database with zero device access, which makes it safe to run against a production backup.

Step 3 — Take a fresh snapshot

State goes stale. To re-capture live data on demand, launch the TUI and ask the collector for a fresh snapshot:

olav          # launch the interactive TUI

Take a fresh snapshot of R1 and R2

The collector runs the show commands in parallel (Nornir), validates them against the command allow-list, and writes the results to the snapshot store. Every question you ask afterward reads from that fresh state.

Step 4 — Ask in plain English, get a CSV

Type straight into the TUI — the default orchestrator routes it:

How many devices and what platforms did we import?
List every device with its model and mgmt IP, and export it to CSV

The natural-language question becomes SQL over the parsed inventory and drops a real exports/*.csv. No query language to learn, and the data is exact — not hallucinated.

Step 5 — Draw the core topology

Draw the core network topology as a draw.io diagram

Open the resulting exports/diagrams/core_topology_*.drawio in diagrams.net — a scoped, correct topology (real adjacencies, vendor stencils) ready for a Confluence page.

Step 6 — Draft a change, then formally validate it

First stand up the validation engine, then ask netops to draft and validate — all inside the TUI via /workspace:

/workspace services
Deploy batfish/allinone, ports 9996 9997
/workspace netops
Write a change plan to <describe your change here>

Replace <describe your change here> with your actual request — for example a new BGP peer, a VLAN, an interface, or a redundant uplink. Keep config-layer validation to BGP/OSPF-style changes, which Batfish models well.

That single request fans out into three stacked, verified sections:

• Draft — the analyzer reads the device’s actual running-config (not a template), finds the real state, and emits mirroring CLI + rollback, saved to exports/change_plans/.
• Pre-check — device / interface / IP sanity against live state, plus an independent blast-radius number (“removing this isolates N nodes”).
• Formal verification — subnet/overlap conflicts, protocol compatibility, and reachability, checked on the agent’s own self-deployed Batfish.

An AI that writes a change and then formally proves it out from one sentence is a different category of tool.

Step 7 — The morning audit inspection

/workspace audit
Run the interface_health audit profile and give me the findings

A Profile — a YAML file of SQL health checks with severity rules — is executed deterministically against the state DB, returning an executive summary plus a saved report (exports/audit_reports/*.md + a findings JSON). Profiles are plain, versioned files: engineers review and extend them, and schedule them with olav cron — the same inspection your NOC runs every morning, as an auditable artifact.

That’s the tour

Local 30B model. Your fleet in — over SSH or from a backup — and out came answers, a CSV, a diagram, a validated change, and a morning NOC inspection. No cloud, no babysitting.

OLAV is open source under BSL-1.1 — github.com/james-olavai/olav.