> ## Content Index
> Fetch the complete content index at: https://blog.sudowl.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Ollama — A Local LLM for Sysadmin Tasks
- URL: https://blog.sudowl.com/ollama-local-llm-sysadmin/
- Published: 2026-06-21T10:00:00.000Z
- Updated: 2026-10-11T08:47:37.000Z
- Description: Run a local LLM with Ollama on Linux — install, pull models, use the HTTP API from scripts, and analyze logs offline without sending data to a cloud provider.
- Author: Lukas
- Tags: AI, Linux

I use Claude for most of my scripting and analysis, and I'm not switching. But there's a specific case where a *local* model wins: when the data can't leave the box. Customer logs, an internal config full of hostnames and keys, a client's incident dump — I'm not pasting that into a web form, full stop. For that, Ollama runs a decent model on my own hardware, offline, and the data never touches anyone's API. It's the right tool for "good enough, and private", not "the smartest possible answer".

**TL;DR:**

```bash
# one-line install
curl -fsSL https://ollama.com/install.sh | sh

# pull a model and chat
ollama pull llama3.1:8b
ollama run llama3.1:8b "explain this ufw rule: -A ufw-user-input -p tcp --dport 22 -j DROP"

# or hit the local API from a script
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b", "prompt": "summarize:", "stream": false
}'

```

Details below. This pairs with how I use hosted models in [/how-i-use-claude-powershell-scripts/](https://blog.sudowl.com/how-i-use-claude-powershell-scripts/).

## Install and first run

The install script sets Ollama up as a systemd service listening on `127.0.0.1:11434`:

```bash
curl -fsSL https://ollama.com/install.sh | sh
systemctl status ollama          # it's already running

```

Pull a model — this downloads the weights once, then it's local forever:

```bash
ollama pull llama3.1:8b          # ~4.7 GB, solid general model
ollama pull qwen2.5-coder:7b     # better at reading configs/code
ollama list                      # what you've got

```

`ollama run <model> "<prompt>"` gives you a one-shot answer; `ollama run <model>` with no prompt drops you into an interactive chat. That's the whole basic UX.

## Hardware, honestly

This is where expectations meet reality. The rule of thumb: a model needs roughly its file size in free RAM (or VRAM on a GPU).

- **7–8B models** (llama3.1:8b, qwen2.5-coder:7b) run on any box with \~8 GB free RAM. On CPU only, expect a few tokens per second — usable for a paragraph, tedious for an essay.
- **A consumer GPU** (even an 8 GB card) makes those same models genuinely snappy. Ollama auto-detects NVIDIA/AMD and offloads.
- **70B models** want \~40 GB+ and a real GPU. Don't bother on a typical VPS; you'll wait minutes per answer.

For sysadmin summarizing and Q&A, an 8B model on a machine with a mid-range GPU is the sweet spot. Don't chase the biggest model — chase the one that answers in seconds.

## The API is the useful part

The CLI is a toy compared to the HTTP API. Everything Ollama does is exposed at `localhost:11434`, so you can pipe real work through it from bash or PowerShell. Set `"stream": false` for scripts so you get one JSON blob instead of a token stream:

```bash
curl -s http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b",
  "prompt": "Explain what this cron line does: 0 */6 * * * /opt/sync.sh",
  "stream": false
}' | jq -r .response

```

Because it's OpenAI-API-compatible at `/v1/chat/completions`, most tools and SDKs that speak OpenAI point at it with just a base-URL change — handy for wiring it into something you already have.

## The case it's actually for: offline log analysis

Here's the workflow I keep coming back to — hand a model a log excerpt and ask what's wrong, without any of it leaving the machine:

```bash
#!/usr/bin/env bash
# whatsup.sh — summarize the last hour of a service's journal, locally
LOG=$(journalctl -u "$1" --since "1 hour ago" --no-pager | tail -n 200)

curl -s http://localhost:11434/api/generate -d "$(jq -n --arg log "$LOG" '{
  model: "llama3.1:8b",
  prompt: ("You are a Linux sysadmin. Summarize errors and likely causes:\n\n" + $log),
  stream: false
}')" | jq -r .response

```

```bash
./whatsup.sh nginx

```

`jq -n` builds the JSON so newlines and quotes in the log get escaped properly — hand-concatenating strings into that `-d` payload breaks the moment a log line contains a quote. This is the local mirror of the hosted-model pipeline in [/ai-log-analysis-claude-pipe/](https://blog.sudowl.com/ai-log-analysis-claude-pipe/); use the local one when the logs are sensitive, the hosted one when you want the sharpest read.

## Gotchas

- **It's not Claude.** An 8B local model is fine for summarizing, explaining a command, or a first-pass triage. It will confidently invent flags and hallucinate config syntax. Verify anything it tells you to *run*. Local means private, not infallible.
- **First token is slow, then it warms up.** The model loads into RAM on first use and stays for a few minutes (`OLLAMA_KEEP_ALIVE`). A cold call feels broken; it isn't.
- **RAM is the wall, not disk.** Pulling a model you can't fit just makes it swap and crawl. Match model size to free memory, not free disk.
- **Don't expose 11434 to the network naively.** There's no auth. If you must reach it from another host, put it behind a reverse proxy with a token — see [/nginx-reverse-proxy-setup/](https://blog.sudowl.com/nginx-reverse-proxy-setup/) — don't just bind it to 0.0.0.0.
- **CPU-only 70B is a trap.** It technically runs. You'll age waiting. Stay at 7–13B without a GPU.

## Where it fits

Local models aren't a replacement for a good hosted one — they're the answer to "this data can't go to a cloud". Keep both: Ollama for the private, offline, good-enough jobs, and a hosted model for the hard problems. If you want to go further and give a model real tools instead of just chat, that's the MCP path in [/build-mcp-server-sysadmin-tools/](https://blog.sudowl.com/build-mcp-server-sysadmin-tools/). Running it in a container instead? [/docker-install-ubuntu/](https://blog.sudowl.com/docker-install-ubuntu/) plus the official `ollama/ollama` image gets you the same thing isolated.

---

### Related posts

- [Piping Logs Through Claude — AI Log Triage That Actually Helps](https://blog.diengdoh.com/ai-log-analysis-claude-pipe/?ref=blog.sudowl.com)
- [The Sysadmin Prompt Cheat Sheet — AI Commands That Actually Work](https://blog.diengdoh.com/sysadmin-ai-prompt-cheatsheet/?ref=blog.sudowl.com)
- [Proxmox + ZFS — A Storage Pool That Survives a Disk Death](https://blog.diengdoh.com/proxmox-zfs-storage-pool-setup/?ref=blog.sudowl.com)