Set Up Ollama

Ollama runs a language model as a local service on port 11434. It is the default provider for splam’s Chat panel, and the reason that panel can be the default at all: nothing you type and nothing the model reads leaves the host.

This page covers Ubuntu 24.04 and the distributions built on it, including Pop!_OS. Commands assume sudo and a systemd init, which is what you already need for splam itself.

Check the machine can run it

A local model is the one part of this setup with hardware requirements. Three commands tell you where you stand:

free -h                     # RAM: the "available" column, not "total"
df -h /usr                  # disk: models land under /usr/share/ollama
lspci | grep -iE "vga|3d"   # graphics

Size the model against available RAM. A working rule is that you want roughly twice the download size free while the model is loaded, leaving room for the context window.

Model	Download	Comfortable in
`llama3.2` (3B)	about 2 GB	8 GB RAM
`llama3.1` (8B)	about 4.7 GB	16 GB RAM
`qwen2.5:14b`	about 9 GB	32 GB RAM

Graphics decides speed, not whether it works. Ollama accelerates on NVIDIA through CUDA and on AMD through ROCm. Everything else, including Intel integrated graphics and the NPU on recent Intel laptop chips, runs on the CPU:

nvidia-smi                  # NVIDIA present if this prints a table
ls /dev/kfd                 # AMD ROCm present if this exists
lscpu | grep -o avx2        # CPU path wants AVX2, which any recent chip has

Neither of the first two existing means CPU inference, which is supported and correct, just slower. On CPU prefer the 3B model. The 8B models answer better but you wait long enough per reply that the panel stops getting used.

Install it

The official script is the supported path on Ubuntu. There is no apt repository:

curl -fsSL https://ollama.com/install.sh | sh

Read it first if that matters to you, and it should on a regulated host:

curl -fsSL https://ollama.com/install.sh | less

The script needs root and makes four changes worth knowing about before you run it on a machine you have to account for:

Change	Where
The `ollama` binary	`/usr/local/bin/ollama`
A system user and group named `ollama`	`/etc/passwd`, `/etc/group`
A `systemd` unit enabled at boot	`/etc/systemd/system/ollama.service`
Downloaded models	`/usr/share/ollama/.ollama/models`

That last one is why the disk check above looks at /usr rather than $HOME. Models are owned by the ollama user, not by you.

Confirm the service is up

The unit is started and enabled by the installer:

systemctl status ollama

Ask the service itself, which is the check that matters:

curl http://127.0.0.1:11434

It answers Ollama is running. If the unit is masked or you would rather not have a boot service, ollama serve runs the same server in the foreground and the rest of this page is unchanged.

It binds to 127.0.0.1 by default, so nothing off the host can reach it. Leave it that way. If some other tool has already claimed 11434, find it with ss -tlnp | grep 11434 before changing anything.

Pull a model

llama3.2 is what splam asks for unless told otherwise:

ollama pull llama3.2

Confirm what you have, and check the size against what you measured:

ollama list

Then talk to it directly, before involving splam at all. This separates a model problem from an app problem:

ollama run llama3.2 "reply with the single word: ready"

An answer here means the service, the model, and the hardware are all fine, and anything that goes wrong afterward is configuration.

Point splam at it

Nothing to configure. ollama is already the default provider and llama3.2 the default model, so with the chat extra installed the panel finds it:

.venv/bin/pip install -e ".[chat]"
shiny run splam.app:app

Open the Chat tab and ask something only your own admin-task notes can answer, such as “what’s the compliance note for this service?”. An answer in your own wording proves the whole path. See How-To: Chatbot for what the panel can and cannot do once it is talking.

To use a different local model, pull it and name it:

ollama pull qwen2.5:14b
SPLAM_CHAT_MODEL=qwen2.5:14b shiny run splam.app:app

Keep an eye on disk

Models accumulate. Every pull is a few GB and nothing removes them:

du -sh /usr/share/ollama/.ollama/models

ollama rm llama3.1

To keep them somewhere other than /usr, override the service environment rather than editing the unit file, which an update will overwrite:

sudo systemctl edit ollama

Add:

[Service]
Environment="OLLAMA_MODELS=/srv/ollama/models"

Then create the directory, give it to the ollama user, and restart:

sudo mkdir -p /srv/ollama/models

sudo chown -R ollama:ollama /srv/ollama

sudo systemctl restart ollama

Update it

Re-running the install script upgrades in place and keeps your models:

curl -fsSL https://ollama.com/install.sh | sh

systemctl restart ollama

ollama --version

Remove it

Uninstalling is four steps, and none of them is apt remove:

sudo systemctl disable --now ollama

sudo rm /etc/systemd/system/ollama.service /usr/local/bin/ollama

sudo rm -r /usr/share/ollama

sudo userdel ollama && sudo groupdel ollama

With the service gone, splam shows the same setup help it showed before the install. The app keeps working; only the Chat tab changes. To take the chat code path out entirely, uninstall the extra instead, as described in Turn it off for everyone.

When it doesn’t work

What you see	What it means
`Can't find locally running ollama.` in the Chat tab	The service isn’t running. `systemctl start ollama`
`Unit ollama.service could not be found`	The install script didn’t finish. Re-run it and read the output
`model "llama3.2" not found`	Installed but nothing pulled. `ollama pull llama3.2`
`ollama: command not found` after installing	`/usr/local/bin` isn’t on this shell’s `PATH`. Start a new shell
Replies take minutes	CPU inference on too large a model. Pull `llama3.2` and set `SPLAM_CHAT_MODEL`
The service dies partway through a reply	Out of memory. Check `journalctl -u ollama -n 50` for the OOM kill, then use a smaller model

The service writes to the journal like anything else on the box, so the same Logs tab you use for the rest of the system works here:

journalctl -u ollama -n 100 --no-pager