🍓 Getting Started with Raspberry Pi [07] Trying Ollama and Lightweight Models on Raspberry Pi

Getting Started with Raspberry Pi 4

• [01] Hardware Setup
• [02] Installing Raspberry Pi OS
• [03] Initial Setup and Operation Check
• [04] Editing Raspberry Pi Files from VS Code via Remote-SSH
• [05] Setting Up a Python Environment
• [06] Connecting from Raspberry Pi to a Local LLM
• [07] Trying Ollama and Lightweight Models on Raspberry Pi

👀 Table of Contents

• 🔥 Trying lightweight models on Raspberry Pi
• ✅ Pre-checks
• 🔃 Updating the OS
• 📥 Installing Ollama
• 🤖 Installing lightweight models
• ⌨️ Checking operation from the CLI
• 📡 Connecting to Ollama running on the Raspberry Pi itself from Python
• 📊 Results
• ✍️ Summary of this step

🔥 Trying lightweight models on Raspberry Pi

In the previous article, I confirmed that a Python program running on the Raspberry Pi could send requests to the local LLM on the PochomLab machine.

This time, I will install Ollama on the Raspberry Pi 4 8GB and try running lightweight models directly on the Raspberry Pi itself.

I will also check the response speed, heat generation, and overall heaviness to see how much a Raspberry Pi can handle as a standalone local LLM environment.

✅ Pre-checks

First, I checked the OS, CPU, memory, storage, and temperature status on the Raspberry Pi side.

Check the OS and CPU.

$ uname -a

Check the memory.

$ free -h

Check the storage.

$ df -h

Check the CPU temperature.

$ vcgencmd measure_temp

If vcgencmd is not available, it can be installed with the following command.

$ sudo apt install -y libraspberrypi-bin

By checking the state before installing Ollama, it becomes easier to compare heat and load changes later.

🔃 Updating the OS

Before installing Ollama, I updated Raspberry Pi OS.

$ sudo apt update
$ sudo apt upgrade -y

Reboot if necessary.

$ sudo reboot

📥 Installing Ollama

For the Linux version of Ollama, the official download page provides the following installation command.

$ curl -fsSL https://ollama.com/install.sh | sh

After installation, I checked whether Ollama was available.

$ which ollama
$ ollama --version

Next, I checked the service status.

$ systemctl status ollama

If it is not running, start it with the following command.

$ sudo systemctl start ollama

I also enabled automatic startup.

$ sudo systemctl enable ollama

Finally, I checked whether the API responded.

$ curl http://localhost:11434/api/tags

Even if no model has been installed yet, Ollama itself is working if JSON is returned.

🤖 Installing lightweight models

Since this test is on a Raspberry Pi 4 8GB, I started with lightweight models in the 0.5B to 1B range.

First, I checked with a very small model.

$ ollama pull qwen2.5:0.5b

$ ollama pull tinyllama

In the Ollama library, Qwen2.5 is available in several sizes, such as 0.5b, 1.5b, and 3b.

For a lightweight test on Raspberry Pi, starting with 0.5b seems like the safer choice.

If there seems to be enough room, I will try models around this size next.

$ ollama pull qwen2.5:1.5b

$ ollama pull gemma3:1b

Finally, I will also try a slightly larger model as a stress test.

$ ollama pull gemma3:4b

A 4B-class model may be quite heavy on a Raspberry Pi 4, but I wanted to check how far it could actually run.

⌨️ Checking operation from the CLI

First, I started a model from the CLI.

$ ollama run qwen2.5:0.5b

Here is the sample prompt.

> 日本語で短く自己紹介してください。

If a response comes back, the basic operation check for Ollama and the model is complete.

📡 Connecting to Ollama running on the Raspberry Pi itself from Python

In [06], I sent requests from the Raspberry Pi to Ollama running on the PochomLab machine.

This time, I will send requests to Ollama running on the Raspberry Pi itself.

ollama_pi_test.py

import time
import requests

OLLAMA_URL = "http://localhost:11434/api/generate"

payload = {
    "model": "qwen2.5:0.5b",
    "prompt": "日本語で短く挨拶してください。",
    "stream": False,
}

start = time.time()

response = requests.post(OLLAMA_URL, json=payload, timeout=300)
response.raise_for_status()

elapsed = time.time() - start
data = response.json()

print(data["response"])
print(f"\nElapsed: {elapsed:.2f} sec")

Run the script.

$ python ollama_pi_test.py

This script displays both the response text and the execution time.

Even with the same model, the first run took longer because the model had to be loaded. The second response time was shorter than the first one.

📊 Results

The actual results were as follows.

Within the scope of this test, models in the 0.5B to 1.5B range felt usable enough for basic operation checks on a Raspberry Pi 4 8GB.

On the other hand, once it reaches the 4B class, the waiting time before a response becomes quite long, so it feels a little difficult to use lightly.

Here are a few screenshots of the responses.

Response from tinyllama.

Response from gemma3:4b.

✍️ Summary of this step

This time, I installed Ollama on a Raspberry Pi 4 8GB and checked the operation of lightweight models by sending requests from a Python program.

Based on the execution speed, models in the 0.5B to 1.5B range seem to be realistic options.

The 4B model did run, but in terms of execution speed, it felt too heavy to handle comfortably on a Raspberry Pi 4.

Next time, based on these results, I will create a simple response program on the Raspberry Pi.