Ollama Tutorial: Run LLMs Locally with CLI, Cloud & Python

Ollama lets you download and run language models locally. This guide covers installation, model selection, CLI commands, and connecting a web interface.

What is Ollama?

Ollama is an open-source platform for running and managing large-language-model (LLM) packages entirely on your local machine. It bundles model weights, configuration, and data into a single Modelfile package. Ollama offers a command-line interface (CLI), a REST API, and a Python/JavaScript SDK, allowing users to download models, run them offline, and even call user-defined functions. Running models locally gives users privacy, removes network latency, and keeps data on the user’s device.

Install Ollama

Visit the official website to download Ollama https://ollama.com/. It’s available for Mac, Windows, and Linux.

If you want a browser chat interface alongside Ollama, follow the Llama and Open WebUI setup.

Linux:

curl -fsSL https://ollama.com/install.sh | sh

macOS:

brew install ollama

Windows: download the .exe installer and run it.

How to Run Ollama

Before running models, it is essential to understand Quantization. Ollama typically runs models quantized to 4 bits (q4_0), which significantly reduces memory usage with minimal loss in quality.

Recommended Hardware:

• 7B Models (e.g., Llama 3, Mistral): Requires ~8GB RAM (runs on most modern laptops).
• 13B to 30B models: require 16GB to 32GB RAM.
• 70B+ Models: Requires 64GB+ RAM or dual GPUs.
• GPU: An NVIDIA GPU or Apple Silicon (M1/M2/M3) is highly recommended for speed.

Go to the Ollama website and click on the “Models” and select the model for your test.

After that, click on the model name and copy the terminal command:

Then, open the terminal window and paste the command:

It will allow you to download and chat with a model immediately.

Ollama CLI: Core Commands

For a coding-agent example that uses this model runner, see how to connect Claude Code to Ollama.

Ollama’s CLI is central to model management. Common commands include:

• ollama pull : Download a model
• ollama run : Run a model interactively
• ollama list or ollama ls: List downloaded models
• ollama rm : Remove a model
• ollama create -f : Create a custom model
• ollama serve: Start the Ollama API server
• ollama ps: Show running models
• ollama stop : Stop a running model
• ollama help: Show help

Advanced Customization: Custom model with Modelfiles

You can “fine-tune” a model’s personality and constraints using a Modelfile. This is similar to a Dockerfile.

• Create a file named Modelfile
• Add the following configuration:

# 1. Base the model on an existing one  
FROM llama3  
# 2. Set the creative temperature (0.0 = precise, 1.0 = creative)  
PARAMETER temperature 0.7  
# 3. Set the context window size (default is 4096 tokens)  
PARAMETER num_ctx 4096  
# 4. Define the System Prompt (The AI’s “brain”)  
SYSTEM """  
You are a Senior Python Backend Engineer.  
Only answer with code snippets and brief technical explanations.  
Do not be conversational.  
"""

FROM defines the base model

SYSTEM sets a system prompt

PARAMETER controls inference behavior

After that, you need to build the model by using this command:

ollama create [change-to-your-custom-name] -f Modelfile

This wraps the model + prompt template together into a reusable package.

Then run in:

ollama run [change-to-your-custom-name]

Ollama Server (Local API)

Ollama can run as a local server that apps can call. To start the server use the command:

ollama serve

It listens on http://localhost:11434/ by default.

### Raw HTTP

import requests  
r = requests.post(  
    "http://localhost:11434/api/chat",  
    json={  
        "model": "llama3",  
        "messages": [{"role":"user","content":"Hello Ollama"}]  
    }  
)  
print(r.json()["message"]["content"])

This lets you embed Ollama into apps or services.

Python Integration

Use Ollama inside Python applications with the official library. Run these commands:

Create and activate virtual environments:

python3 -m venv .venv  
source .venv/bin/activate

Install the official library:

pip install ollama

Use this simple Python code:

import ollama  
  
# This sends a message to the model 'gemma:2b'  
response = ollama.chat(model='gemma:2b', messages=[  
  {  
    'role': 'user',  
    'content': 'Write a short poem about coding.'  
  },  
])  
  
# Print the AI's reply  
print(response['message']['content'])

This works over the local API automatically when Ollama is running.

You can also call a local server:

import requests  
r = requests.post(  
    "http://localhost:11434/api/chat",  
    json={  
        "model": "llama3",  
        "messages": [{"role":"user","content":"Hello Ollama"}]  
    }  
)  
print(r.json()["message"]["content"])

Using Ollama Cloud

Ollama also supports cloud models, which can help when a model is too large for your machine.

First, create an account on https://ollama.com/cloud and sign in. Then, inside the Models pag,e click on the cloud link and select any model you want to test.

In the models list, you will see the model with the -cloud prefix**,** which means it is available in the Ollama cloud.

Click on it and copy the CLI command. Then, inside the terminal, use:

ollama signin

To sign in to your Ollama account. Once you sign in with ollama signin, then run cloud models:

ollama run nemotron-3-nano:30b-cloud

Your Own Model in the Cloud

While Ollama is local-first, Ollama Cloud allows you to push your custom models (the ones you built with Modelfiles) to the web to share with your team or use across devices.

• Create an account at ollama.com.
• Add your public key (found in ~/.ollama/id_ed25519.pub).
• Push your custom model:

ollama push your-username/change-to-your-custom-model-name

Conclusion

Try a model that fits your machine’s memory, then connect it to the interface you prefer. Share your results or questions below.

Cheers! ;)