How to Set Up a Local Ubuntu Server to Host Ollama Models with a WebUI
#ubuntu#ai#ollama
Korak Kurani

Korak Kurani @korak997

About: Full-stack developer with 4+ years' experience in JavaScript, AWS. Passionate about building scalable apps, creative designs, and writing about JS patterns, AWS, and software principles.

Location:
Berlin
Joined:
Jun 11, 2024

How to Set Up a Local Ubuntu Server to Host Ollama Models with a WebUI

Publish Date: Dec 14 '24
17 4

Are you excited to create a powerful local server to host Ollama models and manage them through an intuitive WebUI? This step-by-step guide will walk you through the entire process—from installing Ubuntu Server to setting up Ollama and integrating OpenWebUI for a seamless experience.

Whether you’re new to this or a seasoned Ubuntu user, this detailed tutorial is designed to be clear, approachable, and error-free. Let’s dive in and get your server up and running!

Installing Ubuntu Server on Your PC

Before we configure your server, you’ll need to install Ubuntu Server on your machine. Here’s how to do it smoothly:

Step 1: Download the Ubuntu Server ISO

  1. Head over to the Ubuntu Server Download Page.
  2. Grab the latest Ubuntu Server ISO file to ensure you’re working with the most recent release.

Step 2: Create a Bootable USB Drive

To turn your USB into a bootable installer, use a tool like Rufus (for Windows) or dd (for Linux/Mac):

  • Using Rufus: Launch Rufus, select your downloaded ISO file and USB drive, then hit "Start."
  • Using dd on Linux/Mac: 
   sudo dd if=/path/to/ubuntu-server.iso of=/dev/sdX bs=4M status=progress
Enter fullscreen mode Exit fullscreen mode

Be sure to replace /dev/sdX with your USB device’s identifier (e.g., /dev/sdb).

Step 3: Boot from USB and Install Ubuntu Server

  1. Plug the USB drive into your PC and restart the machine.
  2. Access your BIOS/UEFI settings by pressing a key like DEL, F2, or F12 during startup (check your device’s manual).
  3. Set the USB drive as the first boot option, save your changes, and exit.
  4. Follow the installer’s prompts to set up Ubuntu Server:
    • Choose your preferred language and keyboard layout.
    • Configure your network settings.
    • Partition your disk (the guided option works well for most users).
    • Create a username, password, and hostname for your server.

Once the installation finishes, reboot your system and remove the USB drive during the restart.

Configuring Your Ubuntu Server

With Ubuntu Server installed, let’s get it ready for action.

Step 1: Update and Install Essential Packages

Keep your system current and equipped with key tools by running these commands:

sudo apt update && sudo apt upgrade -y
sudo apt install build-essential dkms linux-headers-$(uname -r) software-properties-common -y
Enter fullscreen mode Exit fullscreen mode

These updates and packages ensure a solid foundation for what’s next.

Step 2: Install Docker

Since we’ll use Docker later to run OpenWebUI, let’s install it now to streamline the process:

  • Add Docker’s official GPG key and repository: 
   sudo apt update
   sudo apt install ca-certificates curl -y
   sudo install -m 0755 -d /etc/apt/keyrings
   sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
   sudo chmod a+r /etc/apt/keyrings/docker.asc
   echo \
     "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
     $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
     sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
   sudo apt update
Enter fullscreen mode Exit fullscreen mode
  • Install Docker and its components: 
   sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
Enter fullscreen mode Exit fullscreen mode
  • Verify Docker is running: 
   sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

You should see it’s active. If not, start it with:

   sudo systemctl enable --now docker
Enter fullscreen mode Exit fullscreen mode
  • (Optional) Allow your user to run Docker without sudo: 
   sudo usermod -aG docker $USER
Enter fullscreen mode Exit fullscreen mode

Log out and back in for this to take effect.

With Docker installed, you’re ready to containerize applications like OpenWebUI later on.

Step 3: Add NVIDIA Repository and Install Drivers

If your server has an NVIDIA GPU, you’ll want to install the proper drivers for optimal performance:

  • Add the NVIDIA PPA repository: 
   sudo add-apt-repository ppa:graphics-drivers/ppa -y
   sudo apt update
Enter fullscreen mode Exit fullscreen mode
  • Check for the recommended driver: 
   ubuntu-drivers devices
Enter fullscreen mode Exit fullscreen mode

You’ll see output like:

   driver   : nvidia-driver-560 - third-party non-free recommended
Enter fullscreen mode Exit fullscreen mode
  • Install the recommended driver (e.g., version 560): 
   sudo apt install nvidia-driver-560 -y
   sudo reboot
Enter fullscreen mode Exit fullscreen mode
  • Confirm the driver is working: 
   nvidia-smi
Enter fullscreen mode Exit fullscreen mode

This should display your GPU details and driver version. If it doesn’t, retrace your steps.

Step 4: Configure NVIDIA GPU as Default

If your system has an integrated GPU, you’ll need to prioritize the NVIDIA GPU:

  • List your GPUs: 
   lspci | grep -i vga
Enter fullscreen mode Exit fullscreen mode
  • Disable the integrated GPU by blacklisting its driver: 
   sudo nano /etc/modprobe.d/blacklist-integrated-gpu.conf
Enter fullscreen mode Exit fullscreen mode

Add these lines depending on your integrated GPU:

For Intel:

   blacklist i915
   options i915 modeset=0
Enter fullscreen mode Exit fullscreen mode

For AMD:

   blacklist amdgpu
   options amdgpu modeset=0
Enter fullscreen mode Exit fullscreen mode
  • Apply changes and reboot: 
   sudo update-initramfs -u
   sudo reboot
Enter fullscreen mode Exit fullscreen mode
  • Double-check with: 
   nvidia-smi
Enter fullscreen mode Exit fullscreen mode

Installing and Configuring Ollama

Now, let’s set up Ollama to host your AI models locally.

Step 1: Install Ollama

Run this simple command to download and install Ollama:

curl -fsSL https://ollama.com/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Step 2: Add Models to Ollama

Ollama supports various models. For example, to pull the llama3 model, use:

ollama pull llama3
Enter fullscreen mode Exit fullscreen mode

Feel free to explore other models that suit your needs!

Setting Up OpenWebUI for Easy Interaction

To elevate your Ollama experience, let’s add OpenWebUI—a sleek, user-friendly interface—using the Docker we installed earlier.

  • Launch OpenWebUI with Docker: 
   sudo docker run -d --network=host -v open-webui:/app/backend/data \
       -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
       --name open-webui --restart always \
       ghcr.io/open-webui/open-webui:main
Enter fullscreen mode Exit fullscreen mode

This command:

  • Runs OpenWebUI in a container with persistent data storage (open-webui volume).
  • Connects it to Ollama via the specified base URL.

  • Ensures it restarts automatically if your server reboots.

    • Open your browser and navigate to your server’s IP address to access the WebUI.

Testing and Troubleshooting

Let’s make sure everything’s working smoothly.

Verify NVIDIA GPU Functionality

Run:

nvidia-smi
Enter fullscreen mode Exit fullscreen mode

If you see GPU details, you’re good to go. If you get Command not found, revisit the driver installation steps.

Common Errors and Fixes

  • Error: ERROR:root:aplay command not found

    • Fix: Install the missing audio utilities: 
    sudo apt install alsa-utils -y

  • Error: udevadm hwdb is deprecated. Use systemd-hwdb instead.

    • Fix: Update your system: 
    sudo hwdb update
sudo apt update && sudo apt full-upgrade -y

Optional: CUDA Setup for Compute Workloads

For heavy computational tasks, add CUDA tools:

  • Install CUDA: 
   sudo apt install nvidia-cuda-toolkit -y
Enter fullscreen mode Exit fullscreen mode
  • Verify it’s working: 
   nvcc --version
Enter fullscreen mode Exit fullscreen mode

Wrap-Up

Congratulations! You’ve successfully built a robust local Ubuntu server, installed Docker, set up Ollama for AI model hosting, and paired it with OpenWebUI for effortless interaction. This setup is ideal for exploring AI models in a secure, local environment.

If you hit any snags, revisit the steps or check the official documentation. Now, go enjoy the power of Ollama and OpenWebUI—happy experimenting!

Comments 4 total

  • Stu De
    Stu DeJan 28, 2025

    I think you missed a section about installing Docker

    and maybe opening a firewall port to allow access to the webui from your local network and adding the --listen flag to webui.py

    • Korak Kurani
      Korak KuraniJan 31, 2025

      You are totaly right thanks for the note :)

      I will update the blog as soon as i can.

  • joja
    jojaFeb 8, 2025

    How would you make it go into a low energy mode after 15 minutes ollama stops the model? And wake up on a remote connection?

    • Korak Kurani
      Korak KuraniFeb 19, 2025

      That is a good idea. I did not added anything like that because its only used by me then i turn off the server completly whenever not in use.

Add comment