Local Chatbot

There are various reasons to run a chatbot locally, ranging from privacy concerns to hobby interests and custom applications. Many options for running local chatbots require a certain level of programming or installing questionable tools. However, I’ve found that Ollama simplifies the process while still providing some basic elements of customization.

Installation

To get started, go to the Ollama website and install the software. Here, I’ll focus on the Windows installation. Once the installation is complete, you’ll have a program running in the background, with a llama icon appearing among the other icons on your computer.

The first step is to select a model. I chose a small one for reasons we’ll discuss later. Open a command prompt (WIN-R then type cmd) and run:

ollama pull gemma2:2b

This command will download the model, which is a couple of gigabytes in size. You can then interact with it using:

ollama run gemma2:2b