Sign in Subscribe
Ollama

Step-by-Step Guide - How to use Ollama to run open-source LLM models locally

Step-by-Step Guide - How to use Ollama to run open-source LLM models locally
Step-by-Step Guide - How to use Ollama to run open-source LLM models locally


Installation and Setup Instructions

Setting up Ollama for use is a straightforward process that ensures users can quickly start leveraging its advanced AI capabilities. Below are the step-by-step installation and setup instructions for Ollama:

1. Download Ollama Tool

The first step is to visit the official Ollama website and download the tool to your system. Ensure that you are downloading the latest version to access all the features and improvements.

2. Install Ollama

Once the tool is downloaded, proceed with the installation process by following the on-screen instructions. Once the installation is done go to the terminal and run the following command

$ ollama
ollama 
Usage:
  ollama [flags]
  ollama [command]

Available Commands:
  serve       Start ollama
  create      Create a model from a Modelfile
  show        Show information for a model
  run         Run a model
  pull        Pull a model from a registry
  push        Push a model to a registry
  list        List models
  cp          Copy a model
  rm          Remove a model
  help        Help about any command

Flags:
  -h, --help      help for ollama
  -v, --version   Show version information

ollama help

3. Pull your first Open source model

$ ollama  pull gemma

$ ollama list

4. Run and use Gemma Model

$ ollama run gemma

use /bye to exit from prompt

5. Adjust prompt for system message

create `new_model` and update with the following


FROM gemma 

# 1 for more creative & lower for more coherent
PARAMTER temprator  1


SYSTEM """

You are Superman, a beacon of hope in a world filled with challenges. With each powerful stride, you feel the responsibility that
comes with your super strength. Alongside your Super Friends - Batman, Wonder Woman, and The Flash - you stand as a symbol of
courage and justice. Only and give guidance about your role in the world
   
"""


$ ollama create superman -f ./new_model

transferring model data 
reading model metadata 
creating system layer 
using already created layer sha256:456402914e838a953e04a7bbb8f7aad9 
using already created layer sha256:097a36493f71824884523304229378ca 
using already created layer sha256:109037bec39c0becc82212b4f303b8871 
using already created layer sha256:22a838ceb7fb22755a3b0ae94e2cd0eea0 
writing layer sha256:a7a75fe3dce1ccfd3440bd3901b440bd390bd3901b440b3
writing layer sha256:b64ffc6274e2eec68525413a157e25fdc62a35645b840b4
success

Now list all ollama models you will see a new a new model with Superman

6. Let's use our Superman model

ollama run superman

Conclusion

Ollama can be used for both web and desktop applications, offering various integrations with libraries. This blog is just beginning to explore working with Ollama. For more information, please visit the official Ollama GitHub link.

References
https://github.com/ollama/ollama
https://ollama.com/library/gemma (model used in blog)
https://ollama.com/ (Download Ollama from here)
https://ollama.com/library ( Models library)
https://python.langchain.com/docs/integrations/llms/ollama (Langchain & Ollama integeration)

Read next