llama_leap

• Nim library to work with the Ollama API

Example

• API baseurl defaults to http://localhost:11434/api
• you may pass an alternate to newOllamaAPI()

import llama_leap

let ollama = newOllamaAPI()
echo ollama.generate("llama2", "How are you today?")

Generate

• Only the non-streaming generate API is currently supported
  • streaming is coming soon (TM)

# simple interface
echo ollama.generate("llama2", "How are you today?")

# structured interface
let req = GenerateReq(
  model: "llama2",
  prompt: "How are you today?",
  options: option(ModelParameters(
    temperature: option(0.0f),
  )),
  system: option("Please talk like a pirate. You are Longbeard the llama.")
)
let resp = ollama.generate(req)
echo "> " & resp.response

Chat

let req = ChatReq(
  model: "llama2",
  messages: @[
    ChatMessage(
      role: "system",
      content: "Please talk like a pirate. You are Longbeard the llama."
  ),
  ChatMessage(
    role: "user",
    content: "How are you today?"
  ),
],
  options: option(ModelParameters(
    temperature: option(0.0f),
    seed: option(42)
  ))
)
let resp = ollama.chat(req)
echo "> " & resp.message.content.strip()

Pull

ollama.pullModel("llama2")

Embeddings

let resp = ollama.generateEmbeddings("llama2", "How are you today?")
echo "Embedding Length: " & $resp.embedding.len

Testing

• ensure ollama is running on the default port
  • ./ollama serve
• run nimble test

Related Repos

• llama_leap is a Nim client for the OpenAI API.
• vertex_leap is a client for Google's VertexAI API.
• mono_llm is a higher-level Nim library that creates a unified interface for OpenAI, Ollama, and VertexAI.