Generate Texts

Generate function that returns either a GenerateResponse type or an error message. It takes a GenerateOps configuration and performs a request to the Ollama generate API.

Examples:

Basic usage without streaming:

let ops = GenerateOps
        { modelName = "llama3.2"
        , prompt = "Tell me a joke."
        , suffix = Nothing
        , images = Nothing
        , format = Nothing
        , system = Nothing
        , template = Nothing
        , stream = Nothing
        , raw = Nothing
        , keepAlive = Nothing
        }
result <- generate ops
case result of
  Left errorMsg -> putStrLn ("Error: " ++ errorMsg)
  Right response -> print response

Usage with streaming to print responses to the console:

void $
  generate
    defaultGenerateOps
      { modelName = "llama3.2"
      , prompt = "what is functional programming?"
      , stream = Just (T.putStr . response_, pure ())
      }

In this example, the first function in the $sel:stream:GenerateOps tuple processes each chunk of response by printing it, and the second function is a simple no-op flush.generate :: GenerateOps -> IO (Either String GenerateResponse)

A function to create a default GenerateOps type with preset values.

Example:

let ops = defaultGenerateOps
generate ops

This will generate a response using the default configuration.

generateJson is a higher level function that takes generateOps (similar to generate) and also takes a Haskell type (that has To and From JSON instance) and returns the response in provided type.

This function simply calls generate with extra prompt appended to it, telling LLM to return the response in certain JSON format and serializes the response. This function will be helpful when you want to use the LLM to do something programmatic.

For Example: > let expectedJsonStrucutre = Example { > sortedList = ["sorted List here"] > , wasListAlreadSorted = False > } > eRes2 <- generateJson > defaultGenerateOps > { modelName = "llama3.2" > , prompt = "Sort given list: [4, 2 , 3, 67]. Also tell whether list was already sorted or not." > } > expectedJsonStrucutre > Nothing > case eRes2 of > Left e -> putStrLn e > Right r -> print ("JSON response: " :: String, r)

Output: > ("JSON response: ",Example {sortedList = ["1","2","3","4"], wasListAlreadSorted = False})

Note: While Passing the type, construct the type that will help LLM understand the field better. For example, in the above example, the sortedList's value is written as "Sorted List here". This will help LLM understand context better.

You can also provide number of retries in case the LLM field to return the response in correct JSON in first attempt.

Input type for generate functions. This data type represents all possible configurations that you can pass to the Ollama generate API.

Example:

let ops = GenerateOps
        { modelName = "llama3.2"
        , prompt = "What is the meaning of life?"
        , suffix = Nothing
        , images = Nothing
        , format = Just "text"
        , system = Nothing
        , template = Nothing
        , stream = Nothing
        , raw = Just False
        , keepAlive = Just "yes"
        }

Result type for generate function containing the model's response and meta-information.