# Ollama Holes

![image](https://github.com/user-attachments/assets/649ffcd2-0560-47d6-bbbe-74bae08cbb70)

## Introduction
This is an example of a typed-hole plugin for GHC that uses the [Ollama](https://ollama.com/) to host a local LLM to fill in holes in Haskell code.

Before using this plugin, make sure you have the Ollama CLI installed and the model you want to use is available.
You can install the Ollama CLI by following the instructions at [https://ollama.com/download](https://ollama.com/download),
and you can install the default model (qwen3) by running `ollama pull qwen3`.

Note that the speed and quality of the hole-fits generated by the plugin depends on
the model you use, and the default model requires a GPU to run efficiently.
For a smaller model, we suggest `gemma3:4b-it-qat`, `phi4-mini-reasoning` or `deepcoder:1.5b`,
or one of the smaller `qwen3` models, such as `qwen3:1.7b` or `qwen3:4b` or even `qwen3:0.6b`,
though results may vary.

This plugin is also availble on Hackage [https://hackage.haskell.org/package/ollama-holes-plugin](https://hackage.haskell.org/package/ollama-holes-plugin)

## Installation

1. Install [Ollama](https://ollama.com/download)
2. Install the `qwen3` model (or any other model you prefer) using the following command:

```bash

ollama pull qwen3
```
3. Clone this repository and navigate to the directory, and build the project using:

```bash
cabal build
```
4. Run the example using:

```bash
cabal build Test
```

5. Enjoy! If you want to change the underlying model, make sure to pass the model name via the plugin arguments (see example)

## Example
Given

```haskell
{-# OPTIONS_GHC -fplugin=GHC.Plugin.OllamaHoles #-}

module Main where

import qualified Data.List as L

main :: IO ()
main = do let k = (_b :: [Int] -> [String])
          print (k [1,2,3])

```

We get the following output:


```text
Main.hs:8:20: error: [GHC-88464]
    • Found hole: _b :: [Int] -> [String]
      Or perhaps ‘_b’ is mis-spelled, or not in scope
    • In the expression: _b :: [Int] -> [String]
      In an equation for ‘k’: k = (_b :: [Int] -> [String])
      In the expression:
        do let k = (_b :: [Int] -> [String])
           print (k [1, 2, ....])
    • Relevant bindings include
        k :: [Int] -> [String] (bound at Main.hs:8:15)
        main :: IO () (bound at Main.hs:8:1)
      Valid hole fits include
        map show
        pure . show
        fmap show
        L.map show
        (\xs -> map show xs)
  |
8 | main = do let k = (_b :: [Int] -> [String])
  |                    ^^
```


## Guidance 

We can also provide some guidance to the LLM, by having an identifier in scope called `_guide`,
defined as `_guide = Proxy :: Proxy (Text "<guidance")`.

Note that this requires `GHC.TypeError` and `Data.Proxy`, with `DataKinds` enabled

Given

```haskell
{-# LANGUAGE DataKinds #-}
{-# OPTIONS_GHC -fplugin=GHC.Plugin.OllamaHoles #-}
-- Use a bigger model tomake it better at following instructions
{-# OPTIONS_GHC -fplugin-opt=GHC.Plugin.OllamaHoles:model=qwen3:14b #-}
{-# OPTIONS_GHC -fplugin-opt=GHC.Plugin.OllamaHoles:n=10 #-}

module Main where

import qualified Data.List as L
import Data.Proxy
import GHC.TypeError

main :: IO ()
main = do
  let _guide = Proxy :: Proxy (Text "The function should sort the list and then show each element")
  let k = (_b :: [Int] -> [String])
  print (k [1, 2, 3])

```

We get:

```text
Main.hs:15:12: error: [GHC-88464]
    • Found hole: _b :: [Int] -> [String]
      Or perhaps ‘_b’ is mis-spelled, or not in scope
    • In the expression: _b :: [Int] -> [String]
      In an equation for ‘k’: k = (_b :: [Int] -> [String])
      In the expression:
        do let _guide = ...
           let k = (_b :: [Int] -> [String])
           print (k [1, 2, ....])
    • Relevant bindings include
        k :: [Int] -> [String] (bound at Main.hs:15:7)
        _guide :: Proxy
                    (Text
                       "The function should sort the list and then show each element")
          (bound at Main.hs:14:7)
        main :: IO () (bound at Main.hs:13:1)
      Valid hole fits include
        map show . L.sort
        \xs -> map show (L.sort xs)
        L.map show . L.sort
        \xs -> L.map show (L.sort xs)
        \xs -> [show x | x <- L.sort xs]
        \xs -> L.sort xs >>= \x -> [show x]
        (Some hole fits suppressed; use -fmax-valid-hole-fits=N or -fno-max-valid-hole-fits)
   |
15 |   let k = (_b :: [Int] -> [String])
   |            ^^
```

## Including Documentation
You can also pass the `-fplugin-opt=GHC.Plugin.OllamaHoles:include-docs`, flag,
which will lookup the Haddock documentation (if available) for the functions in scope
and provide it to the LLM. E.g. if `Data.List` is imported as `L`, the request to the
`LLM` will include

```text
...
Documentation for `L.subsequences`:
 The 'subsequences' function returns the list of all subsequences of the argument.
Documentation for `L.tails`:
 \(\mathcal{O}(n)\). The 'tails' function returns all final segments of the
 argument, longest first.
Documentation for `L.transpose`:
 The 'transpose' function transposes the rows and columns of its argument.
...
```

## Model Options

Using

```
-fplugin-opt=GHC.Plugin.OllamaHoles:model-options={\"num_ctxt\": 32000, \"temperature\": 1.0}
```

You can pass further custom options to the model, e.g. here we increase the context length
and set the temperature.


## OpenAI and Gemini backends

The plugin now supports using the OpenAI API and Gemini APIs to generate valid hole fits.
Simply set the backend flag `-fplugin-opt=GHC.Plugin.OllamaHoles:backend=openai`,
or `-fplugin-opt=GHC.Plugin.OllamaHoles:backend=gemini`, and make sure that
you have the `OPENAI_API_KEY` or `GEMINI_API_KEY` set in your environment.

To use with any other OpenAI compatible api (like groq or OpenRouter), simply set
`-fplugin-opt=GHC.Plugin.OllamaHoles:backend=openai`,
and
`-fplugin-opt=GHC.Plugin.OllamaHoles:openai_base_url=https://api.groq.com/openai`,
`-fplugin-opt=GHC.Plugin.OllamaHoles:openai_key_name=GROQ_API_KEY`,