# `Arcana.Reranker.LLM`
[🔗](https://github.com/georgeguimaraes/arcana/blob/main/lib/arcana/reranker/llm.ex#L1)

LLM-based re-ranker that scores chunk relevance in a single batched call.

Sends all chunks to the LLM in one prompt, gets back JSON scores (0-10),
then filters by threshold and sorts by score descending.

## Cost

One LLM call per search, on the critical path. Measured on a ~230 document
collection at `limit: 8` with `over_fetch: 3`: 16-35 seconds per search against
0.2-0.8 seconds unreranked. That is fine for a batch or background path and
usually too slow for an interactive one - `Arcana.Reranker.CrossEncoder` and
`Arcana.Reranker.ColBERT` score locally if you want reranking without the round
trip.

Also note the threshold filters: it defaults to 7, so chunks the LLM scores
below that are dropped from the results rather than moved down. See
`Arcana.Reranker` for what that trade means.

## Which LLM it uses

In order: `:llm` in these opts (whether passed beside `:reranker` or inside its
option list), then `config :arcana, llm: ...`. `Arcana.Pipeline` supplies
`ctx.llm` through the first. It raises only when neither is set.

## Usage

    # With Arcana.Pipeline (uses ctx.llm automatically)
    ctx
    |> Pipeline.search()
    |> Pipeline.rerank()
    |> Pipeline.answer()

    # Directly
    {:ok, reranked} = Arcana.Reranker.LLM.rerank(
      "What is Elixir?",
      chunks,
      llm: &my_llm/1,
      threshold: 7
    )

## Custom prompt

Pass a `:prompt` function with arity 2 receiving `(question, passages)` where
passages is a list of `{id, chunk}` tuples:

    prompt_fn = fn question, passages ->
      # Build your own prompt using the question and passages
    end

    LLM.rerank("question", chunks, llm: llm, prompt: prompt_fn)

---

*Consult [api-reference.md](api-reference.md) for complete listing*