Reranking

Our reranking API provides partial support for Cohere-compatible rerank requests at /v1/rerank; more detail on where the compatibility is limited below.

As a scoring model, reranking costs only input tokens and has no output token usage or costs. Keep in mind that the reranker runs one full forward pass per query–document pair, so token usage scales with the query plus the sum of all candidate document tokens (image tokens included) in a request - a good reason to keep candidate lists tight.

Request Structure

Reranking takes a single query and a set of candidate documents in one request, and returns a relevance score for each. Unlike embeddings - where documents and queries land as individual requests - a rerank call bundles the query with every candidate it should be scored against.

curl -X POST https://ai.sitehost.nz/v1/rerank \
  -H "Authorization: Bearer YOUR_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_NAME",
    "query": "How tall is the Auckland Sky Tower?",
    "documents": [
      "The Auckland Sky Tower is 328 metres tall.",
      "The Beehive is 72 metres tall.",
      "The Christchurch Te Kaha Stadium is 32 metres tall."
    ],
    "top_n": 3,
    "instruction": "Given a search query, retrieve relevant candidates that answer the query"
  }'

The response is Cohere-compatible: a results array of {index, relevance_score} sorted by score descending. The index refers back to the position of the document in your request.

{
  "id": "score-sthai-9df21b2c533e54c6ceed1cad2a9b9de7",
  "model": "MODEL_NAME",
  "usage": {
    "prompt_tokens": 252,
    "total_tokens": 252
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The Auckland Sky Tower is 328 metres tall.",
        "multi_modal": null
      },
      "relevance_score": 0.8372183442115784
    },
    {
      "index": 1,
      "document": {
        "text": "The Beehive is 72 metres tall.",
        "multi_modal": null
      },
      "relevance_score": 0.002748191822320223
    },
    {
      "index": 2,
      "document": {
        "text": "The Christchurch Te Kaha Stadium is 32 metres tall.",
        "multi_modal": null
      },
      "relevance_score": 0.0007120037917047739
    }
  ]
}

Instruction Awareness

As with the embedding model, the reranker is instruction-aware. The default judging instruction is Given a search query, retrieve relevant candidates that answer the query. You can override it to steer relevance for your task (for example, code search, question answering, or a specific domain). Where the endpoint accepts it, pass an instruction field.

Image Inputs

For multimodal models, the query and/or any document can carry an image. Where the endpoint accepts structured content - the same content-block form as the embedding API - replace the plain string with an array containing an image_url block and an optional caption. This lets you, for example, rank a set of page images against a text query, or score text passages against an image query.

As with embeddings, fetching an external URL is possible but adds significant overhead, so we recommend the base64 approach where possible.

    "query": "Which building is the Auckland Sky Tower?",
    "documents": [
      [
        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,'"$IMAGE_B64"'"}},
        {"type": "text", "text": "Optional caption or page text"}
      ],
      "The Beehive is 72 metres tall."
    ]

Image tokens count towards both the 32k per-pair context limit and your input-token usage, and a single image can be a large number of tokens. This is another reason to rerank a tight shortlist rather than a large candidate set of images.

Python Client

The OpenAI client has no native rerank method, but you can use its .post() helper for the API-key/base-URL niceties (as in the embedding examples), or any HTTP client with the structure outlined above.

from openai import OpenAI

api_key = "YOUR_KEY_HERE"
base_url = "https://ai.sitehost.nz/v1"
model = "MODEL_NAME"
default_instruction = "Given a search query, retrieve relevant candidates that answer the query"

client = OpenAI(
    api_key=api_key,
    base_url=base_url,
)

def rerank(
    query: str,
    documents: list[str],
    top_n: int | None = None,
    instruction: str = default_instruction,
) -> list[dict]:
    body = {
        "model": model,
        "query": query,
        "documents": documents,
        "instruction": instruction,
    }
    if top_n is not None:
        body["top_n"] = top_n

    # cast_to=object returns the decoded JSON body as-is
    resp = client.post("/rerank", cast_to=object, body=body)
    return resp["results"]

results = rerank(
    "How tall is the Auckland Sky Tower?",
    [
        "The Auckland Sky Tower is 328 metres tall.",
        "The Beehive is 72 metres tall.",
        "The Christchurch Te Kaha Stadium is 32 metres tall.",
    ],
)
for r in results:
    print(r["index"], round(r["relevance_score"], 4))

This should rank the Sky Tower document well clear of the others:

0 0.8407
1 0.0029
2 0.0007