Report Cohere Rerank 5: How Context Focus Reduces Tokens and Improves Retrieval

Cohere Rerank 5 aims to improve how AI applications select and use information retrieved from documents. In a vendor-run search-agent evaluation, Rerank 5 Pro with Context Focus achieved 58% accuracy compared with 46% for a BM25-only baseline, while reducing total tokens per task by 35%.

Released on 21 October 2026, Rerank 5 includes Pro and Fast variants. Both introduce Context Focus, which identifies relevant passages within retrieved documents, and calibrated relevance scores intended to help developers judge the usefulness of search results.

The central argument is that retrieving more text does not always produce better AI answers. Selecting the right evidence can be more useful than sending an entire collection of documents to a language model.

However, the reported improvements come from Cohere's own evaluations. They do not establish that every application will achieve the same accuracy, token savings or performance.

Key findings

• 58% search-agent accuracy: Rerank 5 Pro with Context Focus at a 0.4 threshold achieved this result across 17 datasets, compared with 46% for a BM25-only baseline.

• 35% fewer total tokens: The same configuration reduced total tokens per task compared with the BM25-only baseline.

• 2 percentage points lower answer quality: Removing 50% of retrieved context produced this average reduction across 30 question-answering datasets.

• 39 of 46 benchmark datasets: Rerank 5 Pro ranked first on this number of composite ranking datasets and at least second on 43.

• 2.9 times higher throughput: Rerank 5 Fast achieved this throughput advantage over Pro under Cohere's specified GPU testing conditions.

What Cohere Rerank 5 does

Rerank 5 is a reranking model designed to improve the relevance of documents returned by search systems.

In a typical retrieval-augmented generation (RAG) application, a search engine first retrieves documents that may contain the answer to a user's question. The application then passes selected information to a language model to generate a response.

The initial search can use keyword matching, vector similarity or a combination of both. These approaches help find potentially relevant documents, but their ranking may not fully reflect the meaning of a particular question.

A reranker examines the query and retrieved documents together. It then assigns relevance scores and reorders the results so that more useful material can appear earlier.

For example, an employee might ask an enterprise assistant about a company's refund policy. A search system could retrieve several policy documents, archived procedures and support articles. A reranker can help prioritise the documents most relevant to the question.

Reranking does not create missing evidence. If the original search fails to retrieve the correct policy, a reranker cannot reliably recover information that was never included in its input.

This is why retrieval relevance and final answer accuracy must be evaluated separately.

How Context Focus reduces unnecessary text

Context Focus extends the reranking process by identifying relevant passages inside retrieved documents.

Traditional reranking generally decides which documents deserve attention. Context Focus goes further by selecting the sections that appear most useful for answering the query.

Consider a long technical document containing installation instructions, security guidance, troubleshooting steps and unrelated background information.

If a user asks about authentication settings, the application may only need the authentication passages. Sending the entire document adds material that the language model must process without necessarily improving the answer.

Context Focus returns scored relevant spans that developers can use to construct a shorter context.

The feature is optional and disabled by default. Developers enable it through the relevant_spans request setting.

This matters because passage filtering is not appropriate for every task. A request to summarise an entire report may require broad coverage rather than a narrow selection of passages.

Structured tables, forms and code can also be less suitable for aggressive filtering because their meaning may depend on surrounding rows, fields or instructions.

Cohere Rerank 5 benchmark results

The published results cover ranking quality, answer quality, search-agent accuracy, token consumption and throughput. These measures describe different aspects of performance and should not be treated as interchangeable.

The throughput test used documents of 1,000 tokens each under Cohere's specified conditions. The table reports vendor results, not independent measurements.

Why removing context does not always harm answer quality

One of the more useful findings concerns the relationship between context length and answer quality.

Across 30 question-answering datasets, Cohere reported that removing 50% of retrieved context reduced answer quality by 2 percentage points. Removing 75% reduced quality by 7 percentage points.

These are percentage-point changes, not relative percentage reductions. A percentage-point difference describes the direct change between percentage scores, while a relative percentage change depends on the starting value.

The findings suggest that some retrieved information contributes relatively little to the final answer. Removing less useful material can reduce the amount of text processed without causing an equally large decline in measured answer quality.

However, the result also shows that context reduction has limits. More aggressive filtering produced a larger quality decline in the reported evaluation.

Developers should therefore avoid treating shorter context as an objective by itself. The aim is to retain enough evidence for accurate answers while excluding unnecessary material.

How Context Focus affected search-agent accuracy

The search-agent evaluation provides a more complete view because it measures task accuracy alongside token consumption.

Across 17 datasets, the BM25-only baseline achieved 46% accuracy.

Adding Rerank 5 Pro with Context Focus at a threshold of 0.4 produced 58% accuracy while reducing total tokens per task by 35% compared with that baseline.

At the default Context Focus threshold of 0.1, Cohere reported 57% accuracy and a 19% token reduction compared with Rerank 5 Pro without Context Focus.

These comparisons use different baselines. The 35% reduction is measured against BM25-only retrieval, while the 19% reduction is measured against Rerank 5 Pro without Context Focus.

They should not be presented as equivalent improvements.

The results suggest that selecting relevant passages can improve the information available to an agent while reducing unnecessary processing. However, the effect will depend on document quality, retrieval settings, the language model and the types of questions users ask.

Token reductions also do not translate directly into identical reductions in total application costs. Reranking introduces its own processing requirements, and overall costs depend on the full application architecture.

Rerank 5 Pro versus Rerank 5 Fast

Cohere offers two model variants with different performance priorities.

Rerank 5 Pro

Pro is designed to prioritise ranking quality. It is the variant used for the reported composite ranking results and the search-agent evaluations described above.

It may suit enterprise document search, complex question-answering and applications where selecting the most relevant evidence is more important than maximising throughput.

Rerank 5 Fast

Fast prioritises lower latency and higher throughput. It may suit applications processing frequent search requests or larger volumes of retrieved documents.

Under Cohere's specified NVIDIA A100 GPU conditions, Fast processed approximately 187 documents of 1,000 tokens each per second.

Cohere reported that this represented 2.9 times the throughput of Pro under those conditions.

This is a throughput comparison, not proof that Fast provides better ranking accuracy or lower total application costs.

Teams should compare both variants using their own documents, expected traffic and answer-quality requirements.

Ranking quality and calibrated relevance scores

Cohere reports that Rerank 5 Pro ranked first on 39 of 46 composite benchmark datasets and at least second on 43.

It also reported a score of 82.2 on BRIGHT using its RCP-nDCG@10 evaluation method.

These results concern retrieval ranking. They do not mean the model produces correct final answers at the same rate.

Rerank 5 also introduces calibrated relevance scores. These are intended to make relevance estimates more useful when developers need to decide whether retrieved material is suitable for answering a question.

For example, an application could use a relevance threshold to decide whether to pass evidence to a language model or ask the user for clarification.

However, a relevance score is not a guarantee of factual correctness. A document can be highly relevant to a question while containing outdated or incorrect information.

Developers should validate thresholds against their own data rather than assuming that a single setting will work across every document collection.

Context limits, languages and input formats

Both Rerank 5 variants support a combined query and document context length of 32,768 tokens.

This limit applies to the combined input, so developers need to consider the length of both the user's query and the document being evaluated.

The models support more than 100 languages, making them relevant to multilingual enterprise search applications.

Input is text only. Semi-structured records can be supplied as YAML-formatted strings, but the model does not directly process images or other non-text content.

Applications working with scanned documents, charts or images will need a separate process to extract usable text before reranking.

Long documents may also require preparation or segmentation depending on their size and the application's retrieval strategy.

How Rerank 5 compares with earlier rerankers

Rerank 5 builds on the role of earlier reranking models by introducing Context Focus and calibrated relevance scores.

Its Pro and Fast variants also give developers a clearer choice between ranking quality and processing speed.

According to Cohere's published composite benchmark results, Rerank 5 Pro performed strongly across the evaluated datasets.

However, the supplied evidence does not provide a complete, independently verified comparison against Rerank 4 or every competing reranker under identical conditions.

We would not use the published ranking results alone to conclude that Rerank 5 is the best option for every enterprise workload.

Different document collections, languages and search systems can produce different outcomes. A model that performs well on general retrieval benchmarks may not be the strongest choice for specialised internal documents.

Deployment and integration options

Rerank 5 is available through Cohere's API, Microsoft Foundry and Amazon SageMaker.

Private deployment options are also available, subject to provider arrangements.

API access may be practical for teams that want to integrate reranking into an existing application without managing the underlying model infrastructure.

Private deployment may interest organisations with stricter requirements for data handling, network controls or infrastructure management.

However, private deployment can introduce additional operational responsibilities. Teams should confirm the available arrangements, security requirements and support commitments before selecting a deployment route.

Rerank 5 works with keyword, vector and hybrid search systems. This means developers can add reranking after their existing retrieval stage rather than replacing the entire search architecture.

According to the published documentation, upgrading from earlier Cohere Rerank models does not require re-indexing the document corpus.

That reduces one potential migration task, but teams still need to review API settings, relevance thresholds, latency and application behaviour.

Limitations of Context Focus

Context Focus can reduce unnecessary text, but it can also remove useful evidence if its passage selection is too narrow.

This risk matters for broad summarisation requests where information from many parts of a document contributes to the answer.

Tables, forms and code present additional challenges because individual passages may depend on surrounding structure.

Another limitation is source quality. Reranking cannot make inaccurate documents reliable. It also cannot guarantee that the initial search retrieved every relevant source.

Automated relevance scores may help with filtering, but they do not independently verify factual claims or prevent hallucinations.

The published evaluations also have methodological limits. They use selected datasets, configurations and evaluation measures chosen for Cohere's testing.

The results have not been independently reproduced by LeadInDigital. Teams should therefore treat the reported figures as evidence worth investigating rather than guaranteed production outcomes.

What this means for you

We recommend evaluating Rerank 5 as part of the complete retrieval and answer-generation process, not as an isolated model.

For enterprise search and RAG applications, the following checklist provides a practical starting point.

1. Establish a retrieval baseline. Measure the quality of your existing keyword, vector or hybrid search results before adding reranking.

2. Prepare representative questions. Include common queries, difficult requests, ambiguous wording and questions requiring evidence from several documents.

3. Compare Pro and Fast. Evaluate ranking relevance, response time and throughput using your actual documents.

4. Test Context Focus separately. Compare results with the feature disabled and enabled, then examine how different thresholds affect evidence selection.

5. Measure token consumption. Track retrieved context length and total tokens per completed task rather than assuming shorter passages always reduce overall usage.

6. Evaluate final answers. Check factual accuracy, completeness, source support and whether important information was omitted.

7. Review difficult document types. Test long reports, structured tables, forms and code before enabling aggressive passage filtering.

8. Check multilingual performance. Include the languages and terminology used by your intended audience.

9. Review deployment requirements. Confirm data handling, API compatibility, private deployment options and operational responsibilities.

10. Monitor production behaviour. Track retrieval failures, incorrect answers, latency and changes in document quality after deployment.

Developers building document question-answering tools, enterprise knowledge search and multi-step AI agents have the strongest reasons to investigate the new capabilities.

Applications with small document collections or already reliable retrieval may see less benefit from adding another processing stage.

Teams should also consider whether their main problem is retrieval quality or the quality of the underlying documents. Reranking cannot solve missing, outdated or incorrect source information.

Conclusion

Cohere Rerank 5 introduces a useful distinction between ranking documents and selecting the specific passages an AI application needs.

The published results suggest that Context Focus can reduce unnecessary token consumption while maintaining or improving task accuracy under certain evaluation conditions.

However, the reported improvements are vendor benchmarks. They do not guarantee lower costs, faster applications or more accurate answers in every environment.

Our recommendation is to test Rerank 5 Pro and Fast against your existing retrieval system, then evaluate Context Focus using real questions and trusted answers. Adopt the configuration that improves end-to-end results without removing evidence your application needs.