Insight AI Tax Benchmark: Best Model Passes Only 10.8% of Professional Tasks

The strongest AI model in a new professional tax benchmark passed only 10.8% of attempts. Claude Sonnet 5.5 led the DAYJOB: Tax evaluation, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%.

The benchmark, published on 27 October 2026, tests whether AI agents can complete complicated tax assignments from an initial client request to a finished deliverable. It includes 50 tasks and 1,789 supporting files.

These results do not mean that AI models answer only 10.8% of ordinary tax questions correctly. The benchmark measures complete professional assignments under a strict grading system.

An agent must examine documents, identify relevant rules, investigate missing information and produce work that meets the benchmark's requirements. Correct calculations alone are not enough.

We examined the published methodology, leaderboard and evaluation code to understand what the scores mean and how developers and businesses should interpret them.

Key findings

• 10.8% best pass rate: Claude Sonnet 5.5 achieved the highest published score on DAYJOB: Tax.

• 50 professional assignments: The benchmark covers eight areas of US tax practice.

• 1,789 supporting files: Agents must work through documents and records rather than answer isolated questions.

• 95% passing threshold: An attempt must satisfy at least this share of its task-specific grading criteria.

• Five attempts per task: The published evaluation uses repeated attempts to assess each model configuration.

What the DAYJOB: Tax benchmark tests

DAYJOB: Tax evaluates AI agents performing work similar to assignments handled by tax professionals.

Unlike a conventional question-answering benchmark, it does not simply ask a model to identify a tax rule or calculate a figure.

Instead, an agent receives a client request and access to a collection of supporting records. It must determine which documents matter, identify problems and prepare an appropriate final deliverable.

These deliverables can include tax returns, technical reviews, supporting workpapers and client advice.

The benchmark covers individual and family tax, small-business tax, partnerships, property and investments, estates and trusts, international tax, corporate transactions, and tax review and disputes.

Its tasks are designed to test whether an agent can recognise important issues even when the client does not mention them.

That makes the evaluation particularly relevant to autonomous AI systems, which are expected to perform several connected steps without detailed instructions for every decision.

AI tax benchmark results by model

The published leaderboard ranks model configurations by the share of attempts that passed the benchmark's grading requirements.

The leading configurations were from Anthropic, although their scores remained low.

The results show that none of the evaluated configurations reliably completed these professional assignments under the benchmark's requirements.

However, the differences between closely ranked models should not be exaggerated.

The evaluation uses a limited number of tasks and repeated attempts. The published material does not establish that small leaderboard differences will persist across different assignments or evaluation conditions.

The scores also reflect particular reasoning settings. They should not be treated as universal measures of the underlying models.

How the 95% passing threshold works

Each DAYJOB: Tax assignment has a detailed grading rubric.

The rubric covers matters such as calculations, supporting evidence, identification of consequential issues and the completed deliverable.

An attempt passes when it satisfies at least 95% of the applicable criteria.

This is a demanding standard because a response can perform many parts of an assignment correctly while still missing something important.

For example, an agent might calculate the tax consequences of a proposed transaction correctly but fail to identify a deadline that could change the client's decision.

It could also prepare an otherwise convincing report while overlooking conflicting records or making an unsupported assumption.

Such failures matter in professional work because clients rely on advisers to investigate issues they may not know to raise.

The pass rate therefore measures compliance with a broad task rubric, not the percentage of individual tax facts that a model knows.

Why AI agents struggle with professional tax work

Tax assignments combine legal interpretation, document analysis, calculations and professional judgement.

Clients may provide incomplete instructions. Their records may contain contradictions, missing documents or information that changes the appropriate tax treatment.

A useful tax professional must recognise these problems and decide what requires further investigation.

An AI agent can produce a coherent explanation without recognising that an important question remains unanswered.

That creates a particular risk: the final work may appear complete even when it omits a consequential issue.

The benchmark was designed to expose these weaknesses by testing complete assignments rather than isolated technical questions.

Case study: The missed property sale deadline

One published case involved a homeowner considering when to sell a condominium.

The homeowner had moved out in August 2024 and wanted to compare selling in October 2026 with selling in January 2027.

An AI agent correctly concluded that both proposed dates satisfied the residence requirement for a potential exclusion of up to $250,000 in gains.

However, the agent did not identify an important later deadline.

Under the usual residence test, a homeowner must have lived in the property for at least two of the five years before the sale.

Because the homeowner moved out in August 2024, delaying the sale beyond approximately August 2027 could affect eligibility.

The benchmark report states that GPT-6 Astra missed this deadline in all five attempts, while Claude Opus 5.5 identified it in all five.

The example illustrates the difference between answering the question presented and recognising another issue that could materially affect the client.

The models were not simply being tested on arithmetic. They were expected to provide useful professional guidance within the scenario.

Case study: Conflicting evidence about marital status

Another published example concerned a separating couple and the treatment of retirement accounts.

The records raised questions about whether the couple had entered a legally recognised informal marriage while living in Texas.

That issue could affect the assumptions used in tax planning and related advice.

Claude Opus 5.5 proceeded on the assumption that the couple was unmarried without adequately resolving the conflicting evidence.

GPT-6 Astra identified the need to verify marital status in four of five attempts.

This result is important because it shows that the overall leaderboard winner is not necessarily the best model on every individual task.

It also demonstrates why a tax agent must distinguish between established facts and unresolved questions.

A plausible answer based on an unsupported assumption can still be unsuitable for professional use.

Why document handling matters

The benchmark contains 1,789 workspace files.

These include PDFs, spreadsheets, Word documents and other supporting records.

Some assignments involve handwritten receipts, business activity records, property documents or transaction histories.

The agent must locate relevant evidence and reconcile information that may be spread across several files.

For example, determining a company's tax obligations may require reviewing where employees or contractors actually worked rather than relying only on the client's summary.

Missing one relevant document can change the analysis.

This makes document retrieval, evidence tracking and contradiction detection important parts of the task.

Developers evaluating tax agents should therefore test the complete document workflow rather than focusing only on the language model's ability to explain tax rules.

How the benchmark evaluates agents

DAYJOB tasks are packaged as controlled environments using the Harbor evaluation framework.

The published harness runs agents through the OpenHands SDK and records their work.

Each task includes instructions, supporting files and evaluation criteria.

Agents complete their assignments in an environment where external network access is restricted to permitted model endpoints.

The published evaluation harness uses Claude Opus 4.8 as an agentic language-model judge.

This introduces an important limitation. Although tax experts developed and reviewed the tasks, the published scoring process relies on an AI judge rather than independent human grading of every leaderboard attempt.

Automated judges can make mistakes or interpret rubric criteria inconsistently.

They may also exhibit differences in how they assess responses from different model families.

The public material does not establish how much these effects influenced the leaderboard.

What the benchmark does not prove

The 10.8% headline figure should not be interpreted as the accuracy of Claude Sonnet 5.5 on ordinary tax questions.

A benchmark measuring complete professional assignments is fundamentally different from a test asking for short factual answers.

The result also does not establish that a model would fail at the same rate when used by a qualified tax professional.

Human reviewers can provide missing information, challenge assumptions and correct errors before work is delivered.

Conversely, the benchmark does not prove that an AI system is safe merely because it performs well on simpler tax questions.

The tasks concern US tax practice. The results should not automatically be applied to UK, Indian or other national tax systems.

Different jurisdictions have different filing requirements, legal definitions and professional obligations.

Finally, the published leaderboard reflects particular model configurations and an evaluation conducted at a specific time. Later models or different tools may produce different results.

What this means for AI tax software

The benchmark raises questions for developers building autonomous tax preparation and advisory tools.

A system that completes individual calculations accurately may still fail to recognise missing information or conflicting evidence.

For that reason, developers should assess whether an agent can identify unresolved issues before generating a final deliverable.

Applications should distinguish between verified facts, assumptions and questions requiring human confirmation.

They should also preserve references to supporting documents so reviewers can trace important conclusions back to their evidence.

Automated validation can help detect missing fields, inconsistent calculations and incomplete outputs.

However, rule-based checks alone may not identify every issue requiring professional judgement.

High-stakes tax decisions still require appropriate expert oversight.

What this means for you

We recommend using AI tax tools for clearly defined assistance rather than assuming they can independently complete complex professional assignments.

Developers, businesses and tax professionals should consider the following checklist.

1. Define the task boundary. Separate document extraction, calculations, research and final professional advice.

2. Require supporting evidence. Important conclusions should be traceable to records or authoritative tax guidance.

3. Check assumptions. Identify where the system has inferred facts that the documents do not establish.

4. Review missing information. Require the agent to flag unresolved questions rather than silently completing the assignment.

5. Test full workflows. Evaluate finished returns, workpapers and advice, not only individual answers.

6. Include difficult records. Test contradictory documents, handwritten information and incomplete client instructions.

7. Measure consequential errors. Give particular attention to missed deadlines, eligibility conditions and unsupported legal conclusions.

8. Keep human approval. Require qualified professionals to review important outputs before submission or client delivery.

9. Evaluate repeatability. Run representative tasks more than once to identify inconsistent reasoning and outcomes.

10. Maintain audit records. Preserve source references, calculations, tool activity and reviewer decisions.

For individuals, AI can help explain terminology, organise questions and prepare for discussions with a tax adviser.

It should not be treated as a substitute for checking current requirements or obtaining professional advice for consequential decisions.

For businesses, the most useful near-term approach may be combining automation with clear review and approval procedures.

Conclusion

DAYJOB: Tax provides evidence that current AI agents struggle with complete professional tax assignments.

Claude Sonnet 5.5 led the published evaluation with a 10.8% pass rate, while other leading models also scored poorly under the benchmark's strict requirements.

The central problem is broader than tax knowledge. Agents must recognise missing information, resolve conflicting evidence and identify important issues that clients may not know to mention.

We recommend treating AI as an assistant for tax research, document processing and preliminary analysis. Before relying on autonomous workflows, organisations should test complete assignments, verify evidence and retain qualified human review for consequential decisions.