Report Survival LLM: Offline AI Gets 21 of 22 Emergency Answers Right in Developer Tests

Survival LLM's developer reported that its strongest local AI models answered the main point correctly on 21 of 22 emergency questions. The smaller IBM Granite 4.2 8B model managed 14 of 22, while a separate citation test found that every model checked sometimes named sources it had never accessed.

The findings illustrate both the potential and the limitations of offline AI assistants. Survival LLM can search emergency guidance without an internet connection, but its answers still require verification, particularly when health or physical safety is involved.

The software is free and open source. It combines locally running language models with an offline reference library containing medical information, emergency guidance, repair instructions and general knowledge.

We reviewed the project's published test results, technical documentation and release information. We have not independently installed or tested the application.

Key findings

• 21 of 22 correct: Nemotron 3.5 Super and Qwen3.8 27B each captured the main point of official emergency guidance in the developer's evaluation.

• 14 of 22 correct: Granite 4.2 8B performed substantially worse on the same question set.

• Five models invented citations: Every model included in a separate 14-question source-checking exercise named at least one source it had not accessed.

• Approximately 1.5 million articles: The full offline library combines encyclopaedia content, emergency guidance, medical references and practical instructions.

• 80 GB model download: The highest-memory configuration uses Nemotron 3.5 Super, illustrating the substantial hardware requirements of stronger local models.

What Survival LLM does

Survival LLM is a desktop application designed to answer questions when an internet connection is unavailable.

It runs a language model locally and gives that model access to a collection of downloaded reference documents.

Users can ask questions about emergency preparation, food storage, first aid, equipment maintenance and other practical subjects.

The application searches its library, retrieves relevant passages and generates an answer using the information it finds.

Users can open the supporting articles and inspect highlighted passages rather than relying entirely on the model's explanation.

This distinction matters because language models can produce convincing answers that contain factual mistakes.

The project is intended to make source checking easier, although the developer's own evaluation shows that citation handling also needs safeguards.

How the offline AI assistant works

Survival LLM uses Ollama to run compatible language models on the user's computer.

Its reference library is stored locally and searched using SQLite full-text search.

When the user submits a question, the model can search the library, read relevant articles and display supporting material.

The search system uses BM25 ranking over passages of approximately 1,500 characters. Official sources receive higher ranking priority.

The application also supports personal documents, including PDFs, Word files, ebooks and text documents.

For example, a user could add a generator manual or an emergency preparation plan and search those files alongside the bundled library.

The application currently cannot extract text from scanned pages or photographs in personal documents.

Its local server listens on the computer's loopback address rather than exposing the chat interface directly to the wider network.

Once installation and downloads are complete, the core assistant can operate without internet access.

Survival LLM emergency benchmark results

The developer evaluated local models using 22 questions covering emergency and first-aid topics.

The questions included food safety, water, generators, carbon monoxide, hypothermia, bleeding, burns, insulin, electrical hazards and other situations.

Each model used the application with its retrieval tools and reference library.

AI judges evaluated answers against official guidance. The process included separate accuracy and safety assessments, followed by additional review of responses flagged as harmful.

The reported accuracy percentages are based on graded answer quality, not simply the proportion of questions marked correct.

That explains why a model can have 21 correct main points without receiving an accuracy score equal to that proportion.

The response times were measured on an M4 Max MacBook Pro with 128 GB of memory. They should not be treated as expected speeds on every supported computer.

The developer also warns that the evaluation used an earlier library build and that some tested model files differ from those distributed through the installer.

Why smaller models performed worse

Local AI applications must work within the available memory and processing capacity of the user's computer.

Smaller models generally require fewer resources, making them practical on ordinary laptops.

However, reduced hardware requirements can come with weaker reasoning or less reliable handling of complicated instructions.

In the developer's evaluation, Granite 4.2 8B captured the main point of 14 questions, compared with 21 for the strongest models.

The even smaller Granite 4.2 3B performed worse, capturing the main point of eight questions and producing five responses classified as harmful.

The developer excluded that model from the recommended downloads.

Qwen 3.6 was also excluded after becoming stuck in repeated reasoning during some questions.

These findings show why model size and hardware compatibility should not be the only criteria for choosing an offline assistant.

A model that fits a computer may still be unsuitable for safety-sensitive tasks.

The problem with fabricated sources

The project's separate citation evaluation exposed another important weakness.

In a test involving 14 questions, models sometimes listed sources they had not opened and had not seen in their search results.

Nemotron 3.5 Super did this in six answers. Nemotron 3.5 Lightning and Gemma 4 each did so in three, Granite in two and Qwen3.8 27B in one.

This matters because a reference can make an answer appear more reliable even when the model did not actually use that document.

The application now checks model-generated source lists against the articles available in its retrieval history.

When a source list contains unsupported references, the application replaces it with a list based on the material actually accessed.

This is a useful design choice for retrieval-based AI systems.

However, a genuine citation does not guarantee that the model interpreted the cited passage correctly.

Users still need to read the relevant material and confirm that it supports the answer.

What is included in the offline library

The full Survival LLM library contains approximately 1.5 million articles.

It includes around 930,000 frequently accessed Wikipedia articles alongside medical references, emergency preparedness guidance, survival manuals, repair instructions and practical knowledge resources.

Some material comes from government and public health organisations, while other content comes from community-maintained references.

The library is not a single uniformly reviewed medical database.

Users should consider the authority, relevance and publication date of each source before acting on its advice.

Documents stored locally also need updating. A library that remains offline for a long period may contain outdated guidance.

Survival LLM offers library and application updates, but users must obtain those updates while they have internet access.

Hardware requirements and model downloads

The application supports Apple Silicon Macs running macOS 14 or newer and compatible 64-bit Windows 10 or 11 computers.

It recommends models based on available system memory.

The full reference library requires a separate download of approximately 8.8 GB, while the smaller Essentials library is approximately 1.3 GB.

Additional disk space is required during installation.

Computers with 8 GB of memory can search the library, but the developer does not recommend an AI model for that memory tier.

These requirements mean Survival LLM is not necessarily practical on every low-power emergency device.

Privacy and offline access

One advantage of local AI is that questions and retrieved documents can remain on the user's computer.

Survival LLM does not require an account for ordinary use and is designed to answer questions without sending them to an external AI service.

This may appeal to people who want to search private emergency plans or equipment documentation without relying on cloud processing.

However, local storage does not eliminate security risks.

Personal documents and saved memory notes remain accessible to anyone who gains appropriate access to the device.

Users should protect their computers, maintain backups and avoid storing unnecessary sensitive information.

They should also verify the software's source and download location before installing it.

Why offline AI still needs caution

Survival LLM is designed for situations where reliable information may be difficult to access.

That makes mistakes particularly consequential.

The developer's own testing found incorrect answers, harmful responses and fabricated citations.

Even the strongest models failed to capture the main point of every question.

The tests also involved only 22 emergency questions, which is too small a sample to establish general reliability across all medical, electrical or disaster scenarios.

AI-based grading introduces another limitation because the judges themselves may make mistakes.

The tested library also differed from the current public release. Some material used during evaluation was not included in the distributed downloads.

For these reasons, the results should be treated as preliminary developer evidence rather than independent safety certification.

In an immediate emergency, users should prioritise qualified emergency services and authoritative instructions whenever available.

What this means for you

We see Survival LLM as a potentially useful offline reference interface, but not a substitute for emergency training or professional advice.

Before relying on it, consider the following checks.

1. Install it before an outage. Download the application, model and reference library while internet access is available.

2. Check hardware capacity. Confirm that the selected model fits your computer's memory and storage.

3. Test offline operation. Disconnect from the internet and confirm that search and document access still work.

4. Open original sources. Read the cited passage rather than trusting the generated summary alone.

5. Prefer authoritative guidance. Give priority to current instructions from recognised emergency, medical and public safety organisations.

6. Keep essential references accessible. Maintain direct copies of important documents that can be read without AI.

7. Check source freshness. Update the library when internet access becomes available.

8. Test realistic questions. Try common scenarios relevant to your location and equipment before an emergency occurs.

9. Protect personal documents. Review which files and memory notes the application stores locally.

10. Do not rely on AI for urgent decisions. Seek qualified help and follow established safety procedures where possible.

For developers, the project offers a useful example of local retrieval, source verification and hardware-aware model selection.

For ordinary users, its strongest feature may be the searchable offline library rather than the generated answers.

Conclusion

Survival LLM combines local language models with a substantial offline reference library and makes supporting passages accessible to users.

Its developer reported promising results for larger models, with 21 of 22 emergency questions answered correctly on the main point.

However, smaller models performed worse, harmful responses appeared in the evaluation and every model included in the separate citation check sometimes invented sources.

We recommend using Survival LLM as a supplementary reference tool. Keep authoritative documents available independently, verify generated answers against their sources and avoid treating the developer's limited benchmark as proof of emergency reliability.