What is AnythingLLM?

AnythingLLM is a open source software that allows you to upload your documents and use it as a knowledge base for large language models (LLMs) locally on your machine to get answers for the questions you ask .

In simple words, a ChatGPT app that runs on your computer without needing an internet connection and allows your to use any Langauage model + you can upload any documents you want

Your chats on AnythingLLM will be stored on your machine locally meaning no one can access them (except you)

What are the Benefits of using AnythingLLM

Privacy & Security

Your chat messages and all other sensitive information you are giving to the language model stays on your local machine so you don't have to worry others access your data.

Offline Access

You don't need access to internet to use AnythingLLM and open source language models, so you can use them anytime and anywhere you want

Cost Savings

You don't have to pay anything to use the open source language models and AnythingLLM.

Flexibility on choosing LLM Models

There are thousands of open source models available so you can try lot of different models. If you don't like a model then you can simply delete it and download a new model

How to install AnythingLLM

The installation process is very straight forward, visit the AnythingLLM download page (opens in a new tab) and choose your operating system then download the installation file and install it like you install any other software.

What are the System Requirements to run AnythingLLM

Requirements to run AnythingLLM depends on many factors and you can use the below table to get an idea to see what you will neeed to run AnythingLLM

Recommended Configuration

This is the recommended value for running AnythingLLM. This will be enough for you to store some documents, chats with the model and use AnythingLLM features.

LLM Selection

Popular hosted solutions like OpenAI (opens in a new tab) tend to provide state-of-the-art models with almost zero overhead. However, you will need an API key (which costs some money)

On AnythingLLM Native option there will be Large storage impact due to model size. Additional very intense CPU/GPU and RAM overhead depending on the model.

Embedding Model Selection

This is the model which you use to "embed" or vectorize text.

Vector Database Selection

This is the solution you wish to use for storage vector data.

Transcription Model Selection

This is the model which detects words in an audio clip and converts it into vectorize text

Using the Default AnythingLLM Built-in (Xenova) models on machines with limited RAM or CPU can stall AnythingLLM when processing media files.

How to install Open Source Language Models on AnythingLLM

Installing Open source langauge models are very easy, follow the below steps:

Step 1. Go to Settings Menu

Step 2. Choose AnythingLLM on LLM Preference

Step 3. Choose the model

Choose the model you want and save your changes, then AnythingLLM will start to download the model automatically

Thats it!!

How to use Open Source Language Models on AnythingLLM

Using Open source langauge models are very easy, follow the below steps:

Step 1. Create a new workspace

Step 2. Go to workspace settings

Step 3. Choose the model you just downloaded

Now you can chat with your local models without any internet connection on AnythingLLM

How to upload custom documents on AnythingLLM

Step 1. Click the Upload Documents

Step 2. Upload your document

You can upload any file to AnythingLLM or you can enter link to any website

Step 3. Add Documents to your workspace

Useful Links

• AnythingLLM Official Website (opens in a new tab)
• AnythingLLM Github (opens in a new tab)
• AnythingLLM Docs (opens in a new tab)
• AnythingLLM Discord Community (opens in a new tab)
• AnythingLLM Cloud Hosted (opens in a new tab)

Summary

AnythingLLM is a open source software that allows you to turn any document, resource, or piece of content into context that any LLM can use as references during chatting

Using language models locally on AnythingLLM can give you privacy and security as all interactions with the models are stored locally on your machine. Some of the benefits of using AnythingLLM are enhanced privacy, offline access, cost savings and flexibility in choosing from a variety of available models.

Frequently Asked Questions

Is AnythingLLM open-source?

Yes, AnythingLLM is Open Source and you can view the source code from their github (opens in a new tab)

What are the requirements for AnythingLLM?

To use AnythingLLM you should have 2GB RAM, 2 Core CPU (any) and 5GB of storage

Is AnythingLLM available on Windows?

Yes, AnythingLLM is available on Windows

How do I delete downloaded models on AnythingLLM?

To remove a model, just go to the AnythingLLM settings menu and go to LLM Preferences, from there you will have an option to uninstall any LLMs

Does AnythingLLM use GPU?

No, AnythingLLM does not use any GPU

Can I use OpenAI ChatGPT on AnythingLLM?

Yes, you can use all OpenAI chat models by adding your OpenAI API Key to AnythingLLM

Can I use models from HuggingFace on AnythingLLM?

Yes, you can use all models from HuggingFace and you will have to provide your HuggingFace Inference Endpoint URL and HuggingFace Access Token to AnythingLLM

Can I use Anthropic Claude on AnythingLLM?

Yes, you can use all Anthropic chat models by adding your Anthropic API Key to AnythingLLM

Does AnythingLLM collect any Telemetry Data?

Yes, AnythingLLM do collect telemetry data by default but they give you an option to completely disable telemetry on the App.

Can I upload Spreadsheet to AnythingLLM?

Yes, you can upload spreadsheet, PDFs, word documentions, website urls, audio files etc.. you can upload pretty much anything to AnythingLLM

Related Articles

Everything you need to know about Meta's Llama 3→Everything you need to know about Anthropic's new Claude 3 Models→Everything you need to know about Ollama in 2024→