AgentsPrivate document Q&A that never leaves your computer
Run an open model locally so staff can ask questions about contracts, SOPs, or manuals without uploading them anywhere.
Why local
Tools like Ollama and LM Studio run open-weight models on your own machine, so documents never go to a cloud API. LM Studio recommends at least 16GB of RAM and 4GB of dedicated VRAM.
How to set it up
Pick a general model sized to your hardware (for example a Llama 3.1 8B tag on a 16GB machine) and an embedding model such as nomic-embed-text for search over your files.
Start with one folder of non-critical documents. Ask it questions you already know the answers to, and keep a log of misses before you widen access.
Watch out for
Local models are usually less capable than the best cloud models, so require citations to the source passage and have a person check anything you'll act on. Check each model's license on its model page before business use.
Sources
Figures from vendors are labeled as vendor-reported. We have no commercial relationship with any company linked here.