The Web-LLM Assistant is an innovative tool combining large language models (LLMs) capabilities with real-time web search functionality. Utilizing platforms like Llama.cpp or Ollama, it provides accurate, context-aware answers to user queries by leveraging up-to-date web data. This project is ideal for anyone needing insights that extend beyond an AI model's training data.
How Web-LLM Assistant Works
The assistant operates through an intelligent combination of web searching and language model processing. Here's how it functions step-by-step:
1. Query Input: Users can ask, "Is the Boeing Starliner still docked at the International Space Station?"
2. Dynamic Search: The assistant generates a relevant search query and determines an appropriate time frame (e.g., results from the past day or year) tailored to the question.
3. Web Search Execution: It collects the top 10 search results and identifies the two most relevant pages for deeper analysis using web scraping.
4. Result Evaluation: The scraped content is analyzed to decide whether enough information is available to provide a complete answer.
5. Iterative Refinement: If the results are insufficient, the assistant refines its search terms or time frames and repeats the process up to five times.
6. Answer Generation: The assistant synthesizes a comprehensive answer based on the collected data, even if the topic is outside its training dataset.
This process ensures precise responses to queries about recent events or niche topics, making Web-LLM Assistant a robust tool for dynamic knowledge discovery.
Features of Web-LLM-Assistant
- Local LLM Integration: Supports Llama.cpp and Ollama for processing queries.
- Web Scraping: Extracts and analyzes content from search results for detailed answers.
- Privacy-Focused Searches: Leverages DuckDuckGo for secure, private web searches.
- Self-Improving Search: Refines search queries based on initial results for better accuracy.
- User-Friendly Console: Features rich visual output with animations for an enhanced experience.
- Comprehensive Knowledge Synthesis: Combines LLM insights, web search results, and extracted content to create thorough answers.
Installation Guide
Follow these steps to set up Web-LLM Assistant:
1. Clone the Repository:
git clone https://github.com/TheBlewish/Web-LLM-Assistant-Llamacpp-Ollama
cd Web-LLM-Assistant-Llamacpp-Ollama
2. Set Up a Virtual Environment (Optional):
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
3. Install Dependencies:
pip install -r requirements.txt
4. Configure Your LLM (Ollama or Llama.cpp):
- For Ollama:
- Start the server: `ollama serve`
- Download a model: `ollama pull gemma2:9b-instruct-q5_K_M`
- Update `llm_config.py`: Set `LLM_TYPE` to `"ollama"` and specify the `model_name`.
- For Llama.cpp:
- Place your model file (e.g., `Phi-3-medium-128k-instruct-Q6_K.gguf`) in a preferred location.
- Update `llm_config.py`: Set `LLM_TYPE` to `"llama_cpp"` and provide the model file path.
5. Run the Assistant:
python Web-LLM.py
Project Demonstration
Web-LLM Assistant redefines how LLMs interact with live data, enabling dynamic responses and bridging gaps in pre-trained AI knowledge. Whether for research, troubleshooting, or keeping up with recent events, this assistant delivers powerful results with ease and efficiency.