multimodal-llava

Topics in the source

• LLaVA - Large Language and Vision Assistant
• When to use LLaVA
• Quick start
• Installation
• Basic usage
• Available models
• CLI usage
• Web UI (Gradio)

Open-source vision-language model for conversational image understanding. Use when: Building vision-language chatbots Visual question answering (VQA) Image description and captioning Multi-turn image conversations Visual instruction following Document understanding with images Metrics: 23,000+ GitHub stars GPT-4V level capabilities (targeted) Apache 2.0 License Multiple model sizes (7B-34B params) Use alternatives instead: GPT-4V: Highest quality, API-based CLIP: Simple zero-shot classification BLIP-2: Better for captioning only Flamingo: Research, not open-source | Model | Parameters | VRAM | Quality | |-------|------------|------|---------| | LLaVA-v1.5-7B | 7B | 14 GB | Good | | LLaVA-v1.5-13B | 13B | 28 GB | Better | | LLaVA-v1.6-34B | 34B | 70 GB | Best | 1. Start with 7B model - Good quality, manageable VRAM 2. Use 4-bit quantization - Reduces VRAM significantly 3. GPU required - CPU inference extremely slow 4. Clear prompts - Specific questions get better answers 5. Multi-turn conversations - Maintain conversation context 6. Temperature 0.2-0.7 - Balance creativity/consistency 7. maxnewtokens 512-1024 - For detailed responses 8. Batch processing - Process multiple images seq

This is an excerpt. Refer to the complete source and its current requirements.