huggingface-local-models

Topics in the source

• Hugging Face Local Models
• Default Workflow
• Quick Start
• Install llama.cpp
• Authenticate for gated repos
• Search the Hub
• Run directly from the Hub
• Run an exact GGUF file

Search the Hugging Face Hub for llama.cpp-compatible GGUF repos, choose the right quant, and launch the model with llama-cli or llama-server. 1. Search the Hub with apps=llama.cpp. 2. Open https://huggingface.co/ ?local-app=llama.cpp. 3. Prefer the exact HF local-app snippet and quant recommendation when it is visible. 4. Confirm exact .gguf filenames with https://huggingface.co/api/models/ /tree/main?recursive=true. 5. Launch with llama-cli -hf : or llama-server -hf : . 6. Fall back to --hf-repo plus --hf-file when the repo uses custom file naming. 7. Convert from Transformers weights only if the repo does not already expose GGUF files. Prefer the exact quant that HF marks as compatible on the ?local-app=llama.cpp page. Keep repo-native labels such as UD-Q4KM instead of normalizing them. Default to Q4KM unless the repo page or hardware profile suggests otherwise. Prefer Q5KM or Q6K for code or technical workloads when memory allows. Consider Q3KM, Q4KS, or repo-specific IQ / UD- variants for tighter RAM or VRAM budgets. Treat mmproj-.gguf files as projector weights, not the main checkpoint. Read hub-discovery.md for URL-first workflows, model search, tree API extraction, and comma

This is an excerpt. Refer to the complete source and its current requirements.