hf-cloud-serving-image-selection

原文涵盖的主题

• Serving Image Selection
• Rule zero: HuggingFace images always win
• Where image URIs come from
• Quick decision
• Rerankers: TEI or vLLM?
• Workflow
• TEI: pick the right variant
• vLLM AMI requirement

The serving container is the single thing most likely to break a SageMaker deployment that "looked correct on paper". Wrong container, stale tag, or the wrong AMI — all produce the same opaque Failed to pass health check error. When both a HuggingFace-curated family (huggingface-vllm, huggingface-vllm-omni, huggingface-sglang, tei, huggingface-pytorch-inference) and a generic family (vllm, vllm-omni, sglang, djl-inference) can serve the model, the HuggingFace one is mandatory, not preferred. The only valid reasons to use a generic image: 1. Verified incompatibility — the model needs an architecture/modality/feature no available HuggingFace tag supports, confirmed against the catalog (not assumed). 2. No HuggingFace tag exists in the target region and mirroring is not an option. 3. The HuggingFace image is in "Known-broken images" below. A newer version number on the generic repo is not a reason. The AWS vllm repo often publishes a higher vLLM version than huggingface-vllm; an older-but-compatible huggingface-vllm tag still wins. "Latest vLLM" is not a requirement anyone stated — compatibility with the model is. If you fall back, record in the deployment log which of the three reaso

这里只展示节选，请以完整原文和当前要求为准。