hf-cloud-sagemaker-production-defaults

Topics in the source

• SageMaker Production Defaults
• What gets created
• Running the deployment
• InService is not success — smoke-test before declaring victory
• Testing a real-time endpoint
• The UTF-8 BOM gotcha (Windows)
• Invoking a generative reranker (vLLM)
• Picking the image URI

The difference between a demo endpoint and one you can leave running is: it scales with traffic, it tells you when it breaks, and you can debug it later. This skill makes those three the default rather than optional extras. By the time this skill runs, the planner has chosen a real-time endpoint, IAM has a usable role, and image-selection has resolved a container URI + AMI version. This skill turns those into an actual deployment. For every endpoint, the skill creates these as a unit: 1. SageMaker Model — image + env vars + execution role + S3 artifacts 2. Endpoint config — instance type, initial count, optional data capture 3. Endpoint — the real-time endpoint serving inference 4. Autoscaling target + policy — target tracking on invocations per instance 5. CloudWatch alarms — latency, errors, platform overhead An inference-component deployment (deployic.py) creates the same set with two changes: the endpoint config carries the execution role and ManagedInstanceScaling, and an inference component carries the model. Its autoscaling target is the component, not the variant. Data capture (logging requests/responses to S3) is off by default — useful for debugging but creates ongoing S3

This is an excerpt. Refer to the complete source and its current requirements.