diff --git a/flash/configuration/parameters.mdx b/flash/configuration/parameters.mdx index b1f551b23..2bda44e60 100644 --- a/flash/configuration/parameters.mdx +++ b/flash/configuration/parameters.mdx @@ -526,6 +526,7 @@ async def process(data): ... | Parameter | Type | Description | Default | |-----------|------|-------------|---------| | `containerDiskInGb` | `int` | Container disk size in GB | 64 | +| `containerRegistryAuthId` | `str` | ID of saved registry credentials for pulling private container images | `None` | | `env` | `list[dict]` | Environment variables as list of `{"key": "...", "value": "..."}` | `None` | ```python @@ -533,6 +534,7 @@ from runpod_flash import PodTemplate template = PodTemplate( containerDiskInGb=100, + containerRegistryAuthId="REGISTRY_CREDENTIAL_ID", env=[ {"key": "PYTHONPATH", "value": "/workspace"}, {"key": "CUDA_VISIBLE_DEVICES", "value": "0"} @@ -540,6 +542,8 @@ template = PodTemplate( ) ``` +To find the registry credential ID for `containerRegistryAuthId`, list your saved credentials with `runpodctl registry list` or check **Container Registry Authentication** in the [Runpod console settings](https://console.runpod.io/user/settings). See [Private images](/flash/custom-docker-images#private-images) for a complete example. + For simple environment variables, use the `env` parameter on `Endpoint` instead of `PodTemplate.env`. diff --git a/flash/custom-docker-images.mdx b/flash/custom-docker-images.mdx index 586aacd08..ec259fef6 100644 --- a/flash/custom-docker-images.mdx +++ b/flash/custom-docker-images.mdx @@ -108,6 +108,36 @@ asyncio.run(main()) +## Private images + +To deploy an image from a private registry, first save your registry credentials in the Runpod console, then reference the credential ID in your endpoint's `PodTemplate` using the `containerRegistryAuthId` field: + +```python +from runpod_flash import Endpoint, GpuType, PodTemplate + +vllm = Endpoint( + name="private-vllm", + image="ghcr.io/your-org/private-worker:latest", + gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, + template=PodTemplate( + containerDiskInGb=64, + containerRegistryAuthId="REGISTRY_CREDENTIAL_ID" + ) +) +``` + + +Without `containerRegistryAuthId`, Runpod cannot pull a private image, and workers fail to start. + + +To find the credential ID, list your saved registry credentials with the Runpod CLI: + +```bash +runpodctl registry list +``` + +To create a new credential, use [`runpodctl registry create`](/runpodctl/reference/runpodctl-registry), or navigate to [Settings](https://console.runpod.io/user/settings) in the Runpod console and scroll down to **Container Registry Authentication**. For GitHub Container Registry, use a personal access token scoped to `read:packages`. + ## Complete example: vLLM inference This example deploys vLLM and makes inference requests: @@ -282,7 +312,7 @@ await job.cancel() # Cancel the job **Solutions**: - Add `HF_TOKEN` to `env` for Hugging Face gated models. -- Configure Docker registry authentication in [Runpod console](https://console.runpod.io/user/settings) for private images. +- For private images, save registry credentials in the [Runpod console](https://console.runpod.io/user/settings) and pass the credential ID to your endpoint with `PodTemplate(containerRegistryAuthId=...)`. See [Private images](#private-images). ## Next steps