Add a model master
Create a model master by sending a model's serving parameters to the CosmicAC API.
Set the default serving parameters that CosmicAC applies to every new job for a model. To fill those parameters from a published vLLM recipe instead, see Add a model master from a vLLM recipe.
Prerequisites
You need the following before you start:
- A running CosmicAC deployment. See Installation.
- Your CosmicAC base URL, the address where your CosmicAC UI is reachable. CosmicAC serves the API under /api, so each request uses <base-url>/api.
- A terminal with curl.
Add the model master
To add a model master, send a POST request with the model's serving parameters. Use the same format as Set up model masters. This example adds MiniMax M2.5:
curl -X POST <base-url>/api/v1/model-masters \
-H "Content-Type: application/json" \
-d '{
"job_type": "INFERENCE_VLLM",
"base_os_image": "Ubuntu 22.04 + CUDA 13.0",
"disk_gb": 500,
"cuda_driver_version": "CUDA 13.0",
"model_name": "MiniMaxAI/MiniMax-M2.5",
"runtime_image": "vllm/vllm-openai:v0.15.1",
"data_type": "Auto",
"quantisation": null,
"tensor_parallel": 4,
"per_replica_gpu_count": 4,
"gpu_memory_utilisation": 0.85,
"max_model_length": 27000,
"max_concurrent_sequences": 256,
"reasoning_parser": "deepseek_r1",
"multimodal": true,
"replica": 1,
"require_auth_header": true,
"max": ["gpu_memory_utilisation"],
"inference_param_overrides": {
"root_disk_size_gb": 500,
"env": [
{ "name": "TRUST_REMOTE_CODE", "value": "true" },
{ "name": "SWAP_SPACE", "value": "0" },
{ "name": "ENABLE_EXPERT_PARALLEL", "value": "true" },
{ "name": "ENFORCE_EAGER", "value": "false" }
]
}
}'For the recommended values, see Recommended model parameters.
Set quantisation to a lowercase vLLM quantisation method, such as fp8 or awq. To apply no quantisation method, set it to null, and vLLM reads any quantisation from the model's own configuration. For any other value, the API returns a 400 error and creates no model master.
Runtime image format
Set runtime_image to a Docker image reference, such as vllm/vllm-openai:v0.15.1. CosmicAC serves the model on the image you name here.
The earlier label format, such as vLLM 0.15.0 + CUDA 12.9, no longer works. If a model master still uses that format, CosmicAC serves the model on a fallback image from your deployment rather than the version you set.