Skip to main content

Write a service definition

A service definition is the template for a process the platform keeps running on a device: an inference server, a collector, a helper daemon. Unlike a job, which runs once and produces a result, a service is meant to stay up, and the platform restarts it when it does not.

The document​

kind: ServiceDefinition
name: "inference-server"
description: "Serves the packaged model over HTTP on the device."
type: process                              # "process" (default) or "compose"
command:
  - "/bin/sh"
  - "-lc"
  - "exec python3 /opt/app/serve.py --port 8000"
env:
  MODEL_PATH: "/opt/models/latest.bin"
auto_restart: true
port: 8000
health_check:
  type: http
  url: "http://127.0.0.1:8000/health"
  interval_sec: 10
  timeout_sec: 5
artifacts:
  - artifact_id: "model-weights"
    dest_path: "/opt/models/latest.bin"
    tag: "latest"
required_resources:
  min_gpu_count: 1
  min_gpu_vram_bytes: 4294967296
resource_limits:
  max_cpu_cores: 4
  max_memory_bytes: 8589934592
cleanup_steps: []

The platform's own model-serving template lives at products/clika-runtime-platform/service-defs/clika-modelverse-serve/config.yaml in the platform repository, and is the reference for a definition that is hydrated per use.

Every key​

KeyRequiredWhat it does
kindyesMust be ServiceDefinition.
nameyesThe definition's identity. Applying the same name twice updates rather than duplicates.
descriptionnoWhat the service is for.
typenoprocess (default) runs a command. compose runs a Docker Compose project on a device that has Docker.
commandyes for processThe command as an argument array.
envnoEnvironment variables for the process.
auto_restartnoRestart the process when it exits. Default false.
portnoThe port the service listens on. Recorded so the platform and the UI can reach it.
health_checknoHow the platform decides the service is up: type, url, interval_sec, timeout_sec. The agent polls it to move the service from starting to running.
artifacts[]noFiles staged on the device before the process starts. Same shape as a job definition's, including tag and credential_name.
required_resourcesnoMinimum hardware, validated before the service is placed. Same fields as a job definition's.
resource_limitsnoBounds applied to the running process: max_cpu_cores, max_memory_bytes. Enforced through cgroups where the platform supports it, and reported back with an out-of-memory flag when the limit is what killed it.
cleanup_stepsnoSteps run when the service is removed.

Starting one​

A definition is a template. Starting a service from it takes a second document:

kind: Service
service_definition: "inference-server"     # name or UUID
device: "jetson-orin-02"                   # or device_ids: [a, b]
service_id: "inference-server"
env:
  MODEL_PATH: "/opt/models/other.bin"      # overrides the definition

clika-rt apply -f service.yaml resolves the definition, merges your overrides over its defaults, and starts the service on each device you named. Fields you can override are the ones the start request carries: command, working_dir, env, artifacts, auto_restart, restart_delay_sec, log_buffer_lines, port, type, resource_limits.

A device that is offline does not refuse the start. The service and its desired state are persisted, the service reads pending, and the platform starts it when the device comes back.

What happens on the device​

The platform stores a desired state per device and reconciles it against what the device reports in its heartbeat. That is why a crashed service comes back, why a start survives an offline device, and why removing a service is a state change rather than a signal you have to time.

Stopping a service stops its whole process group, so a wrapper script that forked workers takes them with it. The stop is graded on Unix (a term signal first, then a kill once the grace period is spent); on Windows the process group is terminated outright, because a service has no console to receive the polite signal.

A service moves through pending, starting, running, and then failed or stopped. A stop whose cleanup could not finish reads cleanup_failed.

Model serving is a service you do not have to write​

Running one of your registered models on your devices as an OpenAI-compatible endpoint needs no definition of your own. The Deployments page (and POST /api/v1/servings) takes a model, the devices, and whether it runs on CPU or GPU, and the platform builds the service for you, stages its runner, watches its health, and exposes an authenticated proxy per device while it is running. Write a definition of your own when you are running something that is not a served model.

Not shipped yet​

  • Declarative desired state through the CLI (coming). The platform stores desired state per device and the API can set it (PUT /api/devices/{id}/desired-state/services/{svc_id}), but clika-rt apply does not accept a DesiredState document today. The kinds it accepts are JobDefinition, ServiceDefinition, Job, Service, Model and Benchmark.
  • Device selectors in an applied document (coming). A Service document names its devices explicitly (device or device_ids). Selecting by tag, platform or GPU model is refused with a message saying so.