Skip to main content

Estimate depth, segment and detect

The vision families answer three per-image questions: how far away everything is (depth), what class each pixel belongs to (segment), and where the named things are (detect). The three verbs share their media handling (any image format the decoder accepts, one artifact per input) and their serving story, so this guide works all three.

Estimate depth​

A depth family (Depth Anything and its relatives) maps each pixel to distance:

clika-modelverse depth-anything/Depth-Anything-V2-Small-hf depth photo.jpg
photo.jpg: 64x64 min=1.877290 max=4.294917

The payload is one line per image: the map's extent and its value range (relative maps: larger means closer). Artifacts opt in with --output <dir>, one per image, and each payload line then ends with -> <path>. The default --output-format raw writes the Float32 map as <stem>_depth.npy, the form a downstream consumer wants; --output-format colormap renders <stem>_depth.png instead, and --colormap picks its lookup table (spectral or gray).

Segment an image​

A segmentation family labels every pixel with a class:

clika-modelverse facebook/detr-resnet-50-panoptic segment photo.jpg

The payload is the class-mask document: which classes appear, their pixel share, and the rendered overlay's path. --mask-output mask.npy also writes the raw class-id mask (UInt8, [H, W]) as a sidecar for programmatic consumers, and --confidence adds per-label confidence to the listing.

Detect, with an open vocabulary or a trained class set​

The open-vocabulary detectors (OWL-ViT, Grounding DINO) find objects you NAME at run time through --labels; the closed-set DETR line (DETR, Deformable DETR, D-FINE, RF-DETR, RT-DETR) detects its checkpoint's trained classes and takes no label prompt. The open-vocabulary form:

clika-modelverse google/owlvit-base-patch32 detect photo.jpg \
    --labels "a red bicycle, a street sign" --confidence 0.3
label            score   box (x1, y1, x2, y2)
a red bicycle    0.812   (118, 204, 371, 468)
a street sign    0.644   (402, 61, 455, 152)

One row per hit, in score order. The labels encode once and every image scores against them, so a many-image sweep pays the text encoding a single time; --max-detections caps the rows per image.

Over HTTP​

Each verb has its serving twin in the route table: POST /v1/depth (multipart image in, depth artifacts out), POST /v1/segmentations (image in, class-mask document out), and POST /v1/detections (image plus an open-vocabulary prompt in, box/score/label rows out):

clika-modelverse google/owlvit-base-patch32 serve --port 8000
curl -s http://127.0.0.1:8000/v1/detections \
  -F image=@photo.jpg -F prompt="a red bicycle, a street sign"

The operational story (binding, health, admission control) is the serve guide's, unchanged. Model sizing lives in Model requirements; the embedding-side image comparisons (which image is LIKE which, rather than what is IN one) are Embed, compare and rerank.