SAM 2 video backend does not actually do any work

[tmvkrpxl0]

I recently installed SAM 2 video backend using docker but it does not do any work. When I click on label after creating bounding box on video, it shows loading icon right next to "Auto Label" toggle. and I see this log in docker terminal:

segment_anything_2_video  | [2025-02-04 09:37:07,279] [DEBUG] [label_studio_ml.api::log_request_info::185] Request headers: Host: localhost:9090
segment_anything_2_video  | Sentry-Trace: a3cb45b81e184a80931943254afad5ee-8ec0945f41c8c965-0
segment_anything_2_video  | Baggage: sentry-environment=production,sentry-release=1.15.0,sentry-public_key=5f51920ff82a4675a495870244869c6b,sentry-trace_id=a3cb45b81e184a80931943254afad5ee,sentry-sample_rate=0.25,sentry-transaction=/projects/%3Aid%28%5Cd%2B%29/data,sentry-sampled=false
segment_anything_2_video  | User-Agent: heartex/1.15.0+0.gdf34d55.dirty
segment_anything_2_video  | Accept-Encoding: gzip, deflate
segment_anything_2_video  | Accept: */*
segment_anything_2_video  | Connection: keep-alive
segment_anything_2_video  | Content-Length: 2065
segment_anything_2_video  | Content-Type: application/json
segment_anything_2_video  | 
segment_anything_2_video  | 
segment_anything_2_video  | [2025-02-04 09:37:07,280] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b'{"tasks": [{"id": 1, "annotations": [], "file_upload": "fd2d6f9a-test.mp4", "drafts": [{"id": 1, "user": "[email protected]", "created_username": "[email protected], 1", "created_ago": "10\\u00a0minutes", "result": [{"value": {"framesCount": 228, "duration": 9.117318, "sequence": [{"frame": 25, "enabled": true, "rotation": 0, "x": 77.46875000000001, "y": 41.166666666666686, "width": 21.281250000000153, "height": 48.00000000000004, "time": 1}]}, "id": "Ms4o_fxlvh", "from_name": "box", "to_name": "video", "type": "videorectangle", "origin": "manual"}], "lead_time": 521.511, "was_postponed": false, "import_id": null, "created_at": "2025-02-04T09:26:19.805267Z", "updated_at": "2025-02-04T09:37:04.575509Z", "task": 1, "annotation": null}], "predictions": [], "data": {"video": "/data/upload/2/fd2d6f9a-test.mp4"}, "meta": {}, "created_at": "2025-02-04T09:25:33.762634Z", "updated_at": "2025-02-04T09:25:33.762649Z", "inner_id": 1, "total_annotations": 0, "cancelled_annotations": 0, "total_predictions": 0, "comment_count": 0, "unresolved_comment_count": 0, "last_comment_updated_at": null, "project": 2, "updated_by": null, "comment_authors": []}], "project": "2.1738657128", "label_config": "<View>\\n   <Labels name=\\"videoLabels\\" toName=\\"video\\" allowEmpty=\\"true\\">\\n     \\n     \\n     \\n   <Label value=\\"Lego\\" background=\\"#FFA39E\\"/></Labels>\\n   \\n   <!-- Please specify FPS carefully, it will be used for all project videos -->\\n   <Video name=\\"video\\" value=\\"$video\\" framerate=\\"25.0\\"/>\\n   <VideoRectangle name=\\"box\\" toName=\\"video\\"/>\\n</View><!--{\\n \\"video\\": \\"/static/samples/opossum_snow.mp4\\"\\n}-->", "params": {"login": null, "password": null, "context": {"result": [{"value": {"framesCount": 228, "duration": 9.117318, "sequence": [{"frame": 25, "enabled": true, "rotation": 0, "x": 77.46875000000001, "y": 41.166666666666686, "width": 21.281250000000153, "height": 48.00000000000004, "time": 1}], "labels": ["Lego"]}, "id": "Ms4o_fxlvh", "from_name": "box", "to_name": "video", "type": "videorectangle", "origin": "manual"}]}}}'
segment_anything_2_video  | [2025-02-04 09:37:07,280] [DEBUG] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::128] Image and upload dirs: image_dir=/root/.local/share/label-studio/media/upload, upload_dir=/root/.local/share/label-studio/media/upload
segment_anything_2_video  | [2025-02-04 09:37:07,280] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::155] Resolving url using hostname [http://192.168.45.194:8080]: http://192.168.45.194:8080/data/upload/2/fd2d6f9a-test.mp4
segment_anything_2_video  | [2025-02-04 09:37:07,281] [DEBUG] [model::predict::228] Video path: /root/.cache/label-studio/c49027ff__fd2d6f9a-test.mp4
segment_anything_2_video  | [2025-02-04 09:37:07,281] [DEBUG] [model::get_prompts::102] Extracting keypoints from context: {'result': [{'value': {'framesCount': 228, 'duration': 9.117318, 'sequence': [{'frame': 25, 'enabled': True, 'rotation': 0, 'x': 77.46875000000001, 'y': 41.166666666666686, 'width': 21.281250000000153, 'height': 48.00000000000004, 'time': 1}], 'labels': ['Lego']}, 'id': 'Ms4o_fxlvh', 'from_name': 'box', 'to_name': 'video', 'type': 'videorectangle', 'origin': 'manual'}]}
segment_anything_2_video  | [2025-02-04 09:37:07,281] [DEBUG] [model::predict::241] Number of prompts: 1, first frame index: 24, last frame index: 24, obj_ids: {'Ms4o_fxlvh': 0}
segment_anything_2_video  | [2025-02-04 09:37:07,281] [DEBUG] [model::split_frames::61] Opening video file: /root/.cache/label-studio/c49027ff__fd2d6f9a-test.mp4
segment_anything_2_video  | [2025-02-04 09:37:07,290] [DEBUG] [model::split_frames::69] Number of frames: 272

However it doesn't do anything after that. The docker container isn't frozen, it responds to blank enter press and whatnot. I can also send another request which also ends up not doing anything.
After a while the loading icon disappears on the webapp and no operation is applied to initial bounding box.

Is it because I used SAM 2.1? This is my docker-compose.yml file. I only changed model config and model name:

version: "3.8"

services:
  segment_anything_2_video:
    container_name: segment_anything_2_video
    image: humansignal/segment_anything_2_video:v0
    build:
      context: .
      args:
        TEST_ENV: ${TEST_ENV}
    environment:
      # specify these parameters if you want to use basic auth for the model server
      - BASIC_AUTH_USER=
      - BASIC_AUTH_PASS=
      # set the log level for the model server
      - LOG_LEVEL=DEBUG
      # any other parameters that you want to pass to the model server
      - ANY=PARAMETER
      # specify the number of workers and threads for the model server
      - WORKERS=1
      - THREADS=8
      # specify the model directory (likely you don't need to change this)
      - MODEL_DIR=/data/models
      # specify device
      - DEVICE=cuda  # or 'cpu' (coming soon)
      # SAM2 model config
      - MODEL_CONFIG=configs/sam2.1/sam2.1_hiera_l.yaml
      # SAM2 checkpoint
      - MODEL_CHECKPOINT=sam2.1_hiera_large.pt

      # Specify the Label Studio URL and API key to access
      # uploaded, local storage and cloud storage files.
      # Do not use 'localhost' as it does not work within Docker containers.
      # Use prefix 'http://' or 'https://' for the URL always.
      # Determine the actual IP using 'ifconfig' (Linux/Mac) or 'ipconfig' (Windows).
      - LABEL_STUDIO_URL=//something secret
      - LABEL_STUDIO_API_KEY=//something secret2
    ports:
      - "9090:9090"
    volumes:
      - "./data/server:/data"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

[tmvkrpxl0]

Here's video file I used for test:

test.mp4

[tmvkrpxl0]

Here's recording of what I did:

2025-02-04.18-47-19.mp4

[bug-programmer]

I am facing the same problem (and I am running the latest version of the ml backend.)