> Documentation index for agents: https://docs.rocketride.ai/llms.txt

# embedding_video

A RocketRide embedding node that samples a video stream into embedded image documents; choose it when video needs frame-level retrieval rather than embedding a standalone image.

## About Hugging Face

Hugging Face provides open-source model tooling and model repositories. This node uses its vision-model interfaces to produce an embedding for each selected frame.

## What it does

The node buffers a video stream, extracts frames with OpenCV, and emits one Image document with an embedding per selected frame. Choose it over `embedding_image` when RocketRide must perform the frame extraction, and over text embedders for visual media. It is marked experimental and serializes shared model access with a lock.

## Lanes

| Lane in | Lane out | Description |
|---|---|---|
| `video` | `documents` | Emits embedded Image documents for sampled frames. |

## Profiles

Default: **OpenAI CLIP - 16x16 - good performance, lower memory** (`openai-patch16`).

| Profile | Model | Context |
|---|---|---|
| `openai-patch16` *(default)* | `openai/clip-vit-base-patch16` | Default frame model. |
| `openai-patch32` | `openai/clip-vit-base-patch32` | Alternative CLIP profile. |
| `google16x224` | `google/vit-base-patch16-224` | ViT profile. |
| `custom` | User-provided | Enter a compatible Hugging Face model identifier. |

## Configuration

Use the default model and sampling limits first. Frame count directly determines how many documents and model calls the node produces, so tune extraction for the retrieval granularity you actually need.

### Model

The default is `openai/clip-vit-base-patch16`; Custom accepts a model name. Choose a model compatible with the target vector index, because every emitted frame carries that model’s vector and model name.

### Frame extraction

Frames are sampled at `interval` seconds, starting at `start_time`. `duration` of zero continues to the end; otherwise the extraction window ends at `start_time + duration` and is clamped to the video length. Increase the interval or shorten the window for a coarse overview; decrease it for short visual events, accepting more documents and embedding work.

### Frame and file limits

`max_frames` defaults to 50 and zero removes that cap. `maxVideoSizeMB` defaults to 500; an oversized video is rejected with a warning and produces no output. Raise either limit only after sizing the worker memory and the downstream index for the added frames.

## Requirements

The node declares GPU capability and uses a vision model plus OpenCV. The source does not define a CPU-only fallback, so run it where the compatible model runtime is available.

## Notes

### Frame documents

Each output contains a base64 PNG Image, `embedding`, `embedding_model`, a frame number, and a timestamp. If the input provides stream details, source metadata and a `<video-stem>.frame<N>` name are retained. A missing reported frame rate falls back to 30 fps for timestamp calculation.

### Supported containers

MP4 (`video/mp4`), AVI (`video/x-msvideo`), QuickTime (`video/quicktime`), and WebM (`video/webm`). An unrecognized MIME type is still processed, treated as MP4.

## Upstream docs

- [Hugging Face Transformers documentation](https://huggingface.co/docs/transformers)
- [OpenCV documentation](https://docs.opencv.org/)

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `embedding.duration` | `number` | **Duration (in seconds) for frame extraction (0=end of video)** | `0` |
| `embedding.interval` | `number` | **Interval (in seconds) between frames**<br/>Time in seconds between extracted frames | `5` |
| `embedding.maxVideoSizeMB` | `number` | **Maximum video file size (MB)**<br/>Maximum allowed video file size in megabytes. Videos exceeding this limit will be rejected. | `500` |
| `embedding.max_frames` | `number` | **Maximum number of frames to extract (0=unlimited)**<br/>Limit the total number of frames extracted from the video. Set to 0 for unlimited. | `50` |
| `embedding.model` | `string` | **Model name**<br/>Hugging Face model to use for frame embedding |  |
| `embedding.profile` | `string` | **Model**<br/>Embedding model for video frames | `"openai-patch16"` |
| `embedding.start_time` | `number` | **Start time (in seconds) for frame extraction (0=beginning)** | `0` |

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/embedding_video)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->
