Skip to main content
View source

OpenAI-Compatible API

View as Markdown

A RocketRide LLM node that connects any OpenAI-compatible API endpoint to a pipeline.

What it does​

Connects any OpenAI-compatible API endpoint to your pipeline. Use this node for providers that implement the OpenAI API spec (Featherless, Together, Groq, Ollama, LM Studio, and others) but do not have a dedicated RocketRide node. Used primarily as an llm invoke connection by agents and other nodes that need an LLM, but can also be used directly via lanes.

Built on langchain-openai (ChatOpenAI) over the official openai SDK. The chat client sends temperature: 0 for deterministic output unless the node sets another value (see Temperature under Configuration), and max_tokens is set from the shared LLM base's output-token budget (default 4096, minimum 1024). When base_url is empty, the SDK falls back to the default OpenAI endpoint.

When the node configuration is saved, it validates the settings by sending a 1-token "Hi" completion request to the configured endpoint (10-second timeout) and surfaces any provider error (bad key, unknown model, unreachable URL) as a warning with the provider's status code and message, truncated to 500 characters.

Lanes​

Lane inLane outDescription
questionsanswersSend a question directly, receive a generated answer

Profiles​

Generic endpoint default: Custom OpenAI-compatible endpoint (custom). Nebius preset default: Llama 3.3 70B Instruct (llama-3-3-70b).

The directory documents the generic endpoint and the Nebius branded preset in one combined profile table.

ProfileModelContext tokensSource
Generic custom (default)(user-specified)32,768Any OpenAI-compatible API
llama-3-3-70b (default)meta-llama/Llama-3.3-70B-Instruct131,072Nebius Token Factory
qwen3-235bQwen/Qwen3-235B-A22B131,072Nebius Token Factory
deepseek-v3deepseek-ai/DeepSeek-V3131,072Nebius Token Factory

Configuration​

Choose custom for a generic provider and set its model ID, base URL, context limit, output limit (4,096 tokens unless changed) and API key. Choose one of the named Nebius profiles when using Token Factory; those profiles pin the model ID, endpoint, and context limit.

Generic OpenAI-compatible endpoints​

Set base_url to the provider's OpenAI-compatible API root and model to its exact model identifier. When base_url is empty, the OpenAI SDK uses its default endpoint. Use a positive modelTotalTokens value that matches the selected model.

Temperature​

temperature (0 to 2, default 0) is sent with every request.

Some models run only at their own default and reject any other value: newer Claude models (Opus 4.7 and 4.8, and the Claude 5 models), OpenAI's reasoning models such as o3, and a gateway deployment in front of one, where the node only sees the deployment name. When the endpoint answers HTTP 400 or 422 and names temperature, the node sends the same request once more without a temperature. If that request goes through, the node writes a warning and leaves the temperature out for the rest of the run, so the model runs at its own default; the first request of such a run is therefore refused once. If it is refused as well, the node keeps its value and reports that second refusal, which names what the endpoint still does not accept. This retry does not send a request again for any other refusal.

The Nebius preset does not show the field. It sends 0 and recovers the same way.

Nebius preset​

services.nebius.json registers Nebius (llm_nebius://) as a branded preset of this node. It uses the same Python implementation and pins the base URL to https://api.tokenfactory.nebius.com/v1/. The three named Nebius profiles in the combined table require no model or endpoint edits.

Nebius profiles default the API key to the ${ROCKETRIDE_NEBIUS_KEY} environment substitution, so no per-node key entry is needed when that variable is set. The Nebius service also exposes its own custom option for another Token Factory model, retaining the pinned endpoint, environment-backed key, and 131,072-token context limit, with an output limit of 4,096 tokens unless changed. The combined table represents the duplicate custom key once, using the generic service's merged 32,768-token default.

Authentication​

For custom, provide the API key expected by the configured endpoint. Nebius named profiles use ${ROCKETRIDE_NEBIUS_KEY} by default; replace it only when the credential must be supplied directly in the node configuration.

Notes​

Error handling and retries​

Provider errors are mapped to friendly messages and classified for retry:

Provider exceptionRetriedSurfaced as
AuthenticationErrornoInvalid API key.
APIErrornoAn error occurred with the API.
RateLimitErroryesRate limit exceeded. Please try again later.
APIConnectionErroryesFailed to connect to the API.

Anything else falls through to the shared LLM base's default retry and mapping logic.

Testing​

services.json includes a test block that runs the custom profile against a local test server with a mocked langchain_openai, so no real API key is required. ROCKETRIDE_MOCK must point to nodes/test/mocks.

nodes/test/llm_openai_api/test_openai_api_temperature.py runs the real driver and the real langchain-openai client against a local HTTP server that refuses a temperature the way OpenAI, Anthropic and LiteLLM do, and checks the request bodies. It needs no key.

Model sync (Nebius service only)​

The profiles in services.nebius.json are maintained by the sync_models tooling (llm_nebius provider, ROCKETRIDE_NEBIUS_KEY), which reads Nebius Token Factory's own model list. New profiles are written with the Nebius base URL and the same ${ROCKETRIDE_NEBIUS_KEY} reference the existing ones use.

Token limits come only from Nebius: it runs open-weight models on its own hardware, so the context window is its choice, and the same model is served with a different window by every other host. The sync runs only with that key, because Nebius IDs (Qwen/Qwen3-235B-A22B) appear in no other catalogue and a keyless run would mark these profiles deprecated — see Why some providers need their own key.

The generic services.json profiles are not synced: the model is whatever the user types for their own OpenAI-compatible endpoint.


Schema​

OpenAI-Compatible API (services.json)​

FieldTypeDescriptionDefault
base_urlstringBase URL
API base URL (e.g. https://api.featherless.ai/v1)
modelstringModel
Model name (e.g. zai-org/GLM-5, meta-llama/Llama-3-70b)
modelOutputTokensintegerOutput Tokens
Most tokens the model may write in one reply (at least 1,024). Leave it empty to keep the node's default. Reasoning models count their thinking against this limit: set it well above the reply you expect, or they can spend it all thinking and return nothing. It cannot exceed Tokens; a larger value is lowered to it.
modelTotalTokensnumberTokens
Total Tokens
openai_api.profilestringModel
Profile selection
"custom"
temperaturenumberTemperature
Sampling temperature: 0 is generally more deterministic, higher values increase variety. Some models run only at their own default and reject any other value (newer Claude models, OpenAI's reasoning models, and gateways in front of them). The node then sends the request again without a temperature and, if that goes through, warns and leaves it out for the rest of the run.
0

Nebius (services.nebius.json)​

FieldTypeDescriptionDefault
base_urlstringBase URL
OpenAI-compatible base URL. Defaults to Nebius Token Factory.
modelstringModel
Nebius Token Factory model id (e.g. meta-llama/Llama-3.3-70B-Instruct). Full list: https://tokenfactory.nebius.com/models
modelOutputTokensintegerOutput Tokens
Most tokens the model may write in one reply (at least 1,024). Leave it empty to keep the node's default. Reasoning models count their thinking against this limit: set it well above the reply you expect, or they can spend it all thinking and return nothing. It cannot exceed Tokens; a larger value is lowered to it.
modelTotalTokensnumberTokens
Total Tokens
openai_api.profilestringModel
Nebius Token Factory model
"llama-3-3-70b"

Dependencies​

  • openai
  • langchain-openai
  • langchain-core
  • langchain