Get Started with OGX

Llama Stack was renamed to OGX in 2026. OGX is an open-source, OpenAI-compatible API server with pluggable providers, including a built-in Cerebras provider. This guide uses the current OGX packages and commands.

Prerequisites

Before you begin, ensure you have:

Cerebras API Key - Get a free API key here
Python 3.12 or higher - Required by current OGX releases
uv package manager - Used to install and run the OGX starter distribution

Configure OGX with Cerebras

Install OGX

Install the OGX starter distribution and the OpenAI SDK in a persistent project environment:

uv init cerebras-ogx && cd cerebras-ogx
uv add "ogx[starter]" openai

For a quick evaluation without creating a project, uvx can install and run the server in one command:

uvx --from "ogx[starter]" ogx stack run starter

Configure environment variables

Export your Cerebras API key as an environment variable:

export CEREBRAS_API_KEY=your-cerebras-api-key-here

You can also add this to your shell profile (e.g., ~/.bashrc or ~/.zshrc) for persistence.

Start the OGX server

Launch the starter distribution. It enables the Cerebras provider automatically when CEREBRAS_API_KEY is set:

uv run ogx stack run starter

The server starts on http://localhost:8321 by default. Confirm that Cerebras models were discovered before sending requests:

curl -s http://localhost:8321/v1/models | python -m json.tool

Make your first inference request

Use any OpenAI-compatible client to make inference requests through OGX.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8321/v1",
    api_key="fake",
)

# Make a chat completion request
response = client.chat.completions.create(
    model="cerebras/gpt-oss-120b",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
)

print(response.choices[0].message.content)

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
        {"role": "user", "content": "What is the capital of France?"}
    ]
  }'

Try streaming responses

OGX supports streaming responses for real-time output. Streaming is particularly useful for interactive applications where you want to display responses as they’re generated.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8321/v1",
    api_key="fake",
)

# Stream the response
stream = client.chat.completions.create(
    model="cerebras/gpt-oss-120b",
    messages=[
        {"role": "user", "content": "Write a short poem about artificial intelligence."}
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

print()

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
        {"role": "user", "content": "Write a short poem about artificial intelligence."}
    ],
    "stream": true
  }'

Using Cerebras Directly with OpenAI SDK

If you prefer to use Cerebras directly without the OGX server, you can use the OpenAI SDK with Cerebras endpoints. This approach gives you direct access to Cerebras while still tracking usage through the integration header.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("CEREBRAS_API_KEY"),
    base_url="https://api.cerebras.ai/v1",
    default_headers={
        "X-Cerebras-3rd-Party-Integration": "ogx"
    }
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
    ],
    max_tokens=500,
)

print(response.choices[0].message.content)

curl -X POST "https://api.cerebras.ai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $CEREBRAS_API_KEY" \
  -H "X-Cerebras-3rd-Party-Integration: ogx" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ],
    "max_tokens": 500
  }'

Advanced Features

Using Multiple Models

OGX discovers the models available to your Cerebras account at startup. Use the provider-prefixed IDs returned by GET /v1/models, for example:

Model ID	Availability	Best For
`cerebras/gpt-oss-120b`	Production	Text, coding, and tool-use workloads
`cerebras/gemma-4-31b`	Preview	Multimodal workloads with text and image input
`cerebras/zai-glm-4.7`	Preview until August 17, 2026	Reasoning workloads during the remaining preview period

Then switch between models in your code:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8321/v1", api_key="fake")

# Use the production model for general workloads
production_response = client.chat.completions.create(
    model="cerebras/gpt-oss-120b",
    messages=[{"role": "user", "content": "Explain quantum computing in one sentence."}],
)
print("Production:", production_response.choices[0].message.content)

# Use the preview multimodal model
preview_response = client.chat.completions.create(
    model="cerebras/gemma-4-31b",
    messages=[{"role": "user", "content": "Describe two uses for image understanding."}],
)
print("Preview:", preview_response.choices[0].message.content)

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
        {"role": "user", "content": "Explain quantum computing in one sentence."}
    ]
  }'

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
        {"role": "user", "content": "What is 2+2?"}
    ]
  }'

System Prompts and Temperature Control

Customize model behavior with system prompts and sampling parameters to fine-tune responses:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8321/v1", api_key="fake")

# Example with system prompt and sampling params
response = client.chat.completions.create(
    model="cerebras/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "Write a Python function to calculate fibonacci numbers."}
    ],
    temperature=0.7,
    top_p=0.9,
    max_tokens=500,
)
print(response.choices[0].message.content)

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
        {"role": "system", "content": "You are a helpful coding assistant."},
        {"role": "user", "content": "Write a Python function to calculate fibonacci numbers."}
    ],
    "temperature": 0.7,
    "top_p": 0.9,
    "max_tokens": 500
  }'

Multi-Turn Conversations

The OpenAI-compatible chat endpoint supports multi-turn conversations:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8321/v1", api_key="fake")

# For agentic applications, use multi-turn conversations
conversation = [
    {"role": "system", "content": "You are a helpful research assistant."},
    {"role": "user", "content": "What are the latest trends in AI?"}
]

response = client.chat.completions.create(
    model="cerebras/gpt-oss-120b",
    messages=conversation,
)
print(response.choices[0].message.content)

# Continue the conversation
conversation.append({"role": "assistant", "content": response.choices[0].message.content})
conversation.append({"role": "user", "content": "Can you elaborate on one of those trends?"})

response2 = client.chat.completions.create(
    model="cerebras/gpt-oss-120b",
    messages=conversation,
)
print(response2.choices[0].message.content)

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
        {"role": "system", "content": "You are a helpful research assistant."},
        {"role": "user", "content": "What are the latest trends in AI?"}
    ]
  }'

curl -X POST "http://localhost:8321/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cerebras/gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a helpful research assistant."},
      {"role": "user", "content": "What are the latest trends in AI?"},
      {"role": "assistant", "content": "[Previous response content here]"},
      {"role": "user", "content": "Can you elaborate on one of those trends?"}
    ]
  }'

FAQ

What's the difference between using OGX and calling Cerebras directly?

OGX provides a provider-agnostic, OpenAI-compatible server plus APIs for responses, files, vector stores, batches, and skills. If you only need basic inference, calling Cerebras directly with the OpenAI SDK is simpler.

Can I use OGX without running a local server?

Yes. Point any OpenAI-compatible client at a remote OGX server that has the Cerebras provider enabled. Alternatively, use the OpenAI SDK directly as shown in the “Using Cerebras Directly” section.

Which Cerebras models work best with OGX?

Use gpt-oss-120b for production text, coding, and tool-use workloads. Use the preview gemma-4-31b model when you need image input. Check the model catalog for current availability before configuring additional preview models.

What Python versions are supported?

Current OGX packages require Python 3.12 or higher.

How do I handle rate limits and errors?

Handle 429 and transient 5xx responses with bounded retries and exponential backoff in your application. Monitor account usage through the Cerebras Cloud dashboard.

Troubleshooting

Server won’t start

If the OGX server fails to start:

Verify your Python version is 3.12 or higher: python --version
Check that your CEREBRAS_API_KEY environment variable is set: echo $CEREBRAS_API_KEY
Confirm the package can start: uvx --from "ogx[starter]" ogx stack run starter
Check the OGX releases page for breaking changes

Connection errors

If you see connection errors when making requests:

Verify the OGX server is running on the expected port (default: 8321)
Check that your Cerebras API key is valid by testing it directly with the OpenAI SDK
Ensure there are no firewall rules blocking localhost connections
Try restarting the OGX server
Verify your network connectivity to api.cerebras.ai

Model not found errors

If you get “model not found” errors:

Use the cerebras/ prefix for model names (e.g., cerebras/gpt-oss-120b)
List available models: curl http://localhost:8321/v1/models
Restart the OGX server after changing provider environment variables
Consult the Cerebras models page for the current list of available models

Slow response times

If responses are slower than expected:

Verify you’re using Cerebras models (not accidentally routing through another provider)
Check your network connection and latency to Cerebras endpoints
Reduce unnecessary prompt and output tokens
Enable streaming to get partial responses faster
Check your Cerebras account for any rate limiting or usage quotas

Import errors with the OpenAI SDK

If you get import errors:

Ensure the OpenAI SDK is installed: uv add openai
Check that you’re using the correct import: from openai import OpenAI
Verify that your base URL includes /v1
Create a fresh virtual environment if issues persist

Next Steps

Explore the OGX documentation for APIs, providers, and deployment options
Review the OGX Cerebras provider reference
Try different Cerebras models to find the best fit for your use case
Review the OGX GitHub repository for releases and examples

OGX was formerly named Llama Stack. The current package is ogx, the current command is ogx, and the server runs on port 8321 by default.

Integrations

Agentic Frameworks

AI Development Kits

Coding Tools

Containerization

Observability & Evaluation

LLM Integration Tools

LLM Application Frameworks

Document Processing

Real-Time Audio

Multi-LLM Management

No-Code/Low-Code Platforms

Prerequisites

Configure OGX with Cerebras

Using Cerebras Directly with OpenAI SDK

Advanced Features

Using Multiple Models

System Prompts and Temperature Control

Multi-Turn Conversations

FAQ

Troubleshooting

Server won’t start

Connection errors

Model not found errors

Slow response times

Import errors with the OpenAI SDK

Next Steps

​Prerequisites

​Configure OGX with Cerebras

​Using Cerebras Directly with OpenAI SDK

​Advanced Features

​Using Multiple Models

​System Prompts and Temperature Control

​Multi-Turn Conversations

​FAQ

​Troubleshooting

​Server won’t start

​Connection errors

​Model not found errors

​Slow response times

​Import errors with the OpenAI SDK

​Next Steps

Prerequisites

Configure OGX with Cerebras

Using Cerebras Directly with OpenAI SDK

Advanced Features

Using Multiple Models

System Prompts and Temperature Control

Multi-Turn Conversations

FAQ

Troubleshooting

Server won’t start

Connection errors

Model not found errors

Slow response times

Import errors with the OpenAI SDK

Next Steps