Structured Outputs

Structured outputs with strict adherence is currently in beta, released in version 1.23 of the Cerebras Inference Cloud SDK. Improvements for speed and expanded support will be released in coming weeks, with documentation updated accordingly.

Structured outputs is a feature that can enforce consistent JSON outputs for models in the Cerebras Inference API. This is particularly useful when building applications that need to process AI-generated data programmatically. Some of the key benefits of using structured outputs are:

Reduced Variability: Ensures consistent outputs by adhering to predefined fields.
Type Safety: Enforces correct data types, preventing mismatches.
Easier Parsing & Integration: Enables direct use in applications without extra processing.

In this tutorial, we’ll explore how to use structured outputs with the Cerebras SDK, from basic JSON responses to strictly enforced schemas.

Initial Setup

First, ensure that you have completed steps 1 and 2 of our Quickstart Guide to set up your API key and install the Cerebras Cloud SDK.

Then, initialize the Cerebras client and import the necessary modules we will use in this tutorial.

import os
from cerebras.cloud.sdk import Cerebras
import json

client = Cerebras(
    api_key=os.environ.get("CEREBRAS_API_KEY")
)

Defining the Schema

To ensure structured responses from the model, we’ll use a JSON schema to define our output structure. Start by defining your schema, which specifies the fields, their types, and which ones are required:

movie_schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "director": {"type": "string"},
        "year": {"type": "integer"},
    },
    "required": ["title", "director", "year"],
    "additionalProperties": False
}

Using Structured Outputs

Next, use the schema in your API call by setting the response_format parameter to include both the type and your schema. Setting strict to true will enforce the schema. Setting strict to false will allow the model to return additional fields that are not specified in the schema, similar to JSON mode.

completion = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant that generates movie recommendations."},
        {"role": "user", "content": "Suggest a sci-fi movie from the 1990s"}
    ],
    response_format={
        "type": "json_schema", 
        "json_schema": {
            "name": "movie_schema",
            "strict": True,
            "schema": movie_schema
        }
    }
)

# Parse the JSON response
movie_data = json.loads(completion.choices[0].message.content)
print(json.dumps(movie_data, indent=2))

Sample output:

{
  "title": "The Fifth Element",
  "director": "Luc Besson",
  "year": 1997
}

Now you have a structured JSON response from the model, which can be used in your application.

Advanced Schema Features

Your schema can include various JSON Schema features:

Types: String, Number, Boolean, Integer, Object, Array, Enum, anyOf (max of 5), null
Nested structures: Define complex objects with nested properties, with support for up to 5 layers of nesting. You can also use definitions to reference reusable schema components.
Required fields: Specify which fields must be present.
Additional properties: Control whether extra fields are allowed. Note: the only accepted value is false.

For example, a more complex schema might look like:

detailed_schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "director": {"type": "string"},
        "year": {"type": "integer"},
        "genres": {
            "type": "array",
            "items": {"type": "string"}
        },
        "cast": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "role": {"type": "string"}
                },
                "required": ["name"]
            }
        }
    },
    "required": ["title", "director", "year", "genres"],
    "additionalProperties": False
}

When used with the API, you might get a response like:

{
  "title": "Jurassic Park",
  "director": "Steven Spielberg",
  "year": 1993,
  "genres": ["Science Fiction", "Adventure", "Thriller"],
  "cast": [
    {"name": "Sam Neill", "role": "Dr. Alan Grant"},
    {"name": "Laura Dern", "role": "Dr. Ellie Sattler"},
    {"name": "Jeff Goldblum", "role": "Dr. Ian Malcolm"}
  ]
}

JSON Mode

In addition to structured outputs, you can also use JSON mode to generate JSON responses from the model. This approach tells the model to return data in JSON format but doesn’t enforce a specific structure. The model decides what fields to include based on the context of your prompt. Note: we recommend using structured outputs (with strict set to true) whenever possible, as it provides more predictable and reliable results.

To use JSON mode, simply set the response_format parameter to json_object:

completion = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
      {"role": "system", "content": "You are a helpful assistant that generates movie recommendations."},
      {"role": "user", "content": "Suggest a sci-fi movie from the 1990s"}
    ],
    response_format={"type": "json_object"}
)