Skip to main content
To get started with a free API key, click here.
The Cerebras API supports streaming responses, allowing messages to be sent back in chunks and displayed incrementally as they are generated. To enable this feature, set the stream parameter to True within the chat.completions.create method. This will result in the API returning an iterable containing the chunks of the message. Similarly, the same can be done in TypeScript by setting the stream property to true within the chat.completions.create method.
1

Initial Setup

Begin by importing the Cerebras SDK and setting up the client.
import os
from cerebras.cloud.sdk import Cerebras

client = Cerebras(
    # This is the default and can be omitted
    api_key=os.environ.get("CEREBRAS_API_KEY"),
)
2

Streaming Responses

Set the stream parameter to True within the chat.completions.create method to enable streaming responses.
stream = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Why is fast inference important?",
        }
    ],
    model="llama-3.3-70b",
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")