Tool Use

Cerebras Inference supports tool calling, which enables programmatic execution of specific tasks by sending requests with clearly defined operations. The Cerebras API endpoint generates structured JSON output, which can directly invoke functions within designated codebases.

​

Use Cases

• Generate summaries from articles: Automatically condense lengthy articles or documents into concise summaries, making it easier to grasp key information quickly.

• Perform calculations from natural language: Convert user input like “What is 15% of 250?” into precise mathematical operations and return the computed result.

• Extract key details from customer feedback: Analyze customer reviews or feedback in natural language to identify important aspects such as product features mentioned, sentiment, and overall satisfaction, and store this data in a structured format.

​

Tool Specifications

tools: An array representing each tool available for the model.

• type: string Specifies the category of the tool.
• function: object: An object detailing:
  • description: string: A guide for the model on when and how to use the function.
  • name: string: The function’s identifier.
  • parameters: object: A JSON Schema object defining the parameters the function accepts.

​

Tool Choice

The tool_choice parameter controls whether the model can use tools and which tools it may invoke. By default, the model is set to automatically choose the appropriate function to call, using the tool_choice: "auto" setting.

There are three ways to adjust this default behavior:

1. Enforce Function Calling: To ensure the model always calls one or more functions, set tool_choice: "required". This approach is beneficial when you want the model to decide between multiple actions to perform next.

2. Specify a Particular Function: To direct the model to call a specific function, configure tool_choice as {"type": "function", "function": {"name": "my_function"}}. This forces the model to use only the designated function during the request.

3. Disable Function Calling: If you prefer the model to only generate a user-facing message without invoking any functions, you can either omit tools altogether or set tool_choice: "none".

When forcing the model to call a function (options 1 or 2), the completion’s finish_reason will be "stop" instead of "tool_calls".

​

Tool Call Structure

When the model uses a tool, it sends back a tool_call object with the details needed to run the tool.

Example Tool Call Structure

{
  "tool_call": {
    "id": "call_xyz789",
    "name": "evaluate_expression",
    "parameters": {
      "expression": "25 * 4 + 10"
    }
  }
}

​

Tool Result Structure

After the tool is executed, the result is sent back in a tool_result object. This object contains the output from the tool and links it to the original tool_call ID.

Example Tool Response Structure

{
  "tool_result": {
    "tool_call_id": "call_xyz789",
    "result": "110"
  }
}

In this example, the tool evaluated the expression "25 * 4 + 10" and returned the result "110", which is linked to the original tool_call ID.

For more details on handling tool responses, refer to our full API documentation.