Azure OpenAI evals

API Version: v1

Server: {endpoint}/openai/v1 — Azure AI Foundry Models APIs

Server Variables:

Variable	Default	Description
endpoint		A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).

Authentication

ApiKeyAuth (API Key)

Pass your API key in the api-key header.

ApiKeyAuth_ (API Key)

Pass your API key in the authorization header.

OAuth2Auth (OAuth 2.0)

Flow: implicit

Authorization URL: https://login.microsoftonline.com/common/oauth2/v2.0/authorize

Scopes:

https://cognitiveservices.azure.com/.default

Security Requirements

Endpoints accept any one of the following authentication methods:

ApiKeyAuth
ApiKeyAuth_
OAuth2Auth (scopes: https://cognitiveservices.azure.com/.default)

List evals

GET {endpoint}/openai/v1/evals

List evaluations for a project.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
after	query	No	string	Identifier for the last eval from the previous pagination request. Serialization: explode: false
limit	query	No	integer (int32)	A limit on the number of evals to be returned in a single pagination response. Serialization: explode: false
order	query	No	string Possible values: `asc`, `desc`	Sort order for evals by timestamp. Use `asc` for ascending order or `desc` for descending order. Serialization: explode: false
order_by	query	No	string Possible values: `created_at`, `updated_at`	Evals can be ordered by creation time or last updated time. Use `created_at` for creation time or `updated_at` for last updated time. Serialization: explode: false

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.EvalList

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Create eval

POST {endpoint}/openai/v1/evals

Create the structure of an evaluation that can be used to test a model's performance.

An evaluation is a set of testing criteria and a datasource. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources.

For more information, see the Evals guide.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.

Request Body

Content-Type: application/json

Name	Type	Description	Required
data_source_config	OpenAI.CreateEvalCustomDataSourceConfig or OpenAI.CreateEvalLogsDataSourceConfig or OpenAI.CreateEvalStoredCompletionsDataSourceConfig	The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation.	Yes
metadata	OpenAI.Metadata or null		No
name	string	The name of the evaluation.	No
statusCode	enum	Possible values: `201`	Yes
testing_criteria	array of OpenAI.CreateEvalLabelModelGrader or OpenAI.EvalGraderStringCheck or OpenAI.EvalGraderTextSimilarity or OpenAI.EvalGraderPython or OpenAI.EvalGraderScoreModel or EvalGraderEndpoint	A list of graders for all eval runs in this group. Graders can reference variables in the data source using double curly braces notation, like `{{item.variable_name}}`. To reference the model's output, use the `sample` namespace (ie, `{{sample.output_text}}`).	Yes

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.Eval

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Get eval

GET {endpoint}/openai/v1/evals/{eval_id}

Retrieve an evaluation by its ID. Retrieves an evaluation by its ID.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.Eval

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Update eval

POST {endpoint}/openai/v1/evals/{eval_id}

Update select, mutable properties of a specified evaluation.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string

Request Body

Content-Type: application/json

Name	Type	Description	Required	Default
metadata	OpenAI.Metadata	Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.	No
name	string		No

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.Eval

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Delete eval

DELETE {endpoint}/openai/v1/evals/{eval_id}

Delete a specified evaluation.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	object

Name	Type	Description	Required
deleted	boolean		Yes
eval_id	string		Yes
object	enum	Possible values: `eval.deleted`	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Get eval runs

GET {endpoint}/openai/v1/evals/{eval_id}/runs

Retrieve a list of runs for a specified evaluation.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string
after	query	No	string	Serialization: explode: false
limit	query	No	integer (int32)	Serialization: explode: false
order	query	No	string Possible values: `asc`, `desc`	Serialization: explode: false
status	query	No	string Possible values: `queued`, `in_progress`, `completed`, `canceled`, `failed`	Serialization: explode: false

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.EvalRunList

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Create eval run

POST {endpoint}/openai/v1/evals/{eval_id}/runs

Create a new evaluation run, beginning the grading process.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string

Request Body

Content-Type: application/json

Name	Type	Description	Required
data_source	OpenAI.CreateEvalJsonlRunDataSource or OpenAI.CreateEvalCompletionsRunDataSource or OpenAI.CreateEvalResponsesRunDataSource	Details about the run's data source.	Yes
metadata	OpenAI.Metadata or null		No
name	string	The name of the run.	No

Responses

Status Code: 201

Description: The request has succeeded and a new resource has been created as a result.

Content-Type	Type	Description
application/json	OpenAI.EvalRun

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Get eval run

GET {endpoint}/openai/v1/evals/{eval_id}/runs/{run_id}

Retrieve a specific evaluation run by its ID.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string
run_id	path	Yes	string

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.EvalRun

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Cancel eval run

POST {endpoint}/openai/v1/evals/{eval_id}/runs/{run_id}

Cancel a specific evaluation run by its ID.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string
run_id	path	Yes	string

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.EvalRun

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Delete eval run

DELETE {endpoint}/openai/v1/evals/{eval_id}/runs/{run_id}

Delete a specific evaluation run by its ID.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string
run_id	path	Yes	string

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	object

Name	Type	Description	Required
deleted	boolean		Yes
eval_run_id	string		Yes
object	enum	Possible values: `eval_run.deleted`	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Get eval run output items

GET {endpoint}/openai/v1/evals/{eval_id}/runs/{run_id}/output_items

Get a list of output items for a specified evaluation run.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string
run_id	path	Yes	string
after	query	No	string	Serialization: explode: false
limit	query	No	integer (int32)	Serialization: explode: false
status	query	No	string Possible values: `fail`, `pass`	Serialization: explode: false
order	query	No	string Possible values: `asc`, `desc`	Serialization: explode: false

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.EvalRunOutputItemList

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Get eval run output item

GET {endpoint}/openai/v1/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}

Retrieve a specific output item from an evaluation run by its ID.

URI Parameters

Name	In	Required	Type	Description
endpoint	server	Yes	string	A supported Azure AI Foundry Models APIs endpoint, including protocol and hostname. For example: https://westus.api.cognitive.microsoft.com).
api-version	query	No	string Possible values: `v1`, `preview`	The explicit Azure AI Foundry Models API version to use for this request. `v1` if not otherwise specified.
eval_id	path	Yes	string
run_id	path	Yes	string
output_item_id	path	Yes	string

Responses

Status Code: 200

Description: The request has succeeded.

Content-Type	Type	Description
application/json	OpenAI.EvalRunOutputItem

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Status Code: default

Description: An unexpected error response.

Content-Type	Type	Description
application/json	object

Name	Type	Required
code	string or null	Yes
inner_error		No
message	string	Yes
param	string or null	Yes
type	string	Yes

Response Headers:

Header	Type	Description
apim-request-id	string	A request ID used for troubleshooting purposes.

Components

AzureAIFoundryModelsApiVersion

Property	Value
Type	string
Values	`v1` `preview`

AzureCompletionsSamplingParams

Sampling parameters for controlling the behavior of completions.

Name	Type	Description	Required	Default
max_completion_tokens	integer		No
max_tokens	integer (int32)	The maximum number of tokens in the generated output.	No
reasoning_effort	OpenAI.ReasoningEffort	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.	No
response_format	OpenAI.ResponseFormatText or OpenAI.ResponseFormatJsonSchema or OpenAI.ResponseFormatJsonObject		No
seed	integer	A seed value initializes the randomness during sampling.	No	42
temperature	number	A higher temperature increases randomness in the outputs.	No	1
tools	array of OpenAI.ChatCompletionTool		No
top_p	number	An alternative to temperature for nucleus sampling; 1.0 includes all tokens.	No	1

AzureResponsesSamplingParams

Sampling parameters for controlling the behavior of responses.

Name	Type	Description	Required	Default
max_tokens	integer (int32)	The maximum number of tokens in the generated output.	No
reasoning_effort	OpenAI.ReasoningEffort	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.	No
seed	integer	A seed value initializes the randomness during sampling.	No	42
temperature	number	A higher temperature increases randomness in the outputs.	No	1
text	OpenAI.CreateEvalResponsesRunDataSourceSamplingParamsText		No
tools	array of OpenAI.Tool		No
top_p	number	An alternative to temperature for nucleus sampling; 1.0 includes all tokens.	No	1

EvalGraderEndpoint

Name	Type	Description	Required	Default
headers	object or null	Optional HTTP headers to include in requests to the endpoint	No
name	string	The name of the grader	Yes
pass_threshold	number or null	Optional threshold score above which the grade is considered passing If not specified, all scores are considered valid	No
rate_limit	integer or null	Optional rate limit for requests per second to the endpoint Must be a positive integer	No
type	enum	Possible values: `endpoint`	Yes	endpoint
url	string	The HTTPS URL of the endpoint to call for grading Constraints: pattern: `^https://`	Yes

OpenAI.ApplyPatchToolParam

Apply patch tool

Allows the assistant to create, delete, or update files using unified diffs.

Name	Type	Description	Required	Default
type	enum	The type of the tool. Always `apply_patch`. Possible values: `apply_patch`	Yes	apply_patch

OpenAI.ApproximateLocation

Name	Type	Description	Required	Default
city	string or null		No
country	string or null		No
region	string or null		No
timezone	string or null		No
type	enum	The type of location approximation. Always `approximate`. Possible values: `approximate`	Yes	approximate

OpenAI.ChatCompletionTool

Function tool

A function tool that can be used to generate a response.

Name	Type	Description	Required	Default
function	OpenAI.FunctionObject		Yes
type	enum	The type of the tool. Currently, only `function` is supported. Possible values: `function`	Yes

OpenAI.CodeInterpreterContainerAuto

CodeInterpreterToolAuto

Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.

Name	Type	Description	Required	Default
file_ids	array of string	An optional list of uploaded files to make available to your code. Constraints: maxItems: 50	No
memory_limit	OpenAI.ContainerMemoryLimit or null		No
type	enum	Always `auto`. Possible values: `auto`	Yes	auto

OpenAI.CodeInterpreterTool

Code interpreter

A tool that runs Python code to help generate a response to a prompt.

Name	Type	Description	Required	Default
container	string or OpenAI.CodeInterpreterContainerAuto	The code interpreter container. Can be a container ID or an object that specifies uploaded file IDs to make available to your code, along with an optional `memory_limit` setting.	Yes
type	enum	The type of the code interpreter tool. Always `code_interpreter`. Possible values: `code_interpreter`	Yes

OpenAI.ComparisonFilter

Comparison Filter

A filter used to compare a specified attribute key to a given value using a defined comparison operation.

Name	Type	Description	Required	Default
key	string	The key to compare against the value.	Yes
type	enum	Specifies the comparison operator: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`. - `eq`: equals - `ne`: not equal - `gt`: greater than - `gte`: greater than or equal - `lt`: less than - `lte`: less than or equal - `in`: in - `nin`: not in Possible values: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`	Yes	eq
value	string or number or boolean or array of OpenAI.ComparisonFilterValueItems	The value to compare against the attribute key; supports string, number, or boolean types.	Yes

OpenAI.ComparisonFilterValueItems

This schema accepts one of the following types:

string
number

OpenAI.CompoundFilter

Compound Filter

Combine multiple filters using and or or.

Name	Type	Description	Required	Default
filters	array of OpenAI.ComparisonFilter or object	Array of filters to combine. Items can be `ComparisonFilter` or `CompoundFilter`.	Yes
type	enum	Type of operation: `and` or `or`. Possible values: `and`, `or`	Yes

OpenAI.ComputerEnvironment

Property	Value
Type	string
Values	`windows` `mac` `linux` `ubuntu` `browser`

OpenAI.ComputerUsePreviewTool

Computer use preview

A tool that controls a virtual computer.

Name	Type	Description	Required	Default
display_height	integer	The height of the computer display.	Yes
display_width	integer	The width of the computer display.	Yes
environment	OpenAI.ComputerEnvironment		Yes
type	enum	The type of the computer use tool. Always `computer_use_preview`. Possible values: `computer_use_preview`	Yes	computer_use_preview

OpenAI.ContainerMemoryLimit

Property	Value
Type	string
Values	`1g` `4g` `16g` `64g`

OpenAI.CreateChatCompletionRequestResponseFormat

An object specifying the format that the model must output. Setting to { "type": "json_schema", "json_schema": {...} } enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the Structured Outputs guide. Setting to { "type": "json_object" } enables the older JSON mode, which ensures the message the model generates is valid JSON. Using json_schema is preferred for models that support it.

Discriminator for OpenAI.CreateChatCompletionRequestResponseFormat

This component uses the property type to discriminate between different types:

Type Value	Schema
`json_schema`	OpenAI.ResponseFormatJsonSchema
`text`	OpenAI.CreateChatCompletionRequestResponseFormatResponseFormatText
`json_object`	OpenAI.CreateChatCompletionRequestResponseFormatResponseFormatJsonObject

Name	Type	Description	Required	Default
type	OpenAI.CreateChatCompletionRequestResponseFormatType		Yes

OpenAI.CreateChatCompletionRequestResponseFormatResponseFormatJsonObject

JSON object

JSON object response format. An older method of generating JSON responses. Using json_schema is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so.

Name	Type	Description	Required	Default
type	enum	The type of response format being defined. Always `json_object`. Possible values: `json_object`	Yes

OpenAI.CreateChatCompletionRequestResponseFormatResponseFormatText

Text

Default response format. Used to generate text responses.

Name	Type	Description	Required	Default
type	enum	The type of response format being defined. Always `text`. Possible values: `text`	Yes

OpenAI.CreateChatCompletionRequestResponseFormatType

Property	Value
Type	string
Values	`text` `json_schema` `json_object`

OpenAI.CreateEvalCompletionsRunDataSource

CompletionsRunDataSource

A CompletionsRunDataSource object describing a model sampling configuration.

Name	Type	Description	Required	Default
input_messages	OpenAI.CreateEvalCompletionsRunDataSourceInputMessagesTemplate or OpenAI.CreateEvalCompletionsRunDataSourceInputMessagesItemReference	Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.	No
model	string	The name of the model to use for generating completions (e.g. "o3-mini").	No
sampling_params	AzureCompletionsSamplingParams	Sampling parameters for controlling the behavior of completions.	No
source	OpenAI.EvalJsonlFileContentSource or OpenAI.EvalJsonlFileIdSource or OpenAI.EvalStoredCompletionsSource	Determines what populates the `item` namespace in this run's data source.	Yes
type	enum	The type of run data source. Always `completions`. Possible values: `completions`	Yes	completions

OpenAI.CreateEvalCompletionsRunDataSourceInputMessagesItemReference

Name	Type	Description	Required	Default
item_reference	string		Yes
type	enum	Possible values: `item_reference`	Yes

OpenAI.CreateEvalCompletionsRunDataSourceInputMessagesTemplate

Name	Type	Description	Required	Default
template	array of OpenAI.EasyInputMessage or OpenAI.EvalItem		Yes
type	enum	Possible values: `template`	Yes

OpenAI.CreateEvalCompletionsRunDataSourceSamplingParams

Name	Type	Description	Required	Default
max_completion_tokens	integer		No
reasoning_effort	OpenAI.ReasoningEffort	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.	No
response_format	OpenAI.ResponseFormatText or OpenAI.ResponseFormatJsonSchema or OpenAI.ResponseFormatJsonObject		No
seed	integer	A seed value initializes the randomness during sampling.	No	42
temperature	number	A higher temperature increases randomness in the outputs.	No	1
tools	array of OpenAI.ChatCompletionTool		No
top_p	number	An alternative to temperature for nucleus sampling; 1.0 includes all tokens.	No	1

OpenAI.CreateEvalCustomDataSourceConfig

CustomDataSourceConfig

A CustomDataSourceConfig object that defines the schema for the data source used for the evaluation runs. This schema is used to define the shape of the data that will be:

Used to define your testing criteria and
What data is required when creating a run

Name	Type	Description	Required	Default
include_sample_schema	boolean	Whether the eval should expect you to populate the sample namespace (ie, by generating responses off of your data source)	No
item_schema	object	The json schema for each row in the data source.	Yes
type	enum	The type of data source. Always `custom`. Possible values: `custom`	Yes	custom

OpenAI.CreateEvalItem

SimpleInputMessage

A chat message that makes up the prompt or context. May include variable references to the item namespace, ie {{item.name}}.

Name	Type	Description	Required	Default
content	string	The content of the message.	Yes
role	string	The role of the message (e.g. "system", "assistant", "user").	Yes

OpenAI.CreateEvalJsonlRunDataSource

JsonlRunDataSource

A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

Name	Type	Description	Required	Default
source	OpenAI.EvalJsonlFileContentSource or OpenAI.EvalJsonlFileIdSource	Determines what populates the `item` namespace in the data source.	Yes
type	enum	The type of data source. Always `jsonl`. Possible values: `jsonl`	Yes	jsonl

OpenAI.CreateEvalLabelModelGrader

LabelModelGrader

A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.

Name	Type	Description	Required
input	array of OpenAI.CreateEvalItem	A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}.	Yes
labels	array of string	The labels to classify to each item in the evaluation.	Yes
model	string	The model to use for the evaluation. Must support structured outputs.	Yes
name	string	The name of the grader.	Yes
passing_labels	array of string	The labels that indicate a passing result. Must be a subset of labels.	Yes
type	enum	The object type, which is always `label_model`. Possible values: `label_model`	Yes

OpenAI.CreateEvalLogsDataSourceConfig

LogsDataSourceConfig

A data source config which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc.

Name	Type	Description	Required	Default
metadata	object	Metadata filters for the logs data source.	No
type	enum	The type of data source. Always `logs`. Possible values: `logs`	Yes	logs

OpenAI.CreateEvalResponsesRunDataSource

CreateEvalResponsesRunDataSource

A ResponsesRunDataSource object describing a model sampling configuration.

Name	Type	Description	Required	Default
input_messages	OpenAI.CreateEvalResponsesRunDataSourceInputMessagesTemplate or OpenAI.CreateEvalResponsesRunDataSourceInputMessagesItemReference	Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace.	No
model	string	The name of the model to use for generating completions (e.g. "o3-mini").	No
sampling_params	AzureResponsesSamplingParams	Sampling parameters for controlling the behavior of responses.	No
source	OpenAI.EvalJsonlFileContentSource or OpenAI.EvalJsonlFileIdSource or OpenAI.EvalResponsesSource	Determines what populates the `item` namespace in this run's data source.	Yes
type	enum	The type of run data source. Always `responses`. Possible values: `responses`	Yes	responses

OpenAI.CreateEvalResponsesRunDataSourceInputMessagesItemReference

Name	Type	Description	Required	Default
item_reference	string		Yes
type	enum	Possible values: `item_reference`	Yes

OpenAI.CreateEvalResponsesRunDataSourceInputMessagesTemplate

Name	Type	Description	Required	Default
template	array of object or OpenAI.EvalItem		Yes
type	enum	Possible values: `template`	Yes

OpenAI.CreateEvalResponsesRunDataSourceSamplingParams

Name	Type	Description	Required	Default
reasoning_effort	OpenAI.ReasoningEffort	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.	No
seed	integer	A seed value initializes the randomness during sampling.	No	42
temperature	number	A higher temperature increases randomness in the outputs.	No	1
text	OpenAI.CreateEvalResponsesRunDataSourceSamplingParamsText		No
tools	array of OpenAI.Tool		No
top_p	number	An alternative to temperature for nucleus sampling; 1.0 includes all tokens.	No	1

OpenAI.CreateEvalResponsesRunDataSourceSamplingParamsText

Name	Type	Description	Required	Default
format	OpenAI.TextResponseFormatConfiguration	An object specifying the format that the model must output. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the The default format is `{ "type": "text" }` with no additional options. Not recommended for gpt-4o and newer models: Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it.	No

OpenAI.CreateEvalRunRequest

CreateEvalRunRequest

Name	Type	Description	Required
data_source	OpenAI.CreateEvalJsonlRunDataSource or OpenAI.CreateEvalCompletionsRunDataSource or OpenAI.CreateEvalResponsesRunDataSource	Details about the run's data source.	Yes
metadata	OpenAI.Metadata or null		No
name	string	The name of the run.	No

OpenAI.CreateEvalStoredCompletionsDataSourceConfig

StoredCompletionsDataSourceConfig

Deprecated in favor of LogsDataSourceConfig.

Name	Type	Description	Required	Default
metadata	object	Metadata filters for the stored completions data source.	No
type	enum	The type of data source. Always `stored_completions`. Possible values: `stored_completions`	Yes	stored_completions

OpenAI.CustomGrammarFormatParam

Grammar format

A grammar defined by the user.

Name	Type	Description	Required	Default
definition	string	The grammar definition.	Yes
syntax	OpenAI.GrammarSyntax1		Yes
type	enum	Grammar format. Always `grammar`. Possible values: `grammar`	Yes	grammar

OpenAI.CustomTextFormatParam

Text format

Unconstrained free-form text.

Name	Type	Description	Required	Default
type	enum	Unconstrained text format. Always `text`. Possible values: `text`	Yes	text

OpenAI.CustomToolParam

Custom tool

A custom tool that processes input using a specified format. Learn more about custom tools

Name	Type	Description	Required	Default
description	string	Optional description of the custom tool, used to provide more context.	No
format	OpenAI.CustomToolParamFormat	The input format for the custom tool. Default is unconstrained text.	No
└─ type	OpenAI.CustomToolParamFormatType		Yes
name	string	The name of the custom tool, used to identify it in tool calls.	Yes
type	enum	The type of the custom tool. Always `custom`. Possible values: `custom`	Yes	custom

OpenAI.CustomToolParamFormat

The input format for the custom tool. Default is unconstrained text.

Discriminator for OpenAI.CustomToolParamFormat

This component uses the property type to discriminate between different types:

Type Value	Schema
`text`	OpenAI.CustomTextFormatParam
`grammar`	OpenAI.CustomGrammarFormatParam

Name	Type	Description	Required	Default
type	OpenAI.CustomToolParamFormatType		Yes

OpenAI.CustomToolParamFormatType

Property	Value
Type	string
Values	`text` `grammar`

OpenAI.EasyInputMessage

Input message

A message input to the model with a role indicating instruction following hierarchy. Instructions given with the developer or system role take precedence over instructions given with the user role. Messages with the assistant role are presumed to have been generated by the model in previous interactions.

Name	Type	Description	Required
content	string or OpenAI.InputMessageContentList	Text, image, or audio input to the model, used to generate a response. Can also contain previous assistant responses.	Yes
role	enum	The role of the message input. One of `user`, `assistant`, `system`, or `developer`. Possible values: `user`, `assistant`, `system`, `developer`	Yes
type	enum	The type of the message input. Always `message`. Possible values: `message`	Yes

OpenAI.Eval

Eval

An Eval object with a data source config and testing criteria. An Eval represents a task to be done for your LLM integration. Like:

Improve the quality of my chatbot
See how well my chatbot handles customer support
Check if o4-mini is better at my usecase than gpt-4o

Name	Type	Description	Required	Default
created_at	integer (unixtime)	The Unix timestamp (in seconds) for when the eval was created.	Yes
data_source_config	OpenAI.EvalCustomDataSourceConfig or OpenAI.EvalLogsDataSourceConfig or OpenAI.EvalStoredCompletionsDataSourceConfig	Configuration of data sources used in runs of the evaluation.	Yes
id	string	Unique identifier for the evaluation.	Yes
metadata	OpenAI.Metadata or null		Yes
name	string	The name of the evaluation.	Yes
object	enum	The object type. Possible values: `eval`	Yes	eval
testing_criteria	array of OpenAI.CreateEvalLabelModelGrader or OpenAI.EvalGraderStringCheck or OpenAI.EvalGraderTextSimilarity or OpenAI.EvalGraderPython or OpenAI.EvalGraderScoreModel or EvalGraderEndpoint	A list of testing criteria.	Yes

OpenAI.EvalApiError

EvalApiError

An object representing an error response from the Eval API.

Name	Type	Description	Required	Default
code	string	The error code.	Yes
message	string	The error message.	Yes

OpenAI.EvalCustomDataSourceConfig

CustomDataSourceConfig

A CustomDataSourceConfig which specifies the schema of your item and optionally sample namespaces. The response schema defines the shape of the data that will be:

Used to define your testing criteria and
What data is required when creating a run

Name	Type	Description	Required	Default
schema	object	The json schema for the run data source items. Learn how to build JSON schemas here.	Yes
type	enum	The type of data source. Always `custom`. Possible values: `custom`	Yes	custom

OpenAI.EvalGraderPython

EvalGraderPython

Name	Type	Description	Required
image_tag	string	The image tag to use for the python script.	No
name	string	The name of the grader.	Yes
pass_threshold	number	The threshold for the score.	No
source	string	The source code of the python script.	Yes
type	enum	The object type, which is always `python`. Possible values: `python`	Yes

OpenAI.EvalGraderScoreModel

EvalGraderScoreModel

Name	Type	Description	Required	Default
input	array of OpenAI.EvalItem	The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings.	Yes
model	string	The model to use for the evaluation.	Yes
name	string	The name of the grader.	Yes
pass_threshold	number	The threshold for the score.	No
range	array of number	The range of the score. Defaults to `[0, 1]`.	No
sampling_params	OpenAI.EvalGraderScoreModelSamplingParams		No
└─ max_completions_tokens	integer or null		No
└─ reasoning_effort	OpenAI.ReasoningEffort	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.	No
└─ seed	integer or null		No
└─ temperature	number or null		No
└─ top_p	number or null		No	1
type	enum	The object type, which is always `score_model`. Possible values: `score_model`	Yes

OpenAI.EvalGraderScoreModelSamplingParams

Name	Type	Description	Required
max_completions_tokens	integer or null		No
reasoning_effort	OpenAI.ReasoningEffort	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.	No
seed	integer or null		No
temperature	number or null		No
top_p	number or null		No

OpenAI.EvalGraderStringCheck

StringCheckGrader

Name	Type	Description	Required
input	string	The input text. This may include template strings.	Yes
name	string	The name of the grader.	Yes
operation	enum	The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. Possible values: `eq`, `ne`, `like`, `ilike`	Yes
reference	string	The reference text. This may include template strings.	Yes
type	enum	The object type, which is always `string_check`. Possible values: `string_check`	Yes

OpenAI.EvalGraderTextSimilarity

EvalGraderTextSimilarity

Name	Type	Description	Required	Default
evaluation_metric	enum	The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`. Possible values: `cosine`, `fuzzy_match`, `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, `rouge_l`	Yes
input	string	The text being graded.	Yes
name	string	The name of the grader.	Yes
pass_threshold	number	The threshold for the score.	Yes
reference	string	The text being graded against.	Yes
type	enum	The type of grader. Possible values: `text_similarity`	Yes	text_similarity

OpenAI.EvalItem

EvalItem

Name	Type	Description	Required
content	OpenAI.EvalItemContent	Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.	Yes
role	enum	The role of the message input. One of `user`, `assistant`, `system`, or `developer`. Possible values: `user`, `assistant`, `system`, `developer`	Yes
type	enum	The type of the message input. Always `message`. Possible values: `message`	No

OpenAI.EvalItemContent

Eval content

Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

Type: OpenAI.EvalItemContentItem or OpenAI.EvalItemContentArray

Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.

OpenAI.EvalItemContentArray

An array of Input text, Output text, Input image, and Input audio

A list of inputs, each of which may be either an input text, output text, input image, or input audio object.

Array of: OpenAI.EvalItemContentItem

OpenAI.EvalItemContentItem

Eval content item

A single content item: input text, output text, input image, or input audio.

Type: OpenAI.EvalItemContentText or OpenAI.EvalItemContentItemObject

A single content item: input text, output text, input image, or input audio.

OpenAI.EvalItemContentItemObject

Eval content item

A single content item: input text, output text, input image, or input audio.

Discriminator for OpenAI.EvalItemContentItemObject

This component uses the property type to discriminate between different types:

Type Value	Schema
`output_text`	OpenAI.EvalItemContentOutputText
`input_image`	OpenAI.EvalItemInputImage
`input_audio`	OpenAI.InputAudio
`input_text`	OpenAI.EvalItemContentItemObjectInputTextContent

Name	Type	Description	Required	Default
type	OpenAI.EvalItemContentItemObjectType		Yes

OpenAI.EvalItemContentItemObjectInputTextContent

Input text

A text input to the model.

Name	Type	Description	Required	Default
text	string	The text input to the model.	Yes
type	enum	The type of the input item. Always `input_text`. Possible values: `input_text`	Yes	input_text

OpenAI.EvalItemContentItemObjectType

Property	Value
Type	string
Values	`input_text` `output_text` `input_image` `input_audio`

OpenAI.EvalItemContentOutputText

Output text

A text output from the model.

Name	Type	Description	Required	Default
text	string	The text output from the model.	Yes
type	enum	The type of the output text. Always `output_text`. Possible values: `output_text`	Yes

OpenAI.EvalItemContentText

Text input

A text input to the model.

Type: string

OpenAI.EvalItemInputImage

Input image

An image input block used within EvalItem content arrays.

Name	Type	Description	Required
detail	string	The detail level of the image to be sent to the model. One of `high`, `low`, or `auto`. Defaults to `auto`.	No
image_url	string (uri)	The URL of the image input.	Yes
type	enum	The type of the image input. Always `input_image`. Possible values: `input_image`	Yes

OpenAI.EvalJsonlFileContentSource

EvalJsonlFileContentSource

Name	Type	Description	Required	Default
content	array of OpenAI.EvalJsonlFileContentSourceContent	The content of the jsonl file.	Yes
type	enum	The type of jsonl source. Always `file_content`. Possible values: `file_content`	Yes	file_content

OpenAI.EvalJsonlFileContentSourceContent

Name	Type	Description	Required	Default
item	object		Yes
sample	object		No

OpenAI.EvalJsonlFileIdSource

EvalJsonlFileIdSource

Name	Type	Description	Required	Default
id	string	The identifier of the file.	Yes
type	enum	The type of jsonl source. Always `file_id`. Possible values: `file_id`	Yes	file_id

OpenAI.EvalList

EvalList

An object representing a list of evals.

Name	Type	Description	Required	Default
data	array of OpenAI.Eval	An array of eval objects.	Yes
first_id	string	The identifier of the first eval in the data array.	Yes
has_more	boolean	Indicates whether there are more evals available.	Yes
last_id	string	The identifier of the last eval in the data array.	Yes
object	enum	The type of this object. It is always set to "list". Possible values: `list`	Yes	list

OpenAI.EvalLogsDataSourceConfig

LogsDataSourceConfig

A LogsDataSourceConfig which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc. The schema returned by this data source config is used to defined what variables are available in your evals. item and sample are both defined when using this data source config.

Name	Type	Description	Required	Default
metadata	OpenAI.Metadata or null		No
schema	object	The json schema for the run data source items. Learn how to build JSON schemas here.	Yes
type	enum	The type of data source. Always `logs`. Possible values: `logs`	Yes	logs

OpenAI.EvalResponsesSource

EvalResponsesSource

A EvalResponsesSource object describing a run data source configuration.

Name	Type	Description	Required
created_after	integer or null		No
created_before	integer or null		No
instructions_search	string or null		No
metadata	object or null		No
model	string or null		No
reasoning_effort	OpenAI.ReasoningEffort or null		No
temperature	number or null		No
tools	array of string or null		No
top_p	number or null		No
type	enum	The type of run data source. Always `responses`. Possible values: `responses`	Yes
users	array of string or null		No

OpenAI.EvalRun

EvalRun

A schema representing an evaluation run.

Name	Type	Description	Required	Default
created_at	integer (unixtime)	Unix timestamp (in seconds) when the evaluation run was created.	Yes
data_source	OpenAI.CreateEvalJsonlRunDataSource or OpenAI.CreateEvalCompletionsRunDataSource or OpenAI.CreateEvalResponsesRunDataSource	Information about the run's data source.	Yes
error	OpenAI.EvalApiError	An object representing an error response from the Eval API.	Yes
eval_id	string	The identifier of the associated evaluation.	Yes
id	string	Unique identifier for the evaluation run.	Yes
metadata	OpenAI.Metadata or null		Yes
model	string	The model that is evaluated, if applicable.	Yes
name	string	The name of the evaluation run.	Yes
object	enum	The type of the object. Always "eval.run". Possible values: `eval.run`	Yes	eval.run
per_model_usage	array of OpenAI.EvalRunPerModelUsage	Usage statistics for each model during the evaluation run.	Yes
per_testing_criteria_results	array of OpenAI.EvalRunPerTestingCriteriaResults	Results per testing criteria applied during the evaluation run.	Yes
report_url	string (uri)	The URL to the rendered evaluation run report on the UI dashboard.	Yes
result_counts	OpenAI.EvalRunResultCounts		Yes
└─ errored	integer		Yes
└─ failed	integer		Yes
└─ passed	integer		Yes
└─ total	integer		Yes
status	string	The status of the evaluation run.	Yes

OpenAI.EvalRunList

EvalRunList

An object representing a list of runs for an evaluation.

Name	Type	Description	Required	Default
data	array of OpenAI.EvalRun	An array of eval run objects.	Yes
first_id	string	The identifier of the first eval run in the data array.	Yes
has_more	boolean	Indicates whether there are more evals available.	Yes
last_id	string	The identifier of the last eval run in the data array.	Yes
object	enum	The type of this object. It is always set to "list". Possible values: `list`	Yes	list

OpenAI.EvalRunOutputItem

EvalRunOutputItem

A schema representing an evaluation run output item.

Name	Type	Description	Required	Default
created_at	integer (unixtime)	Unix timestamp (in seconds) when the evaluation run was created.	Yes
datasource_item	object	Details of the input data source item.	Yes
datasource_item_id	integer	The identifier for the data source item.	Yes
eval_id	string	The identifier of the evaluation group.	Yes
id	string	Unique identifier for the evaluation run output item.	Yes
object	enum	The type of the object. Always "eval.run.output_item". Possible values: `eval.run.output_item`	Yes	eval.run.output_item
results	array of OpenAI.EvalRunOutputItemResult	A list of grader results for this output item.	Yes
run_id	string	The identifier of the evaluation run associated with this output item.	Yes
sample	OpenAI.EvalRunOutputItemSample		Yes
└─ error	OpenAI.EvalApiError	An object representing an error response from the Eval API.	Yes
└─ finish_reason	string		Yes
└─ input	array of OpenAI.EvalRunOutputItemSampleInput		Yes
└─ max_completion_tokens	integer		Yes
└─ model	string		Yes
└─ output	array of OpenAI.EvalRunOutputItemSampleOutput		Yes
└─ seed	integer		Yes
└─ temperature	number		Yes
└─ top_p	number		Yes
└─ usage	OpenAI.EvalRunOutputItemSampleUsage		Yes
status	string	The status of the evaluation run.	Yes

OpenAI.EvalRunOutputItemList

EvalRunOutputItemList

An object representing a list of output items for an evaluation run.

Name	Type	Description	Required	Default
data	array of OpenAI.EvalRunOutputItem	An array of eval run output item objects.	Yes
first_id	string	The identifier of the first eval run output item in the data array.	Yes
has_more	boolean	Indicates whether there are more eval run output items available.	Yes
last_id	string	The identifier of the last eval run output item in the data array.	Yes
object	enum	The type of this object. It is always set to "list". Possible values: `list`	Yes	list

OpenAI.EvalRunOutputItemResult

EvalRunOutputItemResult

A single grader result for an evaluation run output item.

Name	Type	Description	Required
name	string	The name of the grader.	Yes
passed	boolean	Whether the grader considered the output a pass.	Yes
sample	object or null	Optional sample or intermediate data produced by the grader.	No
score	number	The numeric score produced by the grader.	Yes
type	string	The grader type (for example, "string-check-grader").	No

OpenAI.EvalRunOutputItemSample

Name	Type	Description	Required
error	OpenAI.EvalApiError	An object representing an error response from the Eval API.	Yes
finish_reason	string		Yes
input	array of OpenAI.EvalRunOutputItemSampleInput		Yes
max_completion_tokens	integer		Yes
model	string		Yes
output	array of OpenAI.EvalRunOutputItemSampleOutput		Yes
seed	integer		Yes
temperature	number		Yes
top_p	number		Yes
usage	OpenAI.EvalRunOutputItemSampleUsage		Yes

OpenAI.EvalRunOutputItemSampleInput

Name	Type	Description	Required	Default
content	string		Yes
role	string		Yes

OpenAI.EvalRunOutputItemSampleOutput

Name	Type	Description	Required	Default
content	string		No
role	string		No

OpenAI.EvalRunOutputItemSampleUsage

Name	Type	Required
cached_tokens	integer	Yes
completion_tokens	integer	Yes
prompt_tokens	integer	Yes
total_tokens	integer	Yes

OpenAI.EvalRunPerModelUsage

Name	Type	Required
cached_tokens	integer	Yes
completion_tokens	integer	Yes
invocation_count	integer	Yes
model_name	string	Yes
prompt_tokens	integer	Yes
total_tokens	integer	Yes

OpenAI.EvalRunPerTestingCriteriaResults

Name	Type	Required
failed	integer	Yes
passed	integer	Yes
testing_criteria	string	Yes

OpenAI.EvalRunResultCounts

Name	Type	Required
errored	integer	Yes
failed	integer	Yes
passed	integer	Yes
total	integer	Yes

OpenAI.EvalStoredCompletionsDataSourceConfig

StoredCompletionsDataSourceConfig

Deprecated in favor of LogsDataSourceConfig.

Name	Type	Description	Required	Default
metadata	OpenAI.Metadata or null		No
schema	object	The json schema for the run data source items. Learn how to build JSON schemas here.	Yes
type	enum	The type of data source. Always `stored_completions`. Possible values: `stored_completions`	Yes	stored_completions

OpenAI.EvalStoredCompletionsSource

StoredCompletionsRunDataSource

A StoredCompletionsRunDataSource configuration describing a set of filters

Name	Type	Description	Required	Default
created_after	integer or null		No
created_before	integer or null		No
limit	integer or null		No
metadata	OpenAI.Metadata or null		No
model	string or null		No
type	enum	The type of source. Always `stored_completions`. Possible values: `stored_completions`	Yes	stored_completions

OpenAI.FileSearchTool

File search

A tool that searches for relevant content from uploaded files.

Name	Type	Description	Required	Default
filters	OpenAI.Filters or null		No
max_num_results	integer	The maximum number of results to return. This number should be between 1 and 50 inclusive.	No
ranking_options	OpenAI.RankingOptions		No
└─ hybrid_search	OpenAI.HybridSearchOptions	Weights that control how reciprocal rank fusion balances semantic embedding matches versus sparse keyword matches when hybrid search is enabled.	No
└─ ranker	OpenAI.RankerVersionType	The ranker to use for the file search.	No
└─ score_threshold	number	The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.	No
type	enum	The type of the file search tool. Always `file_search`. Possible values: `file_search`	Yes	file_search
vector_store_ids	array of string	The IDs of the vector stores to search.	Yes

OpenAI.Filters

Type: OpenAI.ComparisonFilter or OpenAI.CompoundFilter

OpenAI.FunctionObject

Name	Type	Description	Required
description	string	A description of what the function does, used by the model to choose when and how to call the function.	No
name	string	The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.	Yes
parameters	OpenAI.FunctionParameters	The parameters the functions accepts, described as a JSON Schema object. See the guide for examples, and the JSON Schema reference for documentation about the format. Omitting `parameters` defines a function with an empty parameter list.	No
strict	boolean or null		No

OpenAI.FunctionParameters

The parameters the functions accepts, described as a JSON Schema object. See the guide for examples, and the JSON Schema reference for documentation about the format. Omitting parameters defines a function with an empty parameter list.

Type: object

OpenAI.FunctionShellToolParam

Shell tool

A tool that allows the model to execute shell commands.

Name	Type	Description	Required	Default
type	enum	The type of the shell tool. Always `shell`. Possible values: `shell`	Yes	shell

OpenAI.FunctionTool

Function

Defines a function in your own code the model can choose to call.

Name	Type	Description	Required	Default
description	string or null		No
name	string	The name of the function to call.	Yes
parameters	object or null		Yes
strict	boolean or null		Yes
type	enum	The type of the function tool. Always `function`. Possible values: `function`	Yes	function

OpenAI.GrammarSyntax1

Property	Value
Type	string
Values	`lark` `regex`

OpenAI.HybridSearchOptions

Name	Type	Description	Required	Default
embedding_weight	number	The weight of the embedding in the reciprocal ranking fusion.	Yes
text_weight	number	The weight of the text in the reciprocal ranking fusion.	Yes

OpenAI.ImageDetail

Property	Value
Type	string
Values	`low` `high` `auto`

OpenAI.ImageGenTool

Image generation tool

A tool that generates images using the GPT image models.

Valid models:

gpt-image-1
gpt-image-1-mini

Name	Type	Description	Required	Default
background	enum	Background type for the generated image. One of `transparent`, `opaque`, or `auto`. Default: `auto`. Possible values: `transparent`, `opaque`, `auto`	No	auto
input_fidelity	OpenAI.InputFidelity or null		No
input_image_mask	OpenAI.ImageGenToolInputImageMask		No
└─ file_id	string		No
└─ image_url	string		No
model	string (see valid models below)		No
moderation	enum	Moderation level for the generated image. Default: `auto`. Possible values: `auto`, `low`	No	auto
output_compression	integer	Compression level for the output image. Default: 100. Constraints: min: 0, max: 100	No	100
output_format	enum	The output format of the generated image. One of `png`, `webp`, or `jpeg`. Default: `png`. Possible values: `png`, `webp`, `jpeg`	No	png
partial_images	integer	Number of partial images to generate in streaming mode, from 0 (default value) to 3. Constraints: min: 0, max: 3	No
quality	enum	The quality of the generated image. One of `low`, `medium`, `high`, or `auto`. Default: `auto`. Possible values: `low`, `medium`, `high`, `auto`	No	auto
size	enum	The size of the generated image. One of `1024x1024`, `1024x1536`, `1536x1024`, or `auto`. Default: `auto`. Possible values: `1024x1024`, `1024x1536`, `1536x1024`, `auto`	No	auto
type	enum	The type of the image generation tool. Always `image_generation`. Possible values: `image_generation`	Yes	image_generation

OpenAI.ImageGenToolInputImageMask

Name	Type	Description	Required	Default
file_id	string		No
image_url	string		No

OpenAI.InputAudio

Input audio

An audio input to the model.

Name	Type	Description	Required	Default
input_audio	OpenAI.InputAudioInputAudio		Yes
type	enum	The type of the input item. Always `input_audio`. Possible values: `input_audio`	Yes

OpenAI.InputAudioInputAudio

Name	Type	Description	Required	Default
data	string		Yes
format	enum	Possible values: `mp3`, `wav`	Yes

OpenAI.InputContent

Discriminator for OpenAI.InputContent

This component uses the property type to discriminate between different types:

Type Value	Schema
`input_text`	OpenAI.InputContentInputTextContent
`input_image`	OpenAI.InputContentInputImageContent
`input_file`	OpenAI.InputContentInputFileContent

Name	Type	Description	Required	Default
type	OpenAI.InputContentType		Yes

OpenAI.InputContentInputFileContent

Input file

A file input to the model.

Name	Type	Description	Required	Default
file_data	string	The content of the file to be sent to the model.	No
file_id	string or null		No
file_url	string (uri)	The URL of the file to be sent to the model.	No
filename	string	The name of the file to be sent to the model.	No
type	enum	The type of the input item. Always `input_file`. Possible values: `input_file`	Yes	input_file

OpenAI.InputContentInputImageContent

Input image

An image input to the model. Learn about image inputs.

Name	Type	Description	Required	Default
detail	OpenAI.ImageDetail		Yes
file_id	string or null		No
image_url	string or null		No
type	enum	The type of the input item. Always `input_image`. Possible values: `input_image`	Yes	input_image

OpenAI.InputContentInputTextContent

Input text

A text input to the model.

Name	Type	Description	Required	Default
text	string	The text input to the model.	Yes
type	enum	The type of the input item. Always `input_text`. Possible values: `input_text`	Yes	input_text

OpenAI.InputContentType

Property	Value
Type	string
Values	`input_text` `input_image` `input_file`

OpenAI.InputFidelity

Control how much effort the model will exert to match the style and features, especially facial features, of input images. This parameter is only supported for gpt-image-1. Unsupported for gpt-image-1-mini. Supports high and low. Defaults to low.

Property	Value
Type	string
Values	`high` `low`

OpenAI.InputItem

Discriminator for OpenAI.InputItem

This component uses the property type to discriminate between different types:

Type Value	Schema
`message`	OpenAI.EasyInputMessage
`item_reference`	OpenAI.ItemReferenceParam

Name	Type	Description	Required	Default
type	OpenAI.InputItemType		Yes

OpenAI.InputItemType

Property	Value
Type	string
Values	`message` `item_reference`

OpenAI.InputMessageContentList

Input item content list

A list of one or many input items to the model, containing different content types.

Array of: OpenAI.InputContent

OpenAI.ItemReferenceParam

Item reference

An internal identifier for an item to reference.

Name	Type	Description	Required	Default
id	string	The ID of the item to reference.	Yes
type	enum	The type of item to reference. Always `item_reference`. Possible values: `item_reference`	Yes	item_reference

OpenAI.LocalShellToolParam

Local shell tool

A tool that allows the model to execute shell commands in a local environment.

Name	Type	Description	Required	Default
type	enum	The type of the local shell tool. Always `local_shell`. Possible values: `local_shell`	Yes	local_shell

OpenAI.MCPTool

MCP tool

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.

Name	Type	Description	Required
allowed_tools	array of string or OpenAI.MCPToolFilter or null		No
authorization	string	An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.	No
connector_id	enum	Identifier for service connectors, like those available in ChatGPT. One of `server_url` or `connector_id` must be provided. Learn more about service connectors here. Currently supported `connector_id` values are: - Dropbox: `connector_dropbox` - Gmail: `connector_gmail` - Google Calendar: `connector_googlecalendar` - Google Drive: `connector_googledrive` - Microsoft Teams: `connector_microsoftteams` - Outlook Calendar: `connector_outlookcalendar` - Outlook Email: `connector_outlookemail` - SharePoint: `connector_sharepoint` Possible values: `connector_dropbox`, `connector_gmail`, `connector_googlecalendar`, `connector_googledrive`, `connector_microsoftteams`, `connector_outlookcalendar`, `connector_outlookemail`, `connector_sharepoint`	No
headers	object or null		No
require_approval	OpenAI.MCPToolRequireApproval or string or null		No
server_description	string	Optional description of the MCP server, used to provide more context.	No
server_label	string	A label for this MCP server, used to identify it in tool calls.	Yes
server_url	string (uri)	The URL for the MCP server. One of `server_url` or `connector_id` must be provided.	No
type	enum	The type of the MCP tool. Always `mcp`. Possible values: `mcp`	Yes

OpenAI.MCPToolFilter

MCP tool filter

A filter object to specify which tools are allowed.

Name	Type	Description	Required	Default
read_only	boolean	Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with `readOnlyHint`, it will match this filter.	No
tool_names	array of string	List of allowed tool names.	No

OpenAI.MCPToolRequireApproval

Name	Type	Description	Required	Default
always	OpenAI.MCPToolFilter	A filter object to specify which tools are allowed.	No
never	OpenAI.MCPToolFilter	A filter object to specify which tools are allowed.	No

OpenAI.Metadata

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

Type: object

OpenAI.RankerVersionType

Property	Value
Type	string
Values	`auto` `default-2024-11-15`

OpenAI.RankingOptions

Name	Type	Description	Required
hybrid_search	OpenAI.HybridSearchOptions		No
└─ embedding_weight	number	The weight of the embedding in the reciprocal ranking fusion.	Yes
└─ text_weight	number	The weight of the text in the reciprocal ranking fusion.	Yes
ranker	OpenAI.RankerVersionType		No
score_threshold	number	The score threshold for the file search, a number between 0 and 1. Numbers closer to 1 will attempt to return only the most relevant results, but may return fewer results.	No

OpenAI.ReasoningEffort

Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, and xhigh. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.

gpt-5.1 defaults to none, which does not perform reasoning. The supported reasoning values for gpt-5.1 are none, low, medium, and high. Tool calls are supported for all reasoning values in gpt-5.1.
All models before gpt-5.1 default to medium reasoning effort, and do not support none.
The gpt-5-pro model defaults to (and only supports) high reasoning effort.
xhigh is supported for all models after gpt-5.1-codex-max.

Property	Value
Description	Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`.
Type	string
Values	`none` `minimal` `low` `medium` `high` `xhigh`

OpenAI.ResponseFormatJsonObject

JSON object

Name	Type	Description	Required	Default
type	enum	The type of response format being defined. Always `json_object`. Possible values: `json_object`	Yes

OpenAI.ResponseFormatJsonSchema

JSON schema

JSON Schema response format. Used to generate structured JSON responses. Learn more about Structured Outputs.

Name	Type	Description	Required
json_schema	OpenAI.ResponseFormatJsonSchemaJsonSchema		Yes
└─ description	string		No
└─ name	string		Yes
└─ schema	OpenAI.ResponseFormatJsonSchemaSchema	The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas here.	No
└─ strict	boolean or null		No
type	enum	The type of response format being defined. Always `json_schema`. Possible values: `json_schema`	Yes

OpenAI.ResponseFormatJsonSchemaJsonSchema

Name	Type	Description	Required
description	string		No
name	string		Yes
schema	OpenAI.ResponseFormatJsonSchemaSchema	The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas here.	No
strict	boolean or null		No

OpenAI.ResponseFormatJsonSchemaSchema

JSON schema

The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas here.

Type: object

OpenAI.ResponseFormatText

Text

Default response format. Used to generate text responses.

Name	Type	Description	Required	Default
type	enum	The type of response format being defined. Always `text`. Possible values: `text`	Yes

OpenAI.SearchContextSize

Property	Value
Type	string
Values	`low` `medium` `high`

OpenAI.TextResponseFormatConfiguration

An object specifying the format that the model must output. Configuring { "type": "json_schema" } enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the

The default format is { "type": "text" } with no additional options. Not recommended for gpt-4o and newer models: Setting to { "type": "json_object" } enables the older JSON mode, which ensures the message the model generates is valid JSON. Using json_schema is preferred for models that support it.

Discriminator for OpenAI.TextResponseFormatConfiguration

This component uses the property type to discriminate between different types:

Type Value	Schema
`json_schema`	OpenAI.TextResponseFormatJsonSchema
`text`	OpenAI.TextResponseFormatConfigurationResponseFormatText
`json_object`	OpenAI.TextResponseFormatConfigurationResponseFormatJsonObject

Name	Type	Description	Required	Default
type	OpenAI.TextResponseFormatConfigurationType		Yes

OpenAI.TextResponseFormatConfigurationResponseFormatJsonObject

JSON object

Name	Type	Description	Required	Default
type	enum	The type of response format being defined. Always `json_object`. Possible values: `json_object`	Yes

OpenAI.TextResponseFormatConfigurationResponseFormatText

Text

Default response format. Used to generate text responses.

Name	Type	Description	Required	Default
type	enum	The type of response format being defined. Always `text`. Possible values: `text`	Yes

OpenAI.TextResponseFormatConfigurationType

Property	Value
Type	string
Values	`text` `json_schema` `json_object`

OpenAI.TextResponseFormatJsonSchema

JSON schema

JSON Schema response format. Used to generate structured JSON responses. Learn more about Structured Outputs.

Name	Type	Description	Required
description	string	A description of what the response format is for, used by the model to determine how to respond in the format.	No
name	string	The name of the response format. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.	Yes
schema	OpenAI.ResponseFormatJsonSchemaSchema	The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas here.	Yes
strict	boolean or null		No
type	enum	The type of response format being defined. Always `json_schema`. Possible values: `json_schema`	Yes

OpenAI.Tool

A tool that can be used to generate a response.

Discriminator for OpenAI.Tool

This component uses the property type to discriminate between different types:

Type Value	Schema
`code_interpreter`	OpenAI.CodeInterpreterTool
`function`	OpenAI.FunctionTool
`file_search`	OpenAI.FileSearchTool
`computer_use_preview`	OpenAI.ComputerUsePreviewTool
`web_search`	OpenAI.WebSearchTool
`mcp`	OpenAI.MCPTool
`image_generation`	OpenAI.ImageGenTool
`local_shell`	OpenAI.LocalShellToolParam
`shell`	OpenAI.FunctionShellToolParam
`custom`	OpenAI.CustomToolParam
`web_search_preview`	OpenAI.WebSearchPreviewTool
`apply_patch`	OpenAI.ApplyPatchToolParam

Name	Type	Description	Required	Default
type	OpenAI.ToolType		Yes

OpenAI.ToolType

Property	Value
Type	string
Values	`function` `file_search` `computer_use_preview` `web_search` `mcp` `code_interpreter` `image_generation` `local_shell` `shell` `custom` `web_search_preview` `apply_patch`

OpenAI.WebSearchApproximateLocation

Web search approximate location

The approximate location of the user.

Name	Type	Description	Required	Default
city	string or null		No
country	string or null		No
region	string or null		No
timezone	string or null		No
type	enum	The type of location approximation. Always `approximate`. Possible values: `approximate`	No	approximate

OpenAI.WebSearchPreviewTool

Web search preview

Note: web_search is not yet available via Azure OpenAI.

Name	Type	Description	Required	Default
search_context_size	OpenAI.SearchContextSize		No
type	enum	The type of the web search tool. One of `web_search_preview` or `web_search_preview_2025_03_11`. Possible values: `web_search_preview`	Yes	web_search_preview
user_location	OpenAI.ApproximateLocation or null		No

OpenAI.WebSearchTool

Web search

Note: web_search is not yet available via Azure OpenAI.

Name	Type	Description	Required	Default
filters	OpenAI.WebSearchToolFilters or null		No
search_context_size	enum	High level guidance for the amount of context window space to use for the search. One of `low`, `medium`, or `high`. `medium` is the default. Possible values: `low`, `medium`, `high`	No	medium
type	enum	The type of the web search tool. One of `web_search` or `web_search_2025_08_26`. Possible values: `web_search`	Yes	web_search
user_location	OpenAI.WebSearchApproximateLocation or null		No

OpenAI.WebSearchToolFilters

Name	Type	Description	Required	Default
allowed_domains	array of string or null		No

Last updated on 2026-06-11