provider Evals POST /evals
@utdk/openai /evals
Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and a datasource. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](/docs/guides/evals).

Try it

Authentication
Configure credentials for OpenAI API
Gateway
The gateway proxies requests and injects credentials server-side. Configure credentials above, then enter your gateway URL.

Saved automatically to browser storage.

createeval
POST/evals

Input

The name of the evaluation.

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

required

The configuration for the data source used for the evaluation runs.

required

A list of graders for all eval runs in this group.

Enter a gateway URL above to enable sending.

Code snippet
Updates live as you fill in the form above.

TypeScript

import openai from '@utdk/openai';

await openai.createeval()