provider Evals POST /evals
@utdk/openai /evals
Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and a datasource. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](/docs/guides/evals).

Try it

Authentication
Configure credentials for OpenAI API
createeval
POST/evals

Input

The name of the evaluation.

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

required

The configuration for the data source used for the evaluation runs.

required

A list of graders for all eval runs in this group.

Configure credentials above to enable sending.

Code snippet
Updates live as you fill in the form above.

TypeScript

import openai from '@utdk/openai';

await openai.createeval()