In addition to a selection of powerful models, Airtrain lets you evaluate your own custom models.

Extract inferences

To run inferences from a custom model through Airtrain's evaluation methods, you are responsible for generating individual inferences and storing them in the input dataset.

After adding a "Custom model" configuration to your evaluation job, specify the name of the CSV or JSON field containing the inference into the "Extract inference column name" field.

Prompt templating

If your input examples were used to interpolate a prompt template, please specify the prompt template in the "Prompt" field in order to record the accurate final prompt that was fed to the model.