Evaluate a model
To evaluate a model, go to the Evaluations page. Click on + Create a new evaluation.
Evaluation form
A form such as the one below is displayed.
Fill the Evaluation form:
Choose a name for your evaluation. The name should not exceed 50 characters and can only contain alphanumeric characters and
.-_Select the language pair
Select the model(s) you want to evaluate. You should pick at least one model, and can compare translations obtained with up to three models
Select the files you want to translate. Either drag-and-drop them into the box, or click on the link in the box and load files from a file explorator (the total size of uploaded files should not exceed 20MB)
Click on the trash bin next to a file to remove it
Click on Submit. Clicking on Close or on the cross in the top right corner closes the form and the Evaluation is not launched
Note
You can choose the target locale when the feature is available with the model. For instance, most English target models have EN-GB, EN-US and EN-CA localization options.
Your translation should appear in the evaluations overview table. By default, the most recent evaluations are on the top rows of the table.
Each uploaded file is translated with each selected model, and each translation is scored. For instance, if you uploaded two files and selected two models, you should obtain four scores.
Warning
Files need to be bilingual, for ModelStudio to be able to extract the reference against which the translations will be compared to obtain a score.
As for corpora, only two formats are accepted : application/x-tmx+xml (TMX files) and text/plain (raw bitext).
Bitext files
Uploading bitext files on ModelStudio requires the file to have the following header.
#TM
#XX YY
Where XX stands for the source language code, and YY for the target language code.
The two language codes should be separated with a tabulation.
Evaluation expiration
Files used in Evaluations expire 3 days after the Evaluation ran, as are translation on SYSTRAN Translate PRO. Input (source and reference files) along with translations of the source files are deleted. Filenames and scores are kept in the database for tracking purposes.
Please download the translations if you need to use them later.
If you decide to delete the Evaluation, all the files, including scores, will be deleted.
On the Evaluations page, a tooltip is displayed when hovering over the clock icon at the top of the new expiration date column.
Traffic lights help you seeing quicky how much time is left before the evaluation expires.
| Color | Time left |
|---|---|
| 🔴 | 1 day or less |
| 🟠 | 2 days |
| 🟢 | 3 days |
| ⚪️ | Expired |