Evaluations overview
Evaluations allow you to automatically assess the quality of your model by running translations with it and calculating the BLEU score. You can translate your files with up to three models at once, and compare your model with other available models (either your models or the SYSTRAN baseline models). You can also download the translated files for human evaluation.
Evaluations can also be a good way to check which SYSTRAN baseline model is best suited for your usage.
Main actions
Please refer to the following pages for more information on:
Evaluations overview table
The table displays an overview of all the Evaluations launched by members of your Entity. Click on the arrow on the left of a row to display the Evaluation summary.
Each column can be used to filter the table and display the following information:
The evaluation name
The language pair/language direction
The identity of the user who created the Evaluation
The Evaluation’s status (see the table below)
The Evaluation’s expiration date (see below)
The date the evaluation was launched
Evaluation details
Click on the arrow at the left of a table row to unfold it and display details about the evaluation. Each row corresponds to a test set, and each column to a model.
You can compare translations obtained with up to three models. The number of test sets depend on their respective size. The total size of uploaded files should not exceed 20MB.
Click on the circled arrow next to a file to download, or next to a score to download the corresponding translation.
The last row displays the average score obtained with each model on all the test sets.
Evaluation status
The table below describes the different Evaluation statuses.
| Status | Description |
|---|---|
| Running | The evaluation is currently running. |
| Completed | Translation and scoring of all the selected files have finished successfully. |
| Canceled | The user canceled the evaluation while it was running. |
| Error | Either the translation or the scoring failed. |
If your evaluation concerns more than one file or more than one model, there can be several statuses. In the example below, the user wanted to translate the same file with two models. One of the translation failed.
Evaluation expiration
Files used in Evaluations expire 3 days after the Evaluation ran, as are translation on SYSTRAN Translate PRO. Input (source and reference files) along with translations of the source files are deleted. Filenames and scores are kept in the database for tracking purposes.
Please download the translations if you need to use them later.
If you decide to delete the Evaluation, all the files, including scores, will be deleted.
On the Evaluations page, a tooltip is displayed when hovering over the clock icon at the top of the new expiration date column.
Traffic lights help you seeing quicky how much time is left before the evaluation expires.
| Color | Time left |
|---|---|
| 🔴 | 1 day or less |
| 🟠 | 2 days |
| 🟢 | 3 days |
| ⚪️ | Expired |
Refresh settings
Use this parameter to define how often the page should be updated. Click on the arrow and choose a value from the drop-down list.
Default value for this page is 60 seconds.