Understanding Smart Model Selection

The purpose of this page is to give a broad overview of Language I/O’s automated Smart Model Selection (SMS) process for machine translation for agents, admins and potential customers interested in Language I/O solutions.

What does Smart Model Selection do?

Smart Model Selection is a process that takes translation quality feedback data from all customers to continuously evaluate the top-performing machine translation models by language pair. This means that the feedback for each language pair is evaluated independently. This data is used to adjust both global and per-customer engine priority.

What data does it evaluate to make decisions?

Language I/O evaluates top-performing MT models for each language pair through two main feedback channels:

• Retranslation data (when an agent requests a new translation of previously translated content)
• Quality Flag data (aka Quality Rating Flag, when an agent or end user flags a translation as unsatisfactory)

How often does Smart Model Selection evaluate model changes?

SMS updates model priorities once every seven days.

How much historical data does SMS reference when making global model update suggestion?

For global engine priority updates, global feedback is consolidated every seven days into a 90-day trailing average to determine the need for global engine priority changes.

How does Smart Model Selection adjust to my specific feedback?

Smart Model Selection evaluates both global feedback data and per-customer feedback data for each language pair. This means that, if you or your users provide multiple points of negative feedback for a given language pair during the course of a 7-day look-back window, then during SMS’s weekly performance evaluation, Smart Model Selection recommends an update to your engine priorities specific to that feedback, independent of any global engine updates.

This gives you the security of always having the best available engine, while allowing for the flexibility to quickly adjust the model on a case-by-case basis.

Example

Example of an SMS evaluation and priority update process for a single language pair. Each language pair is evaluated separately. (Click to enlarge)

How do SMS and Engine Failover work together?

If an engine becomes temporarily unavailable or takes too long to return a translation, the system seamlessly fails over to the next best engine, allowing you to always have a translation option available. For more information, see Understanding Engine Failovers.

Do I need to be aware of when translation engines are updated?

No, this is a seamless, behind-the-scenes process. As shown above, our per-customer evaluation can quickly adjust your specific engine priorities if any global changes are found to be sub-optimal for your use case.

Why should I keep it turned on?

This is the easiest, most seamless hands-off solution for you. Language I/O takes care of everything behind the scenes for you, so you can focus on your customer experience.

What are the alternatives?

If you do need to use a particular engine for a specific language pair, please reach out to the Language I/O support team for recommendations. Note that if specific engine preferences are configured, they are no longer eligible for SMS evaluation. In other words, while this gives you greater control over the engines you want to use, disabling SMS means that you lose the flexibility and optimization of your engine options.

The majority of Language I/O customers choose to let Language I/O manage model selection activities.