How do you run a pilot or proof of concept with a translation vendor?

Last reviewed: September 23, 2026

How should you scope a translation pilot?

Five decisions determine whether the result is usable.

• Volume — roughly 1,000 to 2,000 words per round. Large enough to show terminology and style behaviour, small enough that vendors will run it quickly and you can review it properly.
• Language selection — two or three target languages, chosen to include at least one you can have reviewed internally and at least one that is commercially important. Testing only languages nobody on your side reads produces an unverifiable result.
• Content selection — your messiest realistic content, not your cleanest. If strings with placeholders, regulated text, or a specific file format are in scope for the contract, they belong in the pilot.
• Inputs supplied — provide your glossary, style guide, and any existing translation memory. Withholding them to see what happens tests nothing you will ever experience in production.
• Scoring method — agree in advance how errors are recorded and weighted by severity. An industry-standard error framework gives you a number you can defend internally rather than an impression.

What service levels and quality figures should a pilot verify?

What are the steps in a two-round translation pilot?

This sequence is what turns a sample into a decision.

1. Define what a pass looks like before you send anything — write down the error types that would disqualify a vendor and the turnaround you need. A pilot without a stated pass condition becomes a debate afterwards.
2. Send round one with full inputs — roughly 1,000 words of real content in two or three languages, accompanied by your glossary, style guide, and translation memory, and with the turnaround clock stated.
3. Review against a severity scale, not a preference list — have a qualified reviewer mark each issue and rate it neutral, minor, major, or critical. Separate genuine errors from stylistic choices, because vendors can only act on the first.
4. Give written feedback and run round two — send the marked-up review and a second comparable sample. The question this round answers is whether corrections stick, which is the strongest single predictor of how the relationship will feel in year two.
5. Score both rounds and decide — compare the two vendors' round-two results, not their round-one results, and factor in how each handled being corrected.

Este enfoque se adapta a equipos que...

• Are choosing between two or more shortlisted vendors and need a defensible tiebreaker.
• Have content with specific terminology that generic quality claims do not address.
• Can put a qualified reviewer — internal or regional — on at least one target language.
• Need evidence for a stakeholder who was not part of the vendor conversations.
• Are moving from machine-only output to a workflow with human review and need to see the difference on their own content.

Cuando esto puede no ser la prioridad adecuada

• Nobody on your side can review any target language, and you have no regional colleague or third party to do it — the pilot will produce output you cannot assess.
• Your decision is genuinely driven by integration coverage or security review rather than by linguistic quality, in which case a technical proof of concept is the better test.
• You are translating a single short document, where the pilot would cost more coordination than the job itself.
• You have already run a pilot with this vendor on comparable content within the last year.

Evaluation checklist: questions to ask before starting a pilot

Is the pilot free, paid, or credited against a contract?
All three are normal. What matters is knowing before you start, and knowing whether a paid pilot obliges you to anything.

Will the pilot use the same translators who would work on our account?
Ask directly. A pilot staffed by a vendor's strongest available resource tells you about the vendor's ceiling, not about your account.

What quality control or vetting will you apply to the pilot content, and is it the same as in production?
If review steps are added for the pilot and removed afterwards, the result is not predictive.

How will we record and score errors?
Agree the method with the vendor up front. A shared severity scale makes the outcome discussable rather than adversarial.

Can we run a second round after feedback?
A vendor unwilling to be corrected and re-tested is answering an important question about the next three years.

How does a pilot work with Smartling, including for video?

Smartling publishes its own pilot design in How to run a translation test to evaluate a potential language service: a first phase of roughly 1,000 words across three languages, supplied with your translation memory, glossary, and style guide and returned within an agreed turnaround, followed by written feedback and a second phase of comparable volume that tests whether that feedback was applied. For a smaller self-serve test, the Core plan is free to start, which lets a team translate real content without a purchase decision first.

Quality in a pilot is scored rather than debated. Smartling's linguistic quality assurance rates each error neutral, minor, major, or critical, weighting minor, major and critical into a single quality score while neutral errors are recorded for tracking only. MQM scoring is available when an MQM-compatible schema template is used, which gives a pilot a number that can be set against another vendor's rather than an impression. The Translation Satisfaction Guarantee documents what is covered at each service tier and states plainly that a specific MQM score is not guaranteed, only indicative.

For a video or filmed-content pilot, scope it to subtitles rather than to voice-over on the first pass, since subtitle formats round-trip through a translation workflow and recorded audio does not. It is also worth being precise about "real-time" quality checks: in practice this means reviewing translated captions in context against the video before publication, not checking during playback. The subtitle formats, voice-over options, and wider media tooling are covered in multimedia localization services.

Preguntas relacionadas

• When should you move from free translation tools to a translation platform?
• How do you justify translation software to leadership?
• Is migrating to a new translation management system worth the cost?
• What does multimedia localization cover for video and audio content?