Using the Translation API in AdaptNLP

Translation

Translation is the task of producing the input text in another language.

Below, we'll walk through how we can use AdaptNLP's EasyTranslator module to translate text with state-of-the-art models.

Getting Started

We'll first import the EasyTranslator class from AdaptNLP:

from adaptnlp import EasyTranslator

Then we'll write some example text to use:

text = ["Machine learning will take over the world very soon.",
        "Machines can speak in many languages.",]

Followed by instantiating the EasyTranslator class:

translator = EasyTranslator()

Next we can translate our text. We pass in the text we wish to translate, optionally a prefix for the t5 model (only used with t5 models), a model name, and any keyword arguments from Transformers.PreTrainedModel.generate().

Here we'll pass in text, have our model translate from English to German, and use the t5-small model.

translations = translator.translate(text = text, t5_prefix="translate English to German", model_name_or_path="t5-small", mini_batch_size=1, min_length=0, max_length=100, early_stopping=True)

And we can look at the outputs:

print("Translations:\n")
for t in translations['translations']:
    print(t, "\n")
Translations:

Das Maschinenlernen wird die Welt in Kürze übernehmen. 

Maschinen können in vielen Sprachen sprechen. 

Finding a Model with the Model Hub

Using the HFModelHub we can search for any translation models in HuggingFace like so:

from adaptnlp import HFModelHub

hub = HFModelHub()
models = hub.search_model_by_task('translation'); models
[Model Name: t5-11b, Tasks: [summarization, text2text-generation, translation],
 Model Name: t5-3b, Tasks: [summarization, text2text-generation, translation],
 Model Name: t5-base, Tasks: [summarization, text2text-generation, translation],
 Model Name: t5-large, Tasks: [summarization, text2text-generation, translation],
 Model Name: t5-small, Tasks: [summarization, text2text-generation, translation]]

From there we can pass in any HFModelResult from it. Here we'll use the t5-small again:

model = models[-1]
translations = translator.translate(text = text, t5_prefix="translate English to German", model_name_or_path=model, mini_batch_size=1, min_length=0, max_length=100, early_stopping=True)

And see that we get similar results:

print("Translations:\n")
for t in translations['translations']:
    print(t, "\n")
Translations:

Das Maschinenlernen wird die Welt in Kürze übernehmen. 

Maschinen können in vielen Sprachen sprechen.