Translation
Translation is the task of producing the input text in another language.
Below, we'll walk through how we can use AdaptNLP's EasyTranslator
module to translate text with state-of-the-art models.
Getting Started
We'll first import the EasyTranslator
class from AdaptNLP:
from adaptnlp import EasyTranslator
Then we'll write some example text to use:
text = ["Machine learning will take over the world very soon.",
"Machines can speak in many languages.",]
Followed by instantiating the EasyTranslator
class:
translator = EasyTranslator()
Next we can translate our text. We pass in the text we wish to translate, optionally a prefix for the t5 model (only used with t5 models), a model name, and any keyword arguments from Transformers.PreTrainedModel.generate()
.
Here we'll pass in text
, have our model translate from English to German, and use the t5-small
model.
translations = translator.translate(text = text, t5_prefix="translate English to German", model_name_or_path="t5-small", mini_batch_size=1, min_length=0, max_length=100, early_stopping=True)
And we can look at the outputs:
print("Translations:\n")
for t in translations['translations']:
print(t, "\n")
Using the HFModelHub
we can search for any translation models in HuggingFace like so:
from adaptnlp import HFModelHub
hub = HFModelHub()
models = hub.search_model_by_task('translation'); models
From there we can pass in any HFModelResult
from it. Here we'll use the t5-small
again:
model = models[-1]
translations = translator.translate(text = text, t5_prefix="translate English to German", model_name_or_path=model, mini_batch_size=1, min_length=0, max_length=100, early_stopping=True)
And see that we get similar results:
print("Translations:\n")
for t in translations['translations']:
print(t, "\n")