Translation API for AdaptNLP

class TransformersTranslator[source]

TransformersTranslator(tokenizer:PreTrainedTokenizer, model:PreTrainedModel) :: AdaptiveModel

Adaptive model for Transformer's Conditional Generation or Language Models (Transformer's T5 and Bart conditional generation models have a language modeling head)

Usage:

>>> translator = TransformersTranslator.load('transformers-translator-model')
>>> translator.predict(text='Example text', mini_batch_size=32)

Parameters:

  • tokenizer - A tokenizer object from Huggingface's transformers (TODO)and tokenizers
  • model - A transformers Conditional Generation (Bart or T5) or Language model

class EasyTranslator[source]

EasyTranslator()

Translation Module

Usage:

>>> translator = EasyTranslator()
>>> translator.translate(text="translate this text", model_name_or_path="t5-small")

Usage Examples:

Sample code from: 08_translation.ipynb (View Notebook for more context)

text = ["Machine learning will take over the world very soon.",
        "Machines can speak in many languages.",]

translator = EasyTranslator()
translations = translator.translate(text = text, t5_prefix="translate English to German", model_name_or_path="t5-small", mini_batch_size=1, min_length=0, max_length=100, early_stopping=True)
test_eq(translations, ['Das Maschinenlernen wird die Welt in Kürze übernehmen.',
 'Maschinen können in vielen Sprachen sprechen.'])

Sample code from: 08a_tutorial.translation.ipynb (View Notebook for more context)

translator = EasyTranslator()