Summarization

class TransformersSummarizer[source]

TransformersSummarizer(tokenizer:PreTrainedTokenizer, model:PreTrainedModel) :: AdaptiveModel

Adaptive model for Transformer's Conditional Generation or Language Models (Transformer's T5 and Bart conditiional generation models have a language modeling head)

Parameters:

• tokenizer : <class 'transformers.tokenization_utils.PreTrainedTokenizer'>

  A tokenizer object from Huggingface's transformers (TODO)and tokenizers

• model : <class 'transformers.modeling_utils.PreTrainedModel'>

  A transformers Conditional Generation (Bart or T5) or Language model

TransformersSummarizer.load[source]

TransformersSummarizer.load(model_name_or_path:str)

Class method for loading and constructing this classifier

Parameters:

• model_name_or_path : <class 'str'>

  A key string of one of Transformer's pre-trained Summarizer Model

Returns:

• <class 'adaptnlp.model.AdaptiveModel'>

TransformersSummarizer.predict[source]

TransformersSummarizer.predict(text:Union[List[str], str], mini_batch_size:int=32, num_beams:int=4, min_length:int=0, max_length:int=128, early_stopping:bool=True, **kwargs)

Predict method for running inference using the pre-trained sequence classifier model

Parameters:

• text : typing.Union[typing.List[str], str]

  Sentences to run inference on

• mini_batch_size : <class 'int'>, optional

  Mini batch size

• num_beams : <class 'int'>, optional

  Number of beams for beam search. Must be between 1 and infinity. 1 means no beam search.

• min_length : <class 'int'>, optional

  The min length of the sequence to be generated

• max_length : <class 'int'>, optional

  The max length of the sequence to be generated. Between min_length and infinity

• early_stopping : <class 'bool'>, optional

  If set to True beam search is stopped when at least num_beams sentences finished per batch

• kwargs : <class 'inspect._empty'>

Returns:

• typing.List[str]

  A list of predicted summarizations

class EasySummarizer[source]

EasySummarizer()

Summarization Module

EasySummarizer.summarize[source]

EasySummarizer.summarize(text:Union[List[str], str], model_name_or_path:Union[str, HFModelResult]='t5-small', mini_batch_size:int=32, num_beams:int=4, min_length:int=0, max_length:int=128, early_stopping:bool=True, **kwargs)

Predict method for running inference using the pre-trained sequence classifier model

Parameters:

• text : typing.Union[typing.List[str], str]

  Sentences to run inference on

• model_name_or_path : typing.Union[str, adaptnlp.model_hub.HFModelResult], optional

  A model id or path to a pre-trained model repository or custom trained model directory

• mini_batch_size : <class 'int'>, optional

  Mini batch size

• num_beams : <class 'int'>, optional

  Number of beams for beam search. Must be between 1 and infinity. 1 means no beam search

• min_length : <class 'int'>, optional

  The max length of the sequence to be generated. Between min_length and infinity

• max_length : <class 'int'>, optional

  The max length of the sequence to be generated. Between min_length and infinity

• early_stopping : <class 'bool'>, optional

  If set to True beam search is stopped when at least num_beams sentences finished per batch

• kwargs : <class 'inspect._empty'>

Returns:

• typing.List[str]

  A list of predicted summaries