outline: deep

Pipelines

Pipelines are a core feature of TransformersPHP, designed to simplify the use of machine learning models for various natural language processing (NLP) tasks. They encapsulate the entire process of running a model, from input preprocessing to post-processing the output, making it easy to integrate advanced NLP capabilities into your applications.

Creating a Pipeline

To create a pipeline, you use the pipeline function, specifying the task you want to perform. Here's a basic example for sentiment analysis:

use function Codewithkyrian\Transformers\Pipelines\pipeline;

$classifier = pipeline('sentiment-analysis');

This initializes a pipeline for sentiment analysis, automatically handling model downloading, caching, input processing, and output interpretation.

Pipeline with Options

Besides passing the task you want to perform, you can also customize the pipeline instance creation with some additional options. For instance, you can specify different model to use instead of the default model:

$classifier = pipeline('sentiment-analysis', 'nlptown/bert-base-multilingual-uncased-sentiment');

Beyond the task and model name, you can further tailor your pipeline with additional named arguments. Here's a breakdown of these options for better clarity:

task

Specifies the task you wish the pipeline to execute. Refer to the list of supported tasks for available options.

modelName

Specifies the model to be used by the pipeline. You can use any ONNX model from the Hugging Face model repository that is compatible with the specified task. You can also use your custom models, provided you've prepared them as instructed. If not provided, the default model for the task will be used. Eg

$generator = pipeline('text-generation', 'Xenova/codegen-350M-mono');

quantized

A boolean value indicating whether to use a quantized version of the model. Quantization reduces the model size and speeds up inference but may slightly decrease accuracy. This option defaults to false.

config

Allows you to pass a custom configuration for the pipeline. This could include specific model parameters or preprocessing options. Providing a custom config can help tailor the pipeline's behavior to better fit your application' s requirements.

cacheDir

While it's typically recommended to set the cache directory globally, this allows you to modify the cache directory to save and look for models for this pipelie instance.

revision

This specified model version to use. It can be a branch name, a tag name, or a commit id. Since HuggingFace uses a git-based system for storing models and other artifacts, so revision can be any identifier allowed by git.

modelFilename

This specified the filename of the model in the repository. It's particularly used for decoder only models. It defaults to decoder_model_merged but you can set it to use another if the repository doesn't use that nomenclature.

Running a Pipeline

Once you've created a pipeline, running it is straightforward. All pipelines are designed to accept input text as their primary argument. Here's how to run a pipeline for some common NLP tasks.

Basic Usage

For tasks like sentiment analysis, text generation, or named entity recognition (NER), you typically provide a string or an array of strings as input. Here's an example using the sentiment analysis pipeline created earlier:

$result = $classifier("TransformersPHP makes NLP easy and accessible.");

Handling Multiple Inputs

Most pipelines can also process multiple inputs in a single call, which is especially useful for batch processing. Provide an array of strings to analyze multiple texts at once:

$results = $classifier([
    "I love using TransformersPHP for my projects.",
    "The weather today is dreadful."
]);

Additional Options

Additional arguments can be passed to the pipeline function to customize it's behavior, but they are hugely dependent on the task you're using the pipelines for. For example, for translation, you can specify the source and target languages:

$translator = pipeline('translation', 'Xenova/m2m100_418M');

$result = $translator('I love TransformersPHP!', srcLang: 'en', tgtLang: 'fr');

Details on the specific options available for each pipeline task are provided within the documentation for that task.

Pipeline Output

The output generated by a pipeline varies based on the task it's performing and the nature of the input provided. For example:

For the classifier with one input, the output can be:

['label' => 'POSITIVE',  'score' => 0.9995358059835]

and for the multiple input classifier:

[
    ['label' => 'POSITIVE',  'score' => 0.99980061678407],
    ['label' => 'NEGATIVE',  'score' => 0.99842234422764],
]

and for the translation task:

['translation_text' => 'J\'aime TransformersPHP!']

Supported Tasks

Natural Language Processing

Computer Vision

Audio

Multimodal

ge-to-Text](/image-to-text) | image-to-text | Output text from a given image. | ✅ | | Text-to-Image | text-to-image | Generates images from input text. | ❌ | | Visual Question Answering | visual-question-answering | Answering open-ended questions based on an image. | ❌ | | Zero-Shot Audio Classification | zero-shot-audio-classification | Classifying audios into classes that are unseen during training. | ❌ | | Zero-Shot Image Classification | zero-shot-image-classification | Classifying images into classes that are unseen during training. | ✅ | | Zero-Shot Object Detection | zero-shot-object-detection | Identify objects of classes that are unseen during training. | ✅ |

Supported Model Architectures

TransformersPHP supports a wide range of model architectures for various NLP tasks. If the specific model you're interested in isn't listed here, you can open an issue on the repository so we can add support for it. Here's a list of currently tested and supported model architectures:

1. ALBERT (from Google Research and the Toyota Technological Institute at Chicago) released with the paper ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, by Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut.
2. BART (from Facebook) released with the paper BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer.
3. BERT (from Google) released with the paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
4. BERT For Sequence Generation (from Google) released with the paper Leveraging Pre-trained Checkpoints for Sequence Generation Tasks by Sascha Rothe, Shashi Narayan, Aliaksei Severyn.
5. BERTweet (from VinAI Research) released with the paper BERTweet: A pre-trained language model for English Tweets by Dat Quoc Nguyen, Thanh Vu and Anh Tuan Nguyen.
6. BigBird-Pegasus (from Google Research) released with the paper Big Bird: Transformers for Longer Sequences by Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed.
7. BigBird-RoBERTa (from Google Research) released with the paper Big Bird: Transformers for Longer Sequences by Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed.
8. CLIP (from OpenAI) released with the paper Learning Transferable Visual Models From Natural Language Supervision by Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever.
9. CodeGen (from Salesforce) released with the paper A Conversational Paradigm for Program Synthesis by Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, Caiming Xiong.
10. ConvBERT (from YituTech) released with the paper ConvBERT: Improving BERT with Span-based Dynamic Convolution by Zihang Jiang, Weihao Yu, Daquan Zhou, Yunpeng Chen, Jiashi Feng, Shuicheng Yan.
11. DeBERTa (from Microsoft) released with the paper DeBERTa: Decoding-enhanced BERT with Disentangled Attention by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen.
12. DeBERTa-v2 (from Microsoft) released with the paper DeBERTa: Decoding-enhanced BERT with Disentangled Attention by Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen.
13. DETR (from Facebook) released with the paper End-to-End Object Detection with Transformers by Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko.
14. DistilBERT (from HuggingFace), released together with the paper DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter by Victor Sanh, Lysandre Debut and Thomas Wolf. The same method has been applied to compress GPT2 into DistilGPT2, RoBERTa into DistilRoBERTa, Multilingual BERT into DistilmBERT and a German version of DistilBERT.
15. Donut (from NAVER), released together with the paper OCR-free Document Understanding Transformer by Geewook Kim, Teakgyu Hong, Moonbin Yim, Jeongyeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, Seunghyun Park.
16. ELECTRA (from Google Research/Stanford University) released with the paper ELECTRA: Pre-training text encoders as discriminators rather than generators by Kevin Clark, Minh-Thang Luong, Quoc V. Le,....