Transformers is our natural language processing library and our hub is now open to all ML models, with support from libraries like Flair, Asteroid, ESPnet, Pyannote, and more to come.

Read documentation

huggingface@transformers:~

from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForMaskedLM.from_pretrained("bert-base-uncased")

Science

Our Research contributions

We’re on a journey to advance and democratize NLP for everyone. Along the way, we contribute to the development of technology for the better.

🌸

Multitask Prompted Training Enables Zero-Shot Task Generalization

Open source state-of-the-art zero-shot language model out of BigScience.

🐎

DistilBERT

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

A smaller, faster, lighter, cheaper version of BERT obtained via model distillation.

📚

HMTL

Hierarchical Multi-Task Learning

Learning embeddings from semantic tasks for multi-task learning. We have open-sourced code and a demo.

🐸

Dynamical Language Models

Meta-learning for language modeling

A meta learner is trained via gradient descent to continuously and dynamically update language model weights.

🤖

State of the art

Neuralcoref

Our open source coreference resolution library for coreference. You can train it on your own dataset and language.

🦄

Auto-complete your thoughts

Write with Transformers

This web app is the official demo of the Transformers repository's text generation capabilities.

Start writing