BERT ( Bidirectional Encoder Representations from Transformers) is a deep learning model based on a Transformer Neural Network. BERT is very good at language tasks and used to be state-of-the-art in NLP benchmarks before LLMs.