What Are Word Embeddings?

#word2vec #llm

Converting text into numbers is the first step in training any machine learning model for NLP tasks. While one-hot encoding and bag of words provide simple ways to represent text as numbers, they lack semantic and contextual understanding of words, making them unsuitable for NLP tasks like language translation and text generation. Embeddings help represent words as vectors that capture their semantic meaning.

In this video, I provide a detailed explanation of embeddings and popular embedding techniques like Word2Vec, along with custom embeddings used in Transformer architectures for language generation and other NLP tasks.

Watch Videos in Understanding Large Language Model:
1) Introduction to Large Language Model
   • LLMs Like ChatGPT, Explained Visually...  

2) Preparing dataset and Tokenization
   • LLM Training Starts Here: Dataset Pre...  

Timestamp:

0:00 - Intro
0:20 - Representing image into numbers
0:54 - Representing text into numbers
2:20 - One Hot Encoding
3:40 - Bag of Words (Unigram, Bigram and N-Gram)
4:59 - Semantic and Contextual Understanding of text
6:28 - Word Embeddings
9:44 - Visualizing Word2Vec Embeddings
10:30 - Word2Vec Training (CBOW and Skip-Gram)
14:46 - Embedding Layer in Transformer Architecture
17:16 - Positional Encoding
18:46 - Outro

Efficient Estimation of Word Representations in Vector Space:
Word2Vec Paper: https://arxiv.org/abs/1301.3781

Visualize Word2Vec Embeddings Here:
https://projector.tensorflow.org/