Watch and learn how Large Language Models like ChatGPT think and speak!
Explore the inner workings of OpenAI's Generative
Pretrained Transformer (GPT) Large Language Models (LLMs) in this engaging one-minute video that was made with our busy DoD workforce in mind. It demonstrates the simplified version of the Transformer architecture that was used for the GPT models.
The video explains how these models process input prompts
in the Context Window all at once, using Positional Encoding to preserve word
order.
The 'Attention Layers’ determine each word’s importance
to itself and to other words and provide context and understanding.
The model considers the content of the entire context window to predict
the next word. A visual demonstration shows the model's inference
process—predicting the next word in the sequence.
Each predicted word is autoregressively fed back
into the input, continuing the cycle until the sequence is complete. All these
technical details are presented with an animated demonstration, making the
complex technology understandable.
Please watch the video, "How AI Learns to Talk" for a more detailed
explanation: https://media.dau.edu/media/t/1_vje3ghev/310800832
The link to a course playlist of DAU recommended AI
courses is: https://dau.csod.com/ui/lms-learner-playlist/PlaylistDetails?playlistId=00118adb-20e1-4dc5-95a8-9ffd03ab7f70
You will need a DAU account to access these resources. If
you are a DoD member and need a DAU account and you can request one here: https://www.dau.edu/faq/p/New-DAU-Account
In actuality, this is more a demo of a Pre-Trained GPT model, rather than a finished model such as ChatGPT, that has also been Fine-Tuned to be an assistant and answers questions. ChatGPT would likely assume the input prompt, "Jack and Jill" was a question and first tell us of the famous nursery rhyme, before going on to say, "went up the hill...". Nevertheless, both work by predicting the next word as demonstrated here.
Note: the left half of the video's thumbnail is ChatGPT's self-portrait (via DALL-E 3).