Watch and learn how Large Language Models like ChatGPT think and speak!

Explore the inner workings of OpenAI's Generative Pretrained Transformer (GPT) Large Language Models (LLMs) in this engaging one-minute video that was made with our busy DoD workforce in mind. It demonstrates the simplified version of the Transformer architecture that was used for the GPT models.

The video explains how these models process input prompts in the Context Window all at once, using Positional Encoding to preserve word order.

The 'Attention Layers’ determine each word’s importance to itself and to other words and provide context and understanding.

The model considers the content of the entire context window to predict the next word. A visual demonstration shows the model's inference process—predicting the next word in the sequence.

Each predicted word is autoregressively fed back into the input, continuing the cycle until the sequence is complete. All these technical details are presented with an animated demonstration, making the complex technology understandable.

Please watch the video, "How AI Learns to Talk" for a more detailed explanation: https://media.dau.edu/media/t/1_vje3ghev/310800832

The link to a course playlist of DAU recommended AI courses is: https://dau.csod.com/ui/lms-learner-playlist/PlaylistDetails?playlistId=00118adb-20e1-4dc5-95a8-9ffd03ab7f70

You will need a DAU account to access these resources. If you are a DoD member and need a DAU account and you can request one here: https://www.dau.edu/faq/p/New-DAU-Account

In actuality, this is more a demo of a Pre-Trained GPT model, rather than a finished model such as ChatGPT, that has also been Fine-Tuned to be an assistant and answers questions. ChatGPT would likely assume the input prompt, "Jack and Jill" was a question and first tell us of the famous nursery rhyme, before going on to say, "went up the hill...". Nevertheless, both work by predicting the next word as demonstrated here.

Note: the left half of the video's thumbnail is ChatGPT's self-portrait (via DALL-E 3).