In this workshop, we will take a historical journey through the ideas that eventually led to today’s large language models, such as ChatGPT and Claude.
Modern LLMs can seem overwhelmingly complex, but they didn’t appear all at once. Many of the basic ideas behind them have been developing for more than a century. By following this history step by step, we can better understand what language models do and how they evolved.
In part 1 of this workshop, we will begin with Andrey Markov’s early work on sequences and probability in 1913 and Claude Shannon’s influential work on information and language in 1948. From there, we will explore key ideas such as Markov chains, N-grams, and the next-word generation process using simple, accessible examples.
In part 2 of the workshop, which is not in this workshop, we will also look at how an artificial neuron can learn to predict the next word, an important step toward neural language models that later became the foundation of modern LLMs and we will see what perplexity is.
The workshop is designed for anyone who is curious about where today’s AI came from and how a century of ideas about language, probability, and computation helped make modern language models possible. This workshop targets students of any level who are interested in language models or history. No background in computer science or artificial intelligence is required. The hardest part is converting counted numbers to probabilities.
Our historical journey will cover roughly the period from 1918 to 1970s.
Register for the event on SB Engaged: https://cglink.me/25T/r389829