(Welch Labs) The moment we stopped understanding AI [AlexNet]

Auto-generated summary of YouTube video. Summary: Here’s what I learned: Modern AI models like ChatGPT and AlexNet organize information in high-dimensional “embedding spaces,” which are essentially mathematical ways to represent concepts. AlexNet, published in 2012, was a breakthrough—it shocked the computer vision world by showing that deep neural networks could outperform traditional methods when scaled up. AlexNet used 60 million parameters and 1.3 million labeled images, a thousand times more than previous models, thanks to the rise of GPUs. ChatGPT takes this idea even further, stacking up to 120 transformer blocks and now using over a trillion parameters. A key insight is that these models don’t have explicit rules for intelligence; instead, they repeatedly apply simple matrix operations. For example, ChatGPT predicts the next word by mapping text to vectors, running them through layers, and outputting the final result—literally just the last column of its output matrix. AlexNet’s early layers detect edges and colors, while deeper layers identify complex concepts like faces, even though it was never told what a face is. Researchers can visualize what these layers “see” using techniques like activation maps and feature visualization. The main takeaway: The leap in AI performance came not from new algorithms, but from scaling up old ideas with more data and compute. The models learn rich, abstract representations—sometimes of things we don’t even have words for—just by training on massive datasets. Generated by YouTube Summariser