HC2023-K2: Hardware for Deep Learning

Channel:hotchipsvideos · Length:1:05:27 · Published:2023-09-13 · Views:14884

Watch on YouTube

Summary

Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.

Why HN recommends it

HN points to it for an insider view of why NVIDIA's AI gains came from design rather than process, and for data-movement power costs.

The whole talk is brilliant and worth watching. — HN, 2025

Machine learning & deep learning · Computer architecture & CPUs

Subtitles (excerpt)

Welcome to Hot Chips 2023. Keynote two. Hardware for deep learning. Uh Bill has worked uh across the stack. Um very pro-prolific and and uh successful, accomplished research scientist. He has contributed seminal works on parallel computing architecture, networks, interconnects, routing, uh so the whole stack. Uh he has received pretty much all the awards we can get as computer architects, such as Morris Wilkes and Eckert Mauchly. He is a fellow of ACM and IEEE and is advising the US president. So um I think we've all witnessed the tremendous impact Bill had, especially in the last year with ChatGPT, these technologies that would not have been able or possible without Bill's work. And uh he's also a great example for technology transfer. He basically took the work on streaming processors he did with his students at Stanford and within 15 years turned it into the sixth most valuable company in the world. So, without further ado, uh Bill, please take the podium. Thank you. IS THIS MIC WORKING? SO ASSUMING UH YEAH, THE MIC SEEMS TO BE WORKING. SO, UM there probably isn't a person in the audience who hasn't spent the better part of a weekend playing with chat GPT asking it questions, asking it to write stories

On Hacker News

Neuromorphic computation has been hyped up for ~ 20 year by now. So far it has dramatically underperformed, at least vis-a-vis the hype. The article does not distinguish between training and inference. Google Edge TPUs [LINK] each one is capable of performing 4 trillion operations per second (4 TOPS), using 2 watts of power—that's 2 TOPS per watt. So inference is already cheaper than the 20 watts… — Footpost, 2025-06
I think your biggest mistake is ignoring the amount of software that runs on top of the hardware. There were times in the past where others had a higher number of FLOPS, but it didn’t make a dent in the usage of CUDA. My standard assumption about anything in tech (and beyond) is that you don’t win on first principles alone (“parallel design is well understood”). It’s the details that make you sta… — tverbeure, 2023-10
Says the hotchips keynote - [VIDEO] - data that is moved is about 100x higher power than data that is local You have a good point about intermediate data. This would not apply to all algorithms. — gdiamos, 2024-09

Related videos