A Wealth of Computer Architecture (Part 1)

A decade ago, I embarked on a journey in computing. My fascination with Computer Architecture has only grown as I move forward. Over the years, I’ve gathered valuable pointers — this blog post is my way of sharing them with anyone interested. I learn best by ping-ponging between multiple sources until the ‘aha’ moment arrives, so expect some redundancies. While not exhaustive, these pointers reflect a decade of curiosity and learning. Notably, this doesn’t delve into the ongoing AI revolution—that’s a story for another day. For now, I remain deeply grateful to the brilliant minds driving advancements in this ever-evolving field and to those who distill its complexities for others to understand, some of whom I have had the pleasure of working with.

Legend: [V] = Video [B] = Blog [P] = Playlist

1. CPU Architecture

   1. Bits of Architecture [P]

   2. CS6810 - Rajeev Balasubramanian [P]

   3. Modern Microprocessors lightera

   4. Processor Architecture 101

   5. What's new in CPUs since the 80s?

   6. Modern CPU Microarchitecture: An Overview

   7. How CPUs are designed

   8. WTF is SIMD/SIMT/SMT

   9. ISA - ARM vs x86

   10. Understanding CPU Microarchitecture to Increase Performance [V]

   11. Microarchitecture Primer | Complimentary thread

   12. CMU 418/618: Parallel Computer Architecture and Programming

   13. CMU ECE 18-600 Foundations of Computer Systems

   14. To reinvent the processor blog

   15. Microprocessors by Erik Engheim [B]

   16. Easy Perf blog [B]

   17. Agner Fog CPU Blog [B]

   18. Moore's Law Hits the roof

   19. X86 internals [V]
       

2. GPU Architecture

   1. GPU Programming: When, Why and How?

   2. Graphics Beginners' Guide, Part 1 & Part2

   3. Cornell GPU Architecture

   4. Stanford CS149 I Parallel Computing I 2023 I Kayvon Fatahalian

   5. SIMD < SIMT < SMT: parallelism in NVIDIA GPUs

   6. Cuda Programming Guide

   7. Nvidia Deep Learning Architecture

   8. What Every Developer Should Know About GPU Computing

   9. Understanding GPU Arch by Cornell Virtual workshop

   10. An Intro to GPU Architecture and Programming Models [V]

   11. GPUs for Deep Learning

   12. Graphics Pipeline Performance

   13. Life of a Triangle: Nvidias Logical Pipeline

   14. Anadtech Nvidia Turing deepdive [Ray Tracing]

   15. NVIDIA GPU Architecture: from Pascal to Turing to Ampere

   16. Understanding Tensor Cores

   17. Kayvon Fatahlian

   18. Gentle introduction to GPUs inner workings

   19. Caroline Collange Gpu Lectures 2019/20

   20. Towards Microarchitectural Design of Nvidia GPUs

   21. Gpu Architecture & Models

   22. GPU Arch types explained

   23. GPU Programming: When, Why and How?

   24. A Primer On Efficient Rendering Algorithms & Clustered Shading

   25. Mesh and task shaders intro and basics

   26. Mesh Shaders explained

   27. OGL Shaders

   28. AMD GPU Hardware Basics

   29. A trip through the Graphics Pipeline

   30. Intel Gen11

   31. David Kanter - Why CUDA cores aren’t actual cores [V]
       

3. OS Performance Concepts

   1. Linux Performance by Brendann Greg [B]

   2. Huge Pages

   3. How Kernel manages your memory

   4. Swapping

   5. Paging

   6. Mars Pathfinder - Priority Inversion Problem
      

4. Memory Hierarchy

   1. Main Memory Primer

   2. Virtual memory by David Black-Schaffer [P]

   3. What Every Programmer Should Know About Memory

   4. Swap Memory

   5. Bandwidth vs Latency

   6. Memory Bandwidth Napkin Math

   7. Numa Deepdive
      

5. Vectorization

   1. SIMD vs SIMT vs SMT

   2. What is Vectorization

   3. SIMD < SIMT < SMT: parallelism in NVIDIA GPUs

   4. Impact on CPU Frequencies

   5. Apple AMX 

   6. MMX vs SSE vs AVX

   7. Vectorization walkthrough
      

6. Case Studies

   1. Pentium4 Microarchitecture

   2. Spectre vs MeltDown

   3. Why is Apple M1 so fast 

   4. Apple M1 Anandtech

   5. Skylake / Icelake / GoldenCove

   6. Nvidia Grace

   7. ChatGPT Hardware

   8. Which GPU(s) to Get for Deep Learning

   9. Fermi  & Volta & Hopper & Turing & Ampere & Ada Lovelace

   10. Evolution of Nvidia Gpu Arch
       

7. Power

   1. Power Consumption

   2. TechUnrwapped

   3. Why Intel Processors Draw More Power Than Expected: TDP and Turbo Explained

8. System Design

   1. CprE488 - Embedded Systems Design

   2. System Design Blogs
      

9. Conference Talks & Podcasts

   1. Jim Keller: Moore's Law, Microprocessors, and First Principles | Lex Fridman Podcast #70

   2. Jim Keller: The Future of Computing, AI, Life, and Consciousness | Lex Fridman Podcast #162

   3. David Patterson: Computer Architecture and Data Storage | Lex Fridman Podcast

   4. A Decade of Machine Learning Accelerators:Lessons Learned and Carbon Footprint by David Patterson [slides]

   5. Trends in Deep Learning Hardware: Bill Dally (NVIDIA)

   6. A New Golden Age for Computer Architecture by David Patterson - Publication & Lecture