Why do we need CPU cache

Transcript

1. Why do we need CPU cache?
2. Memory DRAM Dynamic Random Access Memory 1 cell = one transistor and one capacitor Facts: 1. Each DRAM cell must be refreshed every 64 ms or less 2. It takes time for RAS and CAS to organize access to cells. 3. Cells need time to return or update value because of 4. capacitor latency.
3. System Bus Facts: 1. System bus speed is crucial as it supply data to CPU or DMA. 2. Modern CPUs are fast. 3. Now system bus is 64 bits wide. Transfer rate = 8 bytes/transfer × 100 MHz × 4 transfers/cycle = 3200 MB/s AMD HyperTransport and Intel QuickPath Interconnect are used in modern PC
4. CPU Caches • One cell requires six transistors. There are variants with four transistors but they have disadvantages. • the cell state is stable, no refresh cycles are needed. • L1 Cache size is usually 1000 times less than Memory size • The CPU core is no longer directly connected to the main memory. • When a small memory content is needed by the processor the entire cache line is loaded into the L1d. Cache Line = 64 bytes. It takes 8 bus operations to fill cache line. Static RAM
5. CPU Caches To Where Cycles Register <= 1 L1d ~3 L2 ~14 Main Memory ~240 Wait state. CPU generally run much faster than the computer's other subsystems. There are tricks to supply data fast enough: • memory prefetch; • branch prediction; • instructions reordering (by compiler); • hyper threading.
6. Multi Cores / CPU Data READ flow: 1. Core1 asks L1d for data at address. 2. L1d asks for L2 for data. 3. L3 asks system bus for data. 4. If Core2 L3 has that data - it returns, otherwise memory should fullfit the request. WRITE behaviors: • write-through cache implementation; • write-back cache implementation.
7. Cache Coherence Developed over the years is the MESI cache coherency protocol (Modified, Exclusive, Shared, Invalid): • Invalid: The cache line is invalid, i.e., unused. • Exclusive: The cache line is not modified but known to not be loaded into any other processor's cache. • Shared: The cache line is not modified and might exist in another processor's cache. • Modified: The local processor has modified the cache line. This also implies it is the only copy in any cache.
8. Cache Effects • Working with small data set is fast. Sequential read from memory is also fast; • Writing to the same address by multiple threads will lead to contention; • False sharing. Cache Oblivious Algorithms (if working set is large - split it to smaller part first, make operation and then join parts) Specialized CPU instructions
9. Takeaway #1. Latency. https://gist.github.com/jboner/2841832 Latency Comparison Numbers (~2012) ---------------------------------- L1 cache reference 0.5 ns Branch mispredict 5 ns L2 cache reference 7 ns 14x L1 cache Mutex lock/unlock 25 ns Main memory reference 100 ns 20x L2 cache, 200x L1 Compress 1K bytes with Zippy 3,000 ns 3 us Send 1K bytes over 1 Gbps network 10,000 ns 10 us Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD Read 1 MB sequentially from memory 250,000 ns 250 us Round trip within same datacenter 500,000 ns 500 us Read 1 MB sequentially from SSD* 1,000,000 ns 1,000 us 1 ms ~1GB/sec SSD, 4X memory Disk seek 10,000,000 ns 10,000 us 10 ms 20x datacenter roundtrip Read 1 MB sequentially from disk 20,000,000 ns 20,000 us 20 ms 80x memory, 20X SSD Send packet CA->Netherlands->CA 150,000,000 ns 150,000 us 150 ms Notes ----- 1 ns = 10^-9 seconds 1 us = 10^-6 seconds = 1,000 ns 1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns
10. Links: • What every programmer should know about memory https://lwn.net/Articles/252125/ • Gallery of Processor Cache Effects http://igoro.com/archive/gallery-of-processor-cache-effects/ https://medium.com/@minimarcel/effect-of-cpu-caches-57db81490a7f • Wikipedia