Why do caches improve performance in the memory hierarchy?

• A. Caches reduce latency by exploiting temporal and spatial locality.
• B. Caches always increase latency.
• C. Cache size does not affect performance.
• D. Only main memory matters for speed.

Answer: (D)

Caches speed up memory access by taking advantage of how programs reuse data. A cache is a small, fast storage layer between the CPU and main memory. When the CPU needs data, the cache is checked first. If the data is there (a cache hit), the CPU gets it quickly, which dramatically lowers latency. If the data isn’t there (a cache miss), the system fetches it from slower main memory, which takes longer, but that data is then kept in the cache for faster access if it’s needed again soon.

This works because of temporal locality—the likelihood that recently accessed data will be accessed again soon—and spatial locality—the likelihood that data near recently accessed data will be used next. By exploiting these patterns, caches reduce the average time to access memory, improving overall performance.

Other statements miss the point: caches don’t always increase latency, and cache size and design do affect performance. Main memory isn’t the sole determinant of speed; the presence and effectiveness of a cache directly shape how fast data can be retrieved.