Limoncello

1. Notes

Hardware prefetchers are too coarse-grained
- Reduces cache misses
- Results in higher memory latency when bandwidth utilization is high

Trend is that bandwidth is stagnant
Hardware prefetchers waste bandwidth
Paper gets experiment results by turning a set of prefetchers off
Software prefetching is more fine grained, wastes less bandwidth on wrong fetches and less latency due to queueing delays

CPU becomes throttled while waiting for memory
OS avoids scheduling extra processes to avoid performance cliff
Hardware prefetching wastes bandwidth by 10%
prefetching tuned to reduces cache miss rates but greatly increase bandwidth utilization

Inserts software prefetches
Can profile at the function granularity
Hardware prefetch-friendly = targets for software prefetching
- Predictable / continuous access
- Can be more precise since we know how the data will be accessed

Design parameters
- Prefetch address: where data will be prefetched
- Prefetch distance: how far in advance data will be prefetched
  - Longer = less accurate, but more timely
- Prefetch degree: how much to prefetch, how fine grained it is
Determining where to add prefetches
- Prioritize large data accesses
- Look at metrics at the instruction level
- Use microbenchmarks and sweep for best parameters
- e.g. memcpy copy, choose large data accesses and find the optimal prefetch distance/degree

For CPUs with high utilization rate, the throughput increased
- Due to less wasted memory bandwidth
- Lowers latency to memory
More CPU utilization
More precise prefetching
No need for warmup period