(MC)²: Lazy MemCopy at the Memory Controller

(MC)² is a lazy memory copy mechanism which can be used within memcpy-like functions to significantly reduce the CPU overhead for copies that are sparsely accessed. It can also hide copy latencies by enhancing the CPU’s ability to execute them asynchronously.

(MC)²’s lazy memcpy avoids copying data at the time of invocation. Instead, (MC)² tracks prospective copies. If copied data is later accessed by a CPU or the cache, (MC)² uses the tracking information to lazily execute a copy, when necessary. Placing (MC)² at the memory controller puts it at the perfect vantage point to eliminate the largest source of memcpy overhead—CPU stalls due to cache misses in the critical path—while imposing minimal overhead itself.

(MC)² consists of three main components: memory controller extensions that implement a lazy memcpy operation, a new instruction exposing the lazy memcpy, and a flexible software wrapper with semantics identical to memcpy. We implement and evaluate (MC)² in the gem5 simulator using a variety of microbenchmarks and workloads, including Google’s Protobuf, where (MC)² provides a 43% speedup and Linux huge page copy-on-write faults, where (MC)² provides 250$\times$ lower latency.

Code