How to optimize memory access latency in NUMA architecture

NUMA memory access latency measures the time your processor spends retrieving data from DRAM. Local socket memory requests complete in 60 to 100 nanoseconds. Crossing UPI or QPI interconnects to remote sockets increases latency to 200 or 300 nanoseconds. This cross-socket traversal creates up to a 3x delay penalty for your workloads.

You optimize memory access latency by maximizing memory locality and eliminating remote socket traffic. Enforce CPU thread pinning, control page allocations, and apply topology-aware kernel tuning to guarantee peak system performance.

Topology Discovery and Latency Profiling

Mapping Locality Distances

You must map your system hardware layout before tuning memory access. The ACPI System Locality Distance Information Table (SLIT) provides relative access cost ratios between NUMA nodes. The Linux kernel reads this matrix and displays distance values using dimensionless integers. A standard local access baseline receives a value of 10. A remote node distance value of 21 indicates that memory requests on that remote socket incur double the latency of a local access.

Run numactl to inspect these relative locality distances:

numactl --hardware

This output displays the node inventory alongside the relative distance matrix.

Profiling Access Latency with Perf

You can measure real-time memory traffic using standard Linux profiling tools. Run numastat to compare memory allocation counters across your system nodes. High numbers in the numa_miss or other_node rows reveal frequent remote access penalties.

For precise hardware sampling, execute perf stat with memory access counters:

perf stat -e numa:numa_hit,numa:numa_miss ./your_application

This command profiles local node hits against remote node misses during process execution.

Evaluating Hardware Interconnect Traffic

Heavy cross-socket traffic saturates hardware interconnects like Intel UPI or AMD Infinity Fabric. You evaluate inter-socket bandwidth and latency bottlenecks using perf c2c (cache-to-cache) analysis. This tool pinpoints false sharing and tracks remote cacheline hits across sockets.

Tip: Monitor numa_foreign metrics in numastat to detect processes fetching data from outside their assigned local socket domain.

Thread Pinning and Memory Locality

Operating system schedulers frequently move active threads across different CPU cores to balance workload distribution. This default behavior destroys memory locality in NUMA systems. When a thread jumps to a core on a remote socket, it must access data stored on its previous node across interconnects. You can eliminate this latency penalty by binding your threads and memory allocations strictly to local hardware resources.

Binding Threads via Taskset

The default Linux scheduler prioritizes overall system throughput over thread-level memory locality. You can override this behavior using the taskset utility to control CPU affinity directly.

When you bind execution threads to specific CPU cores, you keep core L1, L2, and L3 caches warm. This practice prevents the operating system from migrating your processes across socket boundaries.

Execute the following command to bind a process to the first eight cores on NUMA Node 0:

taskset -c 0-7 ./your_application

You can also assign an already running process to specific cores using its process ID (PID):

taskset -p -c 0-7 12345

Thread pinning ensures that instructions execute near their local cache lines. However, taskset only controls CPU execution affinity. You must also manage physical memory allocation to optimize memory access patterns across nodes.

Restricting Allocations with Numactl

Thread pinning alone does not guarantee that your application allocates memory on the local node. A process running on Socket 0 might still allocate RAM on Socket 1 if default policies apply. You must enforce both execution and allocation rules simultaneously using numactl.

Run your workload with explicit CPU and memory node constraints:

numactl --cpunodebind=0 --membind=0 ./your_application

The --membind policy operates as a strict allocation constraint that restricts the allocation exclusively to the designated NUMA nodes. If memory cannot be secured on these specified nodes, the allocation process fails immediately rather than falling back to non-designated nodes.

If your application requires high availability, strict allocation failures might disrupt operations. You can use --preferred instead to create a flexible policy:

numactl --cpunodebind=0 --preferred=0 ./your_application

The --preferred flag instructs the kernel to allocate RAM on Node 0 first. The kernel falls back to remote nodes only when Node 0 runs out of available physical memory.

How to Optimize Memory Access via Migration

Workloads frequently expand beyond single-socket resource limits during peak utilization spikes. When your application spans multiple NUMA domains, static initial binding becomes insufficient. You must implement dynamic memory page migration strategies to maintain peak performance.

Tip: Use the migratepages tool during operational shifts to move full memory footprints between nodes without restarting active production services.

If a process shifts its main computing workload to Node 1, you can manually migrate its existing pages from Node 0 to Node 1:

migratepages 12345 0 1

This command scans process ID 12345, finds physical pages allocated on Node 0, and moves them to Node 1 in real time.

You can also rely on automated kernel mechanisms or custom application logic:

• Kernel AutoNUMA Migration: The Linux kernel continuously scans active process pages. It identifies remote accesses and automatically moves physical pages closer to the accessing thread.

• Explicit User-Space Migration: C/C++ applications can invoke the move_pages() system call to transfer specific memory addresses directly based on runtime metrics.

Dynamic page migration incurs immediate processing overhead during the page transfer phase. However, moving active datasets to local socket memory pays immediate dividends. You drastically reduce cross-socket UPI traffic and permanently optimize the access latency for long-running processes.

Kernel Allocation Policies and Page Tuning

Linux provides memory policies through system calls to help you control page placement across NUMA nodes. You configure these kernel mechanisms to optimize memory access performance for specific application behavior.

Local Allocation versus Interleaving

The Linux kernel supports several policy flags to dictate memory node selection:

• MPOL_BIND restricts the allocations strictly to a set of specified nodes.

• MPOL_PREFERRED targets a single node first. When the preferred NUMA node runs out of available RAM, the policy automatically falls back to allocating memory from other available NUMA nodes rather than returning an error.

• MPOL_INTERLEAVE distributes the allocations evenly across multiple nodes using round-robin page allocation.

While MPOL_BIND offers low local latency, MPOL_INTERLEAVE outperforms policies focused on local latency under specific memory access patterns:

• Large Memory Footprints: Allocations spanning a substantial memory area, typically 1 MB or larger.

• Uniform Access Patterns: Workloads where the requests are distributed evenly across the allocated memory region.

• Sequential or Streaming Access: Access patterns designed to maximize memory bandwidth by spreading page allocations and concurrent accesses across multiple NUMA nodes rather than restricting them to a single node.

Managing Transparent Huge Pages

Transparent Huge Pages (THP) reduce Translation Lookaside Buffer (TLB) misses by allocating 2 MB or 1 GB memory pages instead of standard 4 KB pages. However, real-time workloads often suffer from latency spikes when the kernel allocates or splits these massive pages across remote sockets. You should enable THP in madvise mode to give your application explicit control over huge page usage.

echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

Fine-Tuning AutoNUMA Balancing

AutoNUMA scans active thread memory locations periodically to optimize memory access across sockets. The kernel moves misplaced pages closer to the thread executing the access.

This automatic page scanning introduces CPU overhead and unpredictable latency jitter. Real-time systems should disable AutoNUMA balancing and rely on explicit static memory placement:

sysctl -w kernel.numa_balancing=0

Application and Shared Memory Tuning

Setting Pthread NUMA Affinity

You can optimize application memory access by programmatically binding C and C++ threads to specific NUMA nodes. Standard runtime systems rely on external scripts, but direct C-level thread binding delivers precise hardware control.

Use the POSIX thread library alongside the libnuma development package to enforce explicit affinity inside your application code:

#include <pthread.h>
#include <numa.h>

void bind_thread_to_node(pthread_t thread, int node_id) {
    struct bitmask *cpus = numa_allocate_cpumask();
    numa_node_to_cpus(node_id, cpus);
    pthread_setaffinity_np(thread, sizeof(cpu_set_t), (cpu_set_t *)cpus->maskp);
    numa_free_cpumask(cpus);
}

Binding worker threads during initialization prevents thread migration penalties. Combine thread pinning with numa_alloc_onnode() allocations to guarantee local cache locality.

Optimizing Shared Memory and IPC

Inter-process communication (IPC) through POSIX or System V shared memory requires explicit placement policies. Shared memory segments default to allocating physical memory on the NUMA node of the process that performs the first write operation. This default creates severe latency bottlenecks when multiple worker processes read data across sockets.

Apply explicit mbind() policies to balance shared structures:

#include <numaif.h>

mbind(shared_memory_ptr, size, MPOL_INTERLEAVE, nodemask, maxnode, MPOL_MF_MOVE);

This system call distributes shared cachelines evenly across your participating hardware sockets.

Configuring Container Runtime Affinity

Modern cloud environments run applications inside container runtimes like Docker and Kubernetes. Unconfigured container workloads dynamically float across all host sockets, introducing high interconnect latency.

Tip: Set the Kubernetes Kubelet --topology-manager-policy=single-numa-node flag to guarantee alignable CPU and memory allocations for latency-sensitive pods.

Configure Docker containers using static hardware CPU sets:

docker run -d --cpuset-cpus="0-7" --cpuset-mems="0" your_image

This command pins containerized processes strictly to Node 0 cores and local RAM blocks.

You eliminate cross-node memory latency by controlling where your data lives and where your threads run. Follow this technical checklist to optimize memory access across your systems:

• Map relative socket distances using numactl --hardware.

• Enforce strict CPU and memory affinity with taskset and numactl bindings.

• Select explicit runtime page allocation policies and tune AutoNUMA settings.

Tip: Track hardware performance counters continuously with perf to discover new cross-node interconnect bottlenecks quickly.

FAQ

How do you check if your workload suffers from NUMA latency?

Run numastat -c in your terminal to inspect system memory allocation counters. High numbers in the other_node column signal cross-socket memory access.

numastat -c

You can also run perf stat -e numa:numa_miss to measure real-time remote memory access events during execution.

What is the primary difference between taskset and numactl?

The taskset tool binds execution threads strictly to specific CPU cores. In contrast, numactl offers deeper control over both CPU affinity and physical RAM node placement. Use numactl to pin execution while enforcing local or interleaved allocation policies.

Should you always disable AutoNUMA for real-time applications?

Key Takeaway: Real-time systems demand predictable performance over dynamic balance.

Yes, disable AutoNUMA for real-time workloads. The kernel’s automated page scanner introduces CPU overhead and unpredictable latency jitter. Static thread pinning paired with explicit policies delivers far more consistent access times.

Does NUMA architecture impact PCIe device and NVMe performance?

PCIe slots wire directly to specific processor sockets. When a thread accesses a remote network interface card or NVMe drive, I/O traffic must traverse cross-socket interconnects. You optimize device throughput by pinning I/O-intensive threads directly to the local socket hosting the PCIe device.