HIERARCHICAL NUMA

*
Refereed Presentation
Scheduled: Thursday, September 14, 2017 from 2:00 – 2:45pm in Platinum D

One Line Summary

Redesigning the memory NUMA abstraction taking into account new memory technologies where inter node distance will no longer be the only deciding factor for memory allocation and placement.

Abstract

NUMA (Non Uniform Memory Access) traditionally has been built around
the assumption that inter node distance is the primary factor deciding
the applicable latency and also latency being the only property to be
considered for allocation and placement of memory. New emerging systems
come equipped with variety of memory devices connected with the core
through different kind of interconnects (industry standard and
proprietary) which breaks away from traditional notion of NUMA. Though
they are all coherently accessible from the processor, they possess
different properties like latency, bandwidth, reliability, typical
available size etc. The present notion of NUMA in kernel is unable to
accommodate these variety of properties present in these new memory
devices and give an appropriate abstraction for it’s usage which calls
for a redesign of NUMA for memory representation.

I have been working for some time on CDM (Coherent Device Memory)
representation as a NUMA node in the kernel and proposed various
solutions in this regard. The merits of the proposals got debated
and reviewed from current NUMA design and memory interface perspective,
it seems like while this may be a good solution for short and medium
term but kernel would definitely need a long term NUMA redesign to
accommodate emerging memory technologies.

This presentation will cover a proposed redesign of NUMA memory
representation in kernel which will accommodate different properties
of memory along side their inter nodal distances. It will cover kernel
expectations from the platform firmware how various properties of the
memory devices should be communicated, how they will be represented in
the kernel, how buddy allocator data structure need to be changed, how
NUMA memory API will be augmented to accommodate new user controllable
memory allocations based on requested properties etc. This presentation
will also give an overall idea about how this redesigned NUMA will
benefit user space, its control over memory allocations and its possible
impact on emerging and traditional workloads.

This presentation will try to help build consensus in the kernel
community towards a redesigned NUMA. Developers from kernel development
community especially from memory management background and others who
develop applications on heterogeneous systems will benefit from this
presentation.

Audio

Tags

memory management, NUMA

Presentation Materials

slides

Speaker

  • Photo_anshuman_khandual

    Anshuman Khandual

    IBM India Private Limited

    Biography

    I am a kernel development engineer at Linux Technology Centre (IBM India Pvt Ltd). Currently working on present and future Open POWER technology and platform enablement on Linux. At present working on coherent device memory representation in the kernel.

    My key focus areas include memory management, perf, transactional memory and platform enablement.

    Previously worked as kernel validation engineer during POWER8 development period focusing primarily on perf and performance monitoring unit (PMU) technology enablement. Worked as a memory RAS validation engineer during POWER7 development period.