Memory on a computer is broken up into pages. Pages used to be one size (eg: 4096 bytes). For various reasons nowadays there are lots of page sizes. Depending on the hardware, in addition to a “base” page size you might have the additional ability to declare that certain parts of memory should be treated as pages of different sizes (eg: 8K, 64K, 256K, 1M, 4M, 16M, 256M…). The reason you’d use these “large” or “huge” pages is that in some cases it improves application performance by minimizing low level overheads and maximizing low level optimizations.
All nice in theory, but it’s been a highly manual choice that a programmer has to make: Which size do I make which pages? Knowing the answer probably requires more computer micro-architectural knowledge than most software developers have. Then there are all sorts of additional complications with practical stuff like getting hugepages, managing how many of which size pages are available for use, and being robust in the face of changes to that availability. For the most part there have been very few programs which use hugepages because of this complexity, which in the past even had annoying features like essentially requiring one to reboot in order to change some of these system configuration settings. And it’s just been too hard for the programmer unless they’ve got a fairly simple use of memory, like a large static-sized shared memory region (ie: DB2/Oracle/MySQL).
Things have progressed massively though. Huge page usability on linux has gotten much easier thanks especially to kernel and userspace utility work Mel Gorman has spearheaded over the past couple of years. Mel’s got a series on LWN discussing these advances. The articles cover a bit of the technical details behind how hugepages give your programs benefit, how to use libhugetlbfs with your applications, and how new utilities in libhugetlbfs allow you to quickly and easily do things like manage multiple pools of different pagesized pages, test whether your application benefits from different possible hugepage backings and actually exploit those benefits even without rewriting your application specifically for huge pages.
While you might want the holy grail of an omniscient operating system that automagically always has your application backed by the “right” page sizes in the right places, what we have now can truly be considered user-ready.
Specifically the first part covers the background context of how hardware and low level operating system software deals with memory. The second part gives a quick introduction to some of the interfaces available for using huge pages with the different types of memory regions an application can have (eg: text, data, BSS, heap, stack, shared memory, anonymous mmap’s).
And…as a teaser:
Subsequent articles in the series will be getting into the key commands for hugepage pool management, running an application with hugepages in different configurations and showing off how easy it is now to test/profile an application workload to see if hugepages would provide benefit. Mel’s written up specific examples of testing a couple well known benchmarks with hugepages and discusses how (and how much) hugepages works for certain types of workloads.
