Large Memory Pages for AIX
AIX expert Mark Ray gives a primer on large memory pages
In most modern OSes, it’s all about the memory. You can have the fastest processors on the planet driving the most sophisticated storage and have it all connected to a blazingly fast network. But what will happen if you take those three resources and couple them to inadequate amounts of memory? Try that little thought experiment for two seconds and you’ll all come up with the same answer: absolutely nothing. You’ll have a system that’s slow as molasses and good for little else than service as a rather large doorstop.
All flavors of UNIX do the same thing when you boot them. They will attempt to load as much of the OS—certainly the kernel, at the very least—into memory so system initialization is accomplished as quickly as possible. And when you start your applications, that code is probably modularized so that big chunks of it will follow the OS into RAM—or as many chunks of the code as will fit. Paging, the process of moving pieces of code from memory out to disk storage and back again, only occurs when a program can’t find a physical memory address range in which to plug the code and has to move previously running code out of that address range and store it on disk while it moves the most recently called code into memory. Severe paging can lead to extreme system slowness, and in the worst cases, can crash a system. And all of this can occur with the best CPUs, storage and network devices. So the first thing to do when installing a new system is to make sure you have your memory sized correctly, with a healthy margin over and above your initial requirement to cover you when your workload grows—and they always grow.
In this series, we’re going to take a look at how AIX uses memory, how to deploy and tune that memory for maximum performance, some of the most popular utilities to monitor memory usage in an AIX system, and how to parcel that memory out in different configurations. We’ll also see that we can make different code in the same system use memory in different ways, altering installed executables such that they will adopt new patterns of memory usage. With that, let’s take a look at some basics.
Memory Basics
In any computing system, Memory is allocated in “pages,” chunks that vary in size, depending on the OS. AIX supports four memory page sizes: small (4K), medium (64K), large (16MB) and supreme (16GB). The small memory page size is common to many OSes; it’s a legacy holdover from the early days of computing when memory was very expensive and programming code assumed small amounts of it. You’ll find the small, 4K page size in every version of UNIX, and that includes Linux. Most OSes started life supporting only the 4K size, but over time, they expanded their memory repertoire to include the usage of larger and larger memory page sizes.
In the case of AIX, it was found that coalescing many 4K pages into a larger 64K—or medium-sized—page significantly increased system performance. So the default memory scheme in AIX evolved into one that supports both the small- and medium-sized pages. Let’s take a look at this default memory scheme and how we can measure it.
Default Memory Scheme
When you install an AIX system, the small, 4K sized pages, and medium, 64K pages will be active. How these page sizes are used is determined by AIX as it doles memory out to whatever you’re running on the box, be it applications, databases, middleware or utilities. The usage of medium versus small pages is entirely transparent to running processes; AIX will assign the best memory page size allocation to a process as it runs throughout its life cycle.
Generally, this assignment is a mix of small and medium pages to every process in a system. What AIX is aiming for is maximal efficiency in process execution. So how do you tell how any given system is “mixing” the allocation of small and medium pages? That’s easy, you use vmstat and svmon. Vmstat will give you the 10,000-foot level view of how memory is being used in your system, while svmon lets you get far more detailed in your memory studies, with different flags yielding more and more information.
vmstat
Let’s start our memory study with vmstat. Most of you run constant vmstat statistics on your systems to track memory and CPU usage over time. You can add statistics for different memory page sizes by adding the “-p” flag, like this:
vmstat -w -t -p ALL 2
This invocation says to start vmstat with a wide display so all of the data isn’t scrunched together, timestamp each line and to include statistics for every size of memory page the system currently has configured. The 2 says to output this data every two seconds. Most of you will see output with 4K and 64K under the pgsz column, like the display in the image below . It’s only if you’ve specifically configured the large (16MB) and supreme (16GB) pages that you will see entries for them.