Friday, October 2, 2009

Oracle & Linux Hugepages settings

Question:

Is hugepages still recommended for 64bit Linux + Oracle?

Answer:

Yes!

Why:

Without hugepages, each page is 4kB by default, to maintain a pagetable of 32GB memory addressing, we need 32*1024*1024/4*8/1024/1024=64MB (each page reference point takes 4 bytes in 32bit OS and 8bytes in 64bit OS). For example, we have 100 such processes; we need 6400MB memory to maintain the pagetable only. (But normally we will not hit this worst situation, because of lazy allocation memory algorism)

With hugepages, each page is 2MB on EM64 architecture. To maintain a pagetable of 32GB memory addressing, we only need 32*1024/2*8/1024=128KB. For example, we have 100 such processes, we only need 12.5MB memory for pagetable.

Another reason to configure hugepages is memory associated hugepages are not swappable, a.k.a, always in memory.

One more benefit, hugepages increases the efficiency of TBL.

Another one, "Cached" memory with hugepages does not include SGA any more.

How to set?

[root@VMXDB01 ~]# cat /etc/sysctl.conf
... ...
vm.nr_hugepages = 16640
... ...

Method of calculating vm.nr_hugepages:

Get Hugepagesize from /proc/meminfo, for our case, it's 2MB.

vm.nr_hugepages = sum (all sga_max_target size in MB) / Hugepagesize + min(2,value_you_feel_conformtable)


[root@VMXDB01 ~]# cat /etc/security/limits.conf
... ...
oracle soft memlock 33564672 <-- larger than max SGA, in KB
oracle hard memlock 33564672 <-- larger than max SGA, in KB
... ...

Verify:

[root@VMXDB01 ~]# cat /proc/meminfo
MemTotal: 64951768 kB
MemFree: 29433020 kB
Buffers: 62304 kB
Cached: 701376 kB
SwapCached: 0 kB
Active: 726660 kB
Inactive: 575368 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 64951768 kB
LowFree: 29433020 kB
SwapTotal: 33551744 kB
SwapFree: 33551744 kB
Dirty: 376 kB
Writeback: 0 kB
AnonPages: 549344 kB
Mapped: 114368 kB
Slab: 63620 kB
PageTables: 23268 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 48988268 kB
Committed_AS: 2529688 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 283056 kB
VmallocChunk: 34359455303 kB
HugePages_Total: 16640
HugePages_Free: 13887
HugePages_Rsvd: 13695
Hugepagesize: 2048 kB

1 comment:

  1. Hi LUO,

    I am having some problems with setting hugepages. Hopes you wil help.

    I am using oracle 10g 64bit with ASM on rhel564bit.
    I am facing the performance issue.
    As a performance tuning part i am trying to configure HugePages.
    I am having 24GB Physical memory.
    SGA size is 6GB
    I configured the vm.nr_hugepages=3174 (6.2GB)
    I Configured the memory lock value as 6501171KB
    I rebooted the system and started the DB instance.
    But i can see still HugePages_Free=3174
    That means oracle is not using the HugePages.
    Did i missed any thing? Please suggest if i need to do anything more. Or please suggesst the correct value for my environment.

    Thanks in Advance,
    Arumon

    ReplyDelete