oom-killer

[gaf-mNDKBlG2WHs at public.gmane.org (Jerry Feldman)]

One of my coworkers is having some trouble, but I'm not sure what kernel
he is using. I assume it is RHEL 5.2
I am not sure whether his system is 32-bit, 32-bit with PAE, or 64-bit.

Wile I have a few more details, what he is doing is:
1. running a process that allocates a lot of memory. Again, not sure if
it uses malloc/new, mmap, or just defines a large array in bss.
2. Runs CPLEX solver, and what he sees is that CPLEX either runs very
slow or waits for resources (he's probably doing a top or ps).

I did explain how memory is set up, mapped, and over-committed in Linux
(eg. text section, data section, and bss). I would suspect that he did
not take into account that the text section is mapped to the executable
and does not take up any swap, and that initialized data pages are
loaded copy-on-write so that if a write is not posted to that page, and
space is needed, it too does not go to swap. So, My assumption is that
he hasn't hit the threshold where oom-killer would kill a process
(probably CPLEX). So, I'm looking for some general insights on how
oom-killer works in general. I also know that there are about a half
dozen kernel variables that can be set to change the behavior.
Additionally, I don't know what patches Red Hat applied out of the box
since I don't know which system he is running. It is most likely RHEL
5.2 or RHEL 4.0 with no updates. Without knowing what system,
On my Fedora 11 system 2.6.29.5-191.fc11.x86_64 I have these flags
vm.oom_kill_allocating_task =3D 0
vm.oom_dump_tasks =3D 0
vm.would_have_oomkilled =3D 0

on RHEL 5.2 (64)  2.6.18-92.el5 the above flags are not present (or
visible through sysctl).


--=20
Jerry Feldman <gaf-mNDKBlG2WHs at public.gmane.org>
Boston Linux and Unix
PGP key id: 537C5846
PGP Key fingerprint: 3D1B 8377 A3C0 A5F2 ECBB  CA3B 4607 4319 537C 5846