memory limits....

[adler at stephenadler.com (Stephen Adler)]

Thanks Jerry,

The kernel does allocate more than 2 gigs of memory since I currently
have 4 gigs of physical memory in my server, and the kernel reports
that it "sees" 4 gigs of physical memory. I'm wondering if there is
a difference between memory allocation in user space and memory
allocation in kernel space. If I remember right, the CPU runs in
two different modes, a user mode, and a supervisor mode or something
like that, and the difference is how the CPU addresses memory.
(These are notions I have buried deep in my memory... and could
be false.) So perhaps the user space memory limit is different from
the kernel space memory limit?

Cheers. Steve.

On Thu, 2005-05-19 at 11:09, Jerry Feldman wrote:
> On Thursday 19 May 2005 10:28 am, Stephen Adler wrote:
> > Thanks for the reply. I'm using malloc, under gcc 3.2, red hat,
> > enterprise linux 3, kernel 2.4. After allocating about 2.4 Gigs,
> > subsequent malloc's return NULL. I check for NULL returns from
> > my malloc's and exit my code if this condition is met. The OS
> > seems to be happy, nothing grinds to a halt. My applications
> > allocates all the needed memory at startup, so if not enough
> > is allocated, it informs the user and exits.
> Classicly, the way malloc works requires some explanation:
> First your program is given a virtual memory map. This includes where the 
> program text (eg code) is mapped. This is usually readonly, the program 
> stack, the program's initialized data, and unititalized data. Then you get 
> what is called a heap. In classic Unix and Linux, the heap grows upward 
> using the brk(2) or sbrk(2) system calls. These grow the heap contiguously. 
> At some point they may his a barrier. This could be the top of your virtual 
> address space, or it could be that some other operation, such as mmap(2) or 
> the loader has mapped something else up there. Malloc(3) periodically grabs 
> a chunk of memory from the heap, normally in multiples of page sizes. More 
> modern mallocs may take a different approach by using mmap(2) to allocate 
> its chunks. The advantage of using mmap(2) is that it can allocate 
> non-contiguous chunks. Malloc(3) is simply a set of library functions and 
> has nothing to do with the compiler, BUT the compiler is responsible for 
> setting up the initial virtual mapping in your program, and the loader is 
> responsible for the dynamically loading of your program, and mapping in 
> shared libraries, et. al. Also, remember that a 32 bit signed integer is 
> limited to about 2GB. I think that you may be running into a 32-bit 
> instrinsic limit. This could be imposed by malloc(3) or by the kernel where 
> you have a 2GB limit on heap size.