To swap or not to swap that is the question

[grg22 at ai.mit.edu (Greg Galperin)]

On Mon, Oct 07, 2002 at 06:33:39PM -0400, Kent Borg wrote:
> I can imagine that a fast, 4-CPU machine, with wicked fast disks might
> have use for going a higher, but not as high as, say, 1 GB.  Anyone
> have a concrete scenario where more swap is useful?

I run multiple jobs on multi-gigabyte data sets.  I use over a
gigabyte of swap in two ways on this load:
 1. The working set of each job is about half its total memory consumption,
    so I can run twice as many jobs by letting it swap out the stuff not
    in each job's current working set
 2. I run some interactive environments that I want to be long-lived
    while I run several shorter jobs (same memory consumption though), 
    so I'm happy for my interactive jobs to get swapped to disk while
    the shorter ones run, after which I can return to the original jobs.

I've filled up 4GB of swap multiple times doing the above.  Owing to
the tripped circuit breaker over the weekend I'm not yet back to the
point of stressing the machines (maybe later tonight), but here's the
current "normal" state for one:

             total       used       free     shared    buffers     cached
Mem:       3992852    3987344       5508          0      30008    2094152
-/+ buffers/cache:    1863184    2129668
Swap:      2096472    1234100     862372

(No, the 2G free on the +buffer/cache line really isn't free, since it
includes some large memory-mapped files which are in the working set.
Their pages are clean, but I need them in ram.  There's probably only
a couple hundred meg really free, which is why 1.2GB are swapped.)

I don't like to use more than a couple gig of swap, because it takes a
long time for it to get those pages to disk and back.  It's better
than having jobs killed, though.  But, to answer earlier questions, I
do find it useful to hang out around using 2GB swap, and the machine
works fine doing that.



> I guess it boils down to this: bandwidth to disk is a bottleneck,
> until disks (and controllers and bus bandwidth) are only so fast and
> until they get faster, 256 MB is a lot of swap.  More won't hurt, and
> with modern big disks is no burden so why not, but I see no benefit.

Actually, it can hurt -- the 2.4 kernel sometimes tends to swap things
out just 'cause it feels like it.  Without any swap space, it would
leave the pages in memory, and be much faster to access when you want
them.  I don't think it's a good idea to run without swap space, but
it's not strictly a one-sided issue.

BTW, IMHO the best reason to run with swap is because nearly everyone
else does, so the software gets the most testing in that situation.
Case in point: the 2.4 kernels (starting earlier than you'd want to
use them) have an OOM bug which is triggered if you're running without
swap.  The kernel kills off your processes, claiming it's out of
memory, even when most of memory is only being held by the kernel as a
disk buffer and should be available.  If you have even a small amount
of swap, this bug is hidden.  [Wasn't fixed by 2.4.18, I haven't
looked since then.]


--grg