Technical Linux questions

[mcmanus at ducksong.com (Patrick R. McManus)]

[Jonathan Arnold: Feb 21 10:21]
> 
> 1] What is the default and theoretical maximum number of processes
> that you can run at once?

default is limited to 32K by the pid space.. but the pid allocator
breaks down to be useless well before then. using a bigger pid space,
a new pid allocator, and the new 2.5 threading code people have run
100,000 simultaneous threads which is probably a reasonable
approximation.

I'm sure there are shell and libc limits to be circumvented as well.

http://marc.theaimsgroup.com/?l=linux-kernel&m=103230439008204&w=2


> 
> 2] What is the theoretical maximum number of sockets that a single
> process can listen and respond on?

no real theoretical limit.

/proc/sys/fs/file-max contains the number of descriptors available
system wide.. the default per process is substantially lower (8K I
think).. see setrlimit() to up that in your program.

> The two, as you might imagine, are connected. I need to respond to
> possible input from up to a thousand different ports, and spawning
> a process to listen on each isn't working - probably too
> many processes.

just use poll() in one process. 1000 is a reasonable number for
poll().. if you start getting over the 5K range lookup the epoll()
patches by david libenzi. if handling the poll events takes too long
(from a latency perspective) just assign the work to a pool of
prespawned threads and get back to calling poll().. that's the usual
model anyhow.

there are many servers that regularly jostle around 25,000 sockets at
a time with just a handful of processes.