kernel panic

[jarod-ajLrJawYSntWk0Htik3J/w at public.gmane.org (Jarod Wilson)]

On Feb 15, 2011, at 1:00 PM, Tom Metro wrote:

> Tom Metro wrote:
>> If this is a practical option, I'll dig deeper and see if I can turn up
>> a guide for using it with an Ubuntu kernel.
> 
> Installing kexec-tools did generate this warning:
> 
> update-rc.d: warning: kdump start runlevel arguments (2) do not match
> LSB Default-Start values (0 1 2 3 4 5)
> update-rc.d: warning: kdump stop runlevel arguments (none) do not match
> LSB Default-Stop values (6)
> 
> (Which I think this bug report addresses:
> https://bugs.launchpad.net/ubuntu/+source/kexec-tools/+bug/569980 )
> 
> and a kdump placeholder man page, so that's a good indication it is a
> patched kexec-tools.

Patched kexec-tools shouldn't be necessary these days, as far as I can
recall. It was when we were working on it in the pre-RHEL5.0 GA days,
in late 2006, but I'm pretty sure the necessary support has all been
in kexec-tools for some time now.


> I then ran across:
> https://wiki.ubuntu.com/Kernel/CrashdumpRecipe
> 
> which says to install the linux-crashdump package, then after a reboot,
> kernel panics should automatically be logged.

I've heard of that one, but never used it. Never got a ton of upstream
traction -- kdump is the upstream crash dump method of choice now.


>>> ...which collects an entire vmcore...that can be analyzed.
>> 
>> How does it record that?
> 
> Apparently it saves the dump to RAM, boots a special kernel that then
> provides access to that RAM and writes the drop to a safe area on disk.

Nope. You're on the right track, but not quite. You boot your primary
kernel with reserved memory region. This memory region is completely
untouched by your system during normal use (you lose use of that area).
The crash kernel is loaded into that memory area, and given rights to
use it upon a panic. The trick is that we boot into the crash kernel
without resetting or touching the rest of RAM, leaving it in exactly
the state it was in when the panic occurred. Once the crash kernel has
booted, it essentially dumps a raw copy of the memory from when the
panic happened out to your dump target device, with optional filtering
of the vmcore to reduce size (don't save unused pages, skip userspace
memory, etc).


> I'm curious to see if it can be configured to use a Flash drive. The
> turn-key Ubuntu process makes no mention of that.

Yes, it can. You can dump to essentially any storage device, if you
have the appropriate tools. In Red Hat Enterprise Linux, we provide
some additional infrastructure that allows creating an initrd with all
the bits necessary to dump over ssh to a remote system, over nfs, to
a fibre channel array, or wherever you like, without even having to
mount your root file system (which may be corrupted as a result of
the panic you're trying to capture a vmcore from). Those bits are a
bit distro-specific though, as its a modified version of Red Hat's
initrd creation utility that is at the heart of this. But its very
doable. Just not sure if Ubuntu has the hooks to do anything other
than a local fs dump.


-- 
Jarod Wilson
jarod-ajLrJawYSntWk0Htik3J/w at public.gmane.org