file system check software

[Stephen Adler]

I put together a software raid 5 system with 7 enterprise level segate 1 
terabyte sata disks giving me about a 6 terabyte file system. I created 
an ext3 journalled file system on top of that. About 3 months into using 
the file system, I discovered that doing a copy of a large file on that 
file system created a corrupted copied file. (md5sums of orignial and 
copied file did not match.)

Since then I've done a several tests which indicate that the problem 
lies in the support hardware around the disk system. (i.e. the network 
card is not the problem, or the memory on the system etc.) If I do 
multiple md5sum's on the same file they are always the same.

So basically the test that I need to do is fill up the 6 terabytes of 
the file system with files, large and small and do md5sum checks of the 
files as they are created and copied about on the file system. And do 
this file creation, replaction and deletion over and over again, 
stressing not just the software, (kernel, drivers, etc.) but also the 
underlying hardware, (disks, controllers, cables etc.) Such a tool would 
pretty much validate a disk system so that a bank would trust it to hold 
its database of customer accounts.

Cheers. Steve.

On 03/12/2010 07:44 PM, Edward Ned Harvey wrote:
>> As I move through this corrupted file system problem, I'm going to have
>> to wipe out my 7 terabyte file system and rebuild it. But not that I
>> live in a world of data paranoia, I'd like to run some kind of file
>> system checking software. Does something like that exist? Basically I
>> need to write large (multi gigabyte files) filled with random numbers
>> and copy them around and do diffs and md5sum checks. This should go on
>> for about a week of continuous write, read and checking.
>>      
> Could you be more specific?
>
> Filesystem checking is automatic in ext3 and newer (or any journaling
> filesystem.)  And there's fsck for anything older.  But I don't think that's
> what you're talking about is it?
>
> Maybe you're talking about ... you want to identify when some data error is
> taking place on disk?
>
> Are you required to stay with Linux?  If you could use ZFS, it does all that
> in the background, checksumming every time there's a filesystem read or
> write.
>
> Or maybe you're looking for a way to surface-scan the disks that you
> presently have, to find a failed disk?
>
> I'm not clear on exactly what you're asking.
>
>