[Discuss] any decent NTFS implementations for Linux?

[bogstad at pobox.com (Bill Bogstad)]

On Wed, Jul 23, 2014 at 1:18 PM, John Hall <johnhall2.0 at gmail.com> wrote:
> Hi Bill,
>
> Interesting problem. I wonder if the severe write performance cut has
> something to do with using the NTFS drivers and USB.   Have you done write
> tests with internal hardware or even loop-back deceives?

The problem seems to be 100% bad filesystem software.   I reformated the
drive with ext4 for testing purposes and I get peak performance of >100MB/sec
and sustained of 30-40MB/sec.  This is the exact same hardware setup, OS, etc
The only difference is formatting with ext4 vs. NTFS filesystem.

> I found this report for tests over sata where they got  (15MB/s) - still
> terrible but much better than 1MB/s !
> http://superuser.com/questions/613869/ntfs-write-speed-really-slow-15mb-s-on-ubuntu

I've seen that page and a number of others..   I haven't yet tried the
"big_writes" option
to NTFS-3g which might help a little.   Unfortunately, there doesn't
seem to be a way to provide
that option when using the automounting feature that works when you
just plug in an USB
device.   I suppose I should try manually mounting the drive and see
if it makes a significant
difference.   Then I can spend more time determining if there is a way
to make automounting
of NTFS disks use that option.

> Can you attach your drive using e-sata?

No.   I intend this to be a portable drive used with various devices
and I can't assume they
will have anything besides USB.   In any case, I'm conviced the
problem isn't hardware.  The NTFS
filesystem for Linux is so bad that I suspect it will be the
bottleneck even when used with a USB 2.0 host.

> I wonder...  What sort of write performance folks get from NTFS drives
> attached to a gigabit router?

If they use Tuxera's proprietary NTFS for Linux, I suspect much better then I am
seeing.   Doesn't help for a portable drive which might be attached to
arbitrary machines.


> My overall work-around is to run one primary OS per machine, using others as
> virtual machines, and file sharing to share data. That way native drivers
> for the file systems are used. Treating them as separate systems that work
> together via networking avoids many problems. Of course virtualization
> requires extra RAM, and I have no benchmarks to compare performance. My
> setup using VMWare Workstation (not free) allows me to connect external
> drives to guest operating systems, but I think you can do the same thing now
> with free software. What do you think about this approach?

I've done that kind of thing myself for other reasons, but expect this
drive to be attached to arbitrary
host systems which may have only default software installs.   I was
hoping that NTFS would
be a universal filesystem.   It seems that if I care about performance
with Linux hosts, it is not.
It seems that FAT32/vfat is the closest thing to a universal
filesystem out there.   Unfortunately,
it has file/filesystem size limitations.   So I couldn't (for example)
keep backups of 8GB VM images
there.

There was a suggstion of getting an ext3/4 filesystem driver for
Windows and it seems like Paragon makes
one available for free.   This might solve my performance problem, but
would require a software
install on any Windows box that I wanted to connect the drive too.
Not really my definition of plug and play.
But I might end up going that route anyway.

Thanks,
Bill Bogstad