[Discuss] du question

[jabr at blu.org (John Abreau)]

I encountered a different situation that I feel illustrates du's behavior
even more clearly. When transcoding a video with ffmpeg and monitoring
progress in a separate xterm, I noticed that du reported the size as a
series of doublings. It started out reporting that the output file was 4M,
it stayed at 4M for a while, then jumped to 8M, then 16M, 32M, 64M, 128M,
and 256M, and when the transcoding completed. it dropped to 212M. It seems
that ffmpeg preallocates disk blocks for the output file, presumably to
maximize the efficiency of processing streaming media, and it does so by
doubling the number of preallocated blocks whenever it's close to running
out.

du reports on the number of disk blocks allocated, not the file size.

Another example of this would be a sparse file. You could have, for
example, a database file that ls -l reports as 100 GB but where most of the
zero blocks are not allocated. While the file size is 100 GB, as ls -l
reports, it may happen that only 512M of disk blocks have been allocated,
in which case du would show 512M.



On Fri, Jun 9, 2017 at 3:52 AM, John Abreau <abreauj at gmail.com> wrote:

> du measures disk usage, not individual file size. Two hard links to the
> same underlying file in two different directories don't use twice the disk
> space of one instance.
>
> If you run two separate instances of du on each of the directories
> containing the hard links to that file, each would report 35G. If you run a
> single instance of du to measure both, it would report an aggregate total
> of 35G; the first one displayed would show as 35G of disk space used and
> the second as zero additional disk space used.
>
>
> On Thu, Jun 8, 2017 at 5:35 PM, dan moylan <jdm at moylan.us> wrote:
>
> >
> > dan ritter writes:
> > > On Thu, Jun 08, 2017 at 02:11:22PM -0400, dan moylan wrote:
> >
> > > > dan ritter writes:
> >
> > > > > > I don't know backintime, but I'm guessing it uses links to
> > > > > > do file-level snapshots.   Bet there's a FAQ.
> >
> > > > > > 35G     20170607-120002-419
> > > > > > 86M     20170607-230005-357
> >
> > > almost as if there were 86MB of changes in between the
> > > first and second snapshots.
> >
> > of course, that's the whole point of backintime.  it
> > was/is/has been obvious, even to me, but it is not the
> > question and never has been.
> >
> > my question was and is: why does
> >   du -s *
> > show 86MG for the second and now third backups, and
> >   du -s 20170607-230005-357
> > as well as
> >   du -hd1 20170607-230005-357
> > show 35MB?
> >
> > this is not obvious to me -- if it is to you, please
> > explain.
> >
> > tia,
> > ole dan
> >
> > j. daniel moylan
> > 84 harvard ave
> > brookline, ma 02446-6202
> > 617-777-0207 (cel)
> > jdm at moylan.us
> > www.moylan.us
> > [no html pls]
> > _______________________________________________
> > Discuss mailing list
> > Discuss at blu.org
> > http://lists.blu.org/mailman/listinfo/discuss
> >
>
>
>
> --
> John Abreau / Executive Director, Boston Linux & Unix
> Email: abreauj at gmail.com / WWW http://www.abreau.net / PGP-Key-ID
> 0x920063C6
> PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
> _______________________________________________
> Discuss mailing list
> Discuss at blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email jabr at blu.org / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6