Boston Linux & UNIX was originally founded in 1994 as part of The Boston Computer Society. We meet on the third Wednesday of each month at the Massachusetts Institute of Technology, in Building E51.

BLU Discuss list archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Discuss] GNU xargs trick

There is a trick to GNU xargs that lets you easily parallelize
processes. The particular use I discovered is for optimizing directories
full of PNG files using optipng. I first looked at GNU parallel but it's
way overkill for what I want. I found a couple of scripts that do some
tricks with & and lock files but those ran into problems with
complex-ish commands. Then I learned about the xargs trick:

find . -name "*.png" -print | xargs -I{} -P 4 optipng -o1 -preserve {}

The find part should be obvious, xargs maybe not so much. -I enables
substitution. Whenever the string following -I appears it is replaced
with the the line from stdin. -P 4 says to run a maximum of 4 processes;
the default is 1. Everything after is the optipng command with {}
replaced with each line from the find command.

I dunnow if this has ever come up on the list but it's been quiet lately
and I figured it's a cool enough trick to share.

Rich P.

BLU is a member of BostonUserGroups
BLU is a member of BostonUserGroups
We also thank MIT for the use of their facilities.

Valid HTML 4.01! Valid CSS!

Boston Linux & Unix /