Boston Linux & Unix (BLU) Home | Calendar | Mail Lists | List Archives | Desktop SIG | Hardware Hacking SIG
Wiki | Flickr | PicasaWeb | Video | Maps & Directions | Installfests | Keysignings
Linux Cafe | Meeting Notes | Blog | Linux Links | Bling | About BLU

BLU Discuss list archive


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

ftp directory mirror via squid



On 7/11/05, Bob Gorman <bob at rsi.com> wrote:
> Quick question... when doing ftp via wget through a squid proxy server, how can one do a recursive directory mirror?
> 
> An example script that mirrors an ftp site:
> 
>         wget -m ftp://apt.sw.be/pub/dag/redhat/el4/en/i386/dag/
> 
> Works great.  But try to do it via squid:
> 
>         export ftp_proxy=http://proxy.my.com:3128
>         wget -m ftp://apt.sw.be/pub/dag/redhat/el4/en/i386/dag/
> 
> And one gets a nice html file instead of the directory structure.
> 
> Is there an easy way to work around this?  Or is there a better, simple, command line tool for doing an ftp mirror via squid?

AFAIK, you can't work around it. When used as an "FTP" proxy, Squid
still speaks HTTP to your client (which must support HTTP - pure FTP
clients can't use Squid as a proxy). As such, there is no way for a
client to get a directory listing, just whatever index page Squid
generates.

You might be able to work around this by automatically removing the
index files after wget runs. I'm guessing they all have the same name,
so it should be simple to do with 'find'.
-- 
Andrew Medico <a.medico at gmail.com>




BLU is a member of BostonUserGroups
BLU is a member of BostonUserGroups
We also thank MIT for the use of their facilities.

Valid HTML 4.01! Valid CSS!



Boston Linux & Unix / webmaster@blu.org