BLU Discuss list archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Discuss] Web proxy
- Subject: [Discuss] Web proxy
- From: worley at alum.mit.edu (Dale R. Worley)
- Date: Mon, 28 Dec 2020 23:03:49 -0500
I would like to implement what I believe is a simple web proxy, but it requires a certain amount of customization regarding how the proxy contacts the servers, which I think would be very simple if I used the right proxy. Unfortunately, my knowledge of the features of open-source proxies is very small and searching online is not producing useful results, probably because I don't know the right words for describing the features I want in a proxy. The problem is to sort the HTTP/HTTPS requests based on the host-part of their request-URLs. An ssh session needs to be made to a different host name (derived from the host-part), and that session used to forward the request to the original host name. A simple implementation would be to start a process ssh -N -L [local port]:host-name:443 derived-host-name and then forward the request via localhost:[local port]. However, it seems to me that one can avoid having to carefully track and tear down the ssh processes by instead configuring the proxy to, effectively, "forward requests to host-name by creating a subprocess ssh derived-host-name ncat host-name 443 and using that process's stdin and stdout as the socket to send the request to." The advantage of the latter method being that it automatically tears down the ssh if the main process terminates. Then again, maybe I'm lucky and some proxy implements this behavior directly. Does anyone have any hints? Thanks! Dale
- Prev by Date: [Discuss] Consultant recommendation for web scraping / scripting?
- Previous by thread: [Discuss] Consultant recommendation for web scraping / scripting?
- Index(es):