Comments (19)
Polipo already detects interrupted downloads, by either using the Length
header or detecting unterminated chunked encoding.
from polipo.
do not use cache in polipo! For your case make sure to use HTTP CONNECT method and the last browser version. If you still fail, in Google Chrome press F12, select and activate Network tab and take a look at server's headers. If it doesn't support resuming - no client-side software will help you
from polipo.
@netsafe I'm afraid there is some misunderstood.
- polipo is web cache server , if don't using its cache, I am afraid it is useless.
- I've already made it as transparent proxy with cache by reform the request.
- I don't know what you have said, for current, the case is when Internet suddenly cut down, the downloading operation may timeout on server socket, but on client side it still connect, this is the abnormal case, may not be taken into consideration during the polipo development.
could any one give some suggestion about how to implement such abnormal optimize it
from polipo.
@qianguozheng Am I got you right, that you want help with pt.3 in your list? You want to fix this abnormal behaviour?
from polipo.
@netsafe You got it. Actually I want to author know that polipo may not take such abnormal case into consideration, we need to implement an mechanism to make it work right
from polipo.
it's not a polipo issue : you need to set proper tcp timeouts+maxconns+openfiles in all the config files and sysctl's, also do not forget to add 80,8080,443 and 8443 to long-living ports in Tor config - it will save your day. After tweaking all sysctl's in system-wide sysctl.conf make sure to perform a full reboot. Feel free to ask if you have any further questions!
from polipo.
@netsafe WHY it is not polipo's issue ? shouldn't it check whether the response is a finished or not ?
I think that it should set a flag to check whether the exact response it finished or not, else could cause the issue i mention above.
from polipo.
It can't - under Linux at least - if the session timeout is TOO LONG. Setting explicit timeouts in app code is a very rare condition, unless you're in a great need in ESPECIALLY THIS kind of socket-tuning. It must be tuned on the host itself, host-wide, because it affects not just one program( polipo for example ) but the whole pack of processes running on the box. And to tweak such a parameters is an ABSOLUTELY snadart & basic SA every-day task
from polipo.
I've actually had the same issue; while downloading large files, if the Internet connection becomes slow and/or unusable and the download is stopped it will fail to download the rest of the file properly on the second try (after connectivity is improved). It seems like Polipo downloads part of the file to its cache, and then on the second request from the client the partial file is served (but not the rest of it). The client just hangs, waiting around for Polipo to serve up the rest of the content.
from polipo.
@brodyhoskins Have you tried to disable the cache in Polipo config and in HTTP request headers(the requests made to download the file)?
from polipo.
No I haven't disabled the cache, because I'm using Polipo to do caching.
from polipo.
@brodyhoskins try to disable cache in polipo and use a caching server like Squid, NGinx or Apache if the polipo caching is the problem here. Theese servers won't eat much resources, but their caching mechanisms are a way higher than polipo's one, IMHO
from polipo.
I’ll look into those, especially if they compile on (jailbroken) iOS. But I’be grown found of Polipo. ^^
On Sep 4, 2015, at 6:03 AM, Alexey Vesnin [email protected] wrote:
@brodyhoskins try to disable cache in polipo and use a caching server like Squid, NGinx or Apache if the polipo caching is the problem here. Theese servers won't eat much resources, but their caching mechanisms are a way higher than polipo's one, IMHO
―
Reply to this email directly or view it on GitHub.
from polipo.
The main advantage of polipo caching, in theory at least, is it can cache
partial downloads. None of those others you mention can do partial download
caching. In fact, polipo is the only http cache I know of that claims to
support caching of partial downloads*.
IMHO if polipo's caching doesn't work correctly, then there's not much
point in using polipo at all.
I would argue that this is indeed a bug in polipo. If a fetch failed
because the upstream connection was interrupted, polipo shouldn't behave
like the fragment it has cached is the whole object. When the fetch fails
it should either invalidate that cache entry, or (even better) cache it as
a partial fetch.
- Squid can kind of cache partial downloads if you tune the settings, but
it does it by fetching and caching the whole object even in response to a
small ranged fetch request.
On 4 September 2015 at 23:03, Alexey Vesnin [email protected]
wrote:
@brodyhoskins https://github.com/brodyhoskins try to disable cache in
polipo and use a caching server like Squid, NGinx or Apache if the polipo
caching is the problem here. Theese servers won't eat much resources, but
their caching mechanisms are a way higher than polipo's one, IMHO—
Reply to this email directly or view it on GitHub
#65 (comment).
Donovan Baarda [email protected]
from polipo.
@brodyhoskins Apache and NGinx are compiling, I did it a long time ago. Can't tell the same about the Squid, but I see no problems
from polipo.
@dbaarda the main feature of polipo is a VERY good and handy HTTP/HTTPS-CONNECT -> SOCKS connectivity. It helps alot
from polipo.
True... it is a pretty good HTTP/HTTPS -> SOCKS proxy, and also HTTP 1.0 ->
HTTP 1.1 proxy.
But I don't use it for that, and in theory it should also be a very good
caching proxy, so it's sad minor bugs like this undermine it for that
purpose.
On 5 September 2015 at 20:27, Alexey Vesnin [email protected]
wrote:
@dbaarda https://github.com/dbaarda the main feature of polipo is a
VERY good and handy HTTP/HTTPS-CONNECT -> SOCKS connectivity. It helps alot—
Reply to this email directly or view it on GitHub
#65 (comment).
Donovan Baarda [email protected]
from polipo.
This looks like a bug in Polipo, no argument about that.
from polipo.
@jech , yeah, I think it it a bug in Polipo.
And for solution to this problem, should we discuss about and state for file downloaded ? if we do so, how should we judge a file is downloaded success or fail ? maybe it fail due to the network shutdown ?
any thought about that ? thanks.
from polipo.
Related Issues (20)
- chunkHighMark ignored when it is a big number HOT 3
- polipo ignores specified port number HOT 1
- Error using curl --trace-ascii dump.txt www.google.com .
- Whether the file that holds the pid can be made compatible, for example, if it exists, it will not be created. HOT 2
- Https proxy not working HOT 1
- SOCKS protocol error HOT 4
- Can we set no_proxy url, just like a black list HOT 1
- Couldn't establish listening socket: Unknown error HOT 1
- HTTP violations
- failed start stop polipo web cache HOT 1
- Nothings persisted in cache folder HOT 1
- Please mark project as 'read-only'. HOT 2
- hostname error with format ::ffff:111.30.131.122
- Path had bad ownership/permissions HOT 2
- Dropping "CONNECT" request instead of 401 when auth is required
- Getting more done in GitHub with ZenHub
- How to change default Polipo listening TCP port 8123 from the config file? HOT 1
- Not proxy for given web sites
- Doubts in httpServerHandlerHeaders API HOT 1
- HOSTS file support. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polipo.