Comments (8)
Hi,
Thank you for using goproxy.
I don't think you're reading it correctly. What's happening is, the goproxy hook returns an *http.Response
to the proxy, which will read it and write it to the client. We replaced the response body with a reader that would write the response to a file while reading it. Thus when the proxy will call resp.Body.Read(...)
, it'll write the content to a file in the mean time.
The proxy is also responsible for closing the response after writing it to the client. You only have to worry about providing the response. Let me illustrate that with an example:
C> Hey proxy, give me http://example.com/foo
P> Just a sec client, let me see if I have hooks for that. Actually I do.
Hey hook, filter the response I got from example.com.
H> Here you go, I cooked a new response with a different body.
P> OK client, I'm reading from the cooked response and writing to you
In the meantime COOKED RESPOSNE is secretely writing down everything in the response.
Let me know if you need any other clarification.
from goproxy.
On Sat, Nov 16, 2013 at 10:03 AM, Elazar Leibovich <[email protected]
wrote:
Hi,
Thank you for using goproxy.
I don't think you're reading it correctly. What's happening is, the
goproxy hook returns an *http.Response to the proxy, which will read it
and write it to the client. We replaced the response body with a reader
that would write the response to a file while reading it. Thus when the
proxy will call resp.Body.Read(...), it'll write the content to a file in
the mean time.
Right and that file capture is done with the TeeReederCloser. I'd like to
fire off a background job once the TeeReederCloser is closed. Proxy only
calls resp.Body.Read(...) as the original client consumes the stream and
then I assume golang's builtin http tools call the Close method.
I've modified the TeeReederCloser.Close() to send on a channel I've
created. Is there a more elegant way to be notified when Close is called,
some way that leverages goproxy's framework or golang's built-ins?
The proxy is also responsible for closing the response after writing it to
the client. You only have to worry about providing the response. Let me
illustrate that with an example:C> Hey proxy, give me http://example.com/foo
P> Just a sec client, let me see if I have hooks for that. Actually I do.
Hey hook, filter the response I got from example.com.
H> Here you go, I cooked a new response with a different body.
P> OK client, I'm reading from the cooked response and writing to you
In the meantime COOKED RESPOSNE is secretely writing down everything in the response.Let me know if you need any other clarification.
—
Reply to this email directly or view it on GitHubhttps://github.com//issues/25#issuecomment-28632022
.
from goproxy.
I'm not sure what exactly you want to achieve, but I don't think you have any other option. Note that TeeReaderCloser.Close
might never be called because of a previous Read
error, so you must wait until it's actually called.
That said, make sure the channel is buffered, you don't want to keep your goroutine waiting until the background job fires.
Do you have a better design in mind? If there's a way goproxy
can help you with that, I'd like to hear that.
from goproxy.
I have a pool of workers that need to pull content from external sources ie. the wild internet. The workers do not have access to the internet and the workers will often pass over content more than once.
I need a proxy that serves as a forward proxy to provide internet access to the workers. The same proxy can serve to intercept the content and forward to a storage layer which can be used for future fetches. I'm effectively building a large read-through cache, but one that's fully deconstructed.
I've been getting by w/ squid and nginx, but I want more programmatic control of the proxy behavior.
I don't think there's any better design, my questions so far are about my specific implementation. I'm new to golang; goproxy
has been a great reference for becoming familiar w/ interfaces and golang's concurrency patterns.
Thanks for the note about buffered channels. I keep forgetting that the writer blocks until the reader reads. I guess it's like a socket accept with no listen queue(sort of.)
from goproxy.
BTW, I assume there exists a "go func()" somewhere to provide the concurrency. Is there a goroutine per handler?
from goproxy.
@davidbirdsong I have almost no go func()
s in goproxy (except https), the concurrency is handled by the http
package where the goroutine resides.
Is it possible for you to show me some of your code? Either privately or in the public?
PS,
If you could write me how well goproxy
worked, I'll be delighted.
from goproxy.
Is everything clear? Can I close the issue?
from goproxy.
Hey sorry, totally missed these notifications. Everything's much clearer now. I can share some of the code soon.
from goproxy.
Related Issues (20)
- add method for ProxyHttpServer object
- How to prevent traffic from being swiped? HOT 1
- Facebook and Instagram apps fail to work through GoProxy HOT 1
- Deployment to AWS Lambda ?
- Release versions for go package HOT 2
- How to use code on win10 instead of manual to open the forward proxy Settings?
- An existing connection was forcibly closed by the remote host.
- How best to modify HTTP data and header order
- https blocking does not take effect HOT 3
- goproxy v1.1 was discovered to contain an issue which can lead to Denial of Service (DoS) via unspecified vectors HOT 2
- Cannot connect to HTTP websocket when performing TLS MitM
- 请求头参数大小写格式在哪转换的?
- Semicolon in URL query HOT 1
- Potential Typo in Request Handling - "r.Close" misset to False?
- Question about modifying requests
- Can we do URL A -> proxy A, URL B -> proxy B?
- Limit the number of goroutines HOT 1
- Contant HOT 1
- team.works, with community support
- updating expired certificate
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goproxy.