jpmckinney/image-proxy

An image proxy using the Express Node.js framework

image-proxy's Issues

adding an extension to the URL should return an image converted to that format

This is a small feature request: most of the large images of candidates on YourNextMP's images are stored as PNG files, but it would nice to be able to get a JPEG thumbnail by adding .jpg to the URL requested. So at the moment, this thumbnail returns a PNG:

• https://yournextmp.popit.mysociety.org/image-proxy/http%3A%2F%2Fyournextmp.popit.mysociety.org%2Fpersons%2F4493%2Fimage%2F5481e8f6b150e238702c061b/0/64

... but it would be great if you could get a JPEG by requesting:

• https://yournextmp.popit.mysociety.org/image-proxy/http%3A%2F%2Fyournextmp.popit.mysociety.org%2Fpersons%2F4493%2Fimage%2F5481e8f6b150e238702c061b/0/64.jpg

... or something similar.

Some images suffer gamma distortion on resize

In a few rare cases, images served through the proxy can look much too dark after they've been resized. It appears that it might be fixable by passing colorspace('RGB') to imageMagick as per http://www.imagemagick.org/Usage/resize/#resize_colorspace (although I'm not sure if that could be considered harmful if left on all the time).

As an example...

Before:

After:

More info here: mysociety/yournextrepresentative#288

Remote images which return 403 when no user-agent is set

I've run into an issue with an image which only returns 200 when a user-agent is set.

http://www.riigikogu.ee/?op=emsplain&page=pilt&pid=fe748f4d-3f50-4af8-8069-92a460978d2b

$ curl -sv -H 'User-Agent:' -o /dev/null 'http://www.riigikogu.ee/?op=emsplain&page=pilt&pid=fe748f4d-3f50-4af8-8069-92a460978d2b'
* Hostname was NOT found in DNS cache
*   Trying 195.80.106.68...
* Connected to www.riigikogu.ee (195.80.106.68) port 80 (#0)
> GET /?op=emsplain&page=pilt&pid=fe748f4d-3f50-4af8-8069-92a460978d2b HTTP/1.1
> Host: www.riigikogu.ee
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Date: Wed, 11 Mar 2015 12:46:19 GMT
* Server Apache is not blacklisted
< Server: Apache
< Vary: Accept-Encoding
< Content-Length: 45
< Content-Type: text/html; charset=iso-8859-1
<
{ [data not shown]
* Connection #0 to host www.riigikogu.ee left intact

So it seems that image-proxy needs to set the User-Agent header in order for this request to work correctly.

allow requests from authorized hosts only

validate the url query string parameter

Error: write EPIPE

Hi!

Thanks for the package. I keep receiving the following error when I try to use it per the instructions.

Any idea what this might be?

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: write EPIPE
    at errnoException (net.js:901:11)
    at Object.afterWrite (net.js:718:19)

If image responds after timeout then the node process crashes

The server falls over with a stack trace:

http.js:689
    throw new Error('Can\'t set headers after they are sent.');
    ^
Error: Can't set headers after they are sent.
    at ServerResponse.OutgoingMessage.setHeader (http.js:689:11)
    at ServerResponse.res.setHeader (/app-path/node_modules/express/node_modules/connect/lib/patch.js:133:22)
    at ServerResponse.setWriteHeadHeaders (/app-path/node_modules/express/node_modules/connect/node_modules/on-headers/index.js:80:19)
    at ServerResponse.writeHead (/app-path/node_modules/express/node_modules/connect/node_modules/on-headers/index.js:39:36)
    at gm.<anonymous> (/app-path/node_modules/image-proxy/lib/image-proxy.js:69:19)
    at cb (/app-path/node_modules/image-proxy/node_modules/gm/lib/command.js:265:16)
    at gm._spawn (/app-path/node_modules/image-proxy/node_modules/gm/lib/command.js:253:7)
    at /app-path/node_modules/image-proxy/node_modules/gm/lib/command.js:117:19
    at series (/app-path/node_modules/image-proxy/node_modules/gm/node_modules/array-series/index.js:11:36)
    at gm._preprocess (/app-path/node_modules/image-proxy/node_modules/gm/lib/command.js:154:5)

And pointing the proxy at the image served by this example node app should replicate the error:

https://gist.github.com/chrismytton/5f0ac2fd54e0d3872f0a

look into node-imageable

Not sure why I didn't see/use this earlier: https://github.com/sdepold/node-imageable

limit the domains that the proxy will proxy to

Use a regular expression e.g. /\.gov$/

MIME method updates

I know this hasn't been updated since Nov 2016, but I needed to use it to test something, and it initially did not work because the latest mime methods have been renamed.

I forked this and modified the methods to align:

• mime.lookup() -> mime.getType()
• mime.extension() -> mime.getExtension()

If you want a PR for this let me know, otherwise I will close this.

Image is not transcoded when extension is specified

When you add an extension to the call ( for example http://localhost:5000/http%3A%2F%2Fwww.bmcc.nsw.gov.au%2Ffiles%2FABennett_photo.gif/100/100.jpg ) it return the image with the content type "image/jpeg" as expected. But when you then download the image and inspect it, the format is "gif" (in fact on my mac, the extension is also .gif).

Also, you would expect a jpeg to be a smaller file size than the gif. But when I look at the size of the asset in the network pain of the chrome developer tools, it's the exact same size as when I don't specify an extension 10.3kb . Looks like its not actually transcoding the file, just changing the header.

From the README:

If an optional :extension parameter is provided, the image will be transcoded to the corresponding file format.

I would expect this to reformat the file, not just change the Content-type on the request headers.

PR to follow 🎷

Won't proxy Amazon S3 URLs

Great project - much needed.

But try proxying this url: s3-us-west-2.amazonaws.com/idelog/token_page_images/8cbf37c0-63e5-11e4-a43e-978e01bbbad9_page.png/90/90

Cannot GET /s3-us-west-2.amazonaws.com/idelog/token_page_images/8cbf37c0-63e5-11e4-a43e-978e01bbbad9_page.png/90/90

There is something about Amazon hosts that is screwing things up. Maybe you are use url.normalize? I have found that corrupts Amazon S3 URLs.

Non standalone version starts returning 504 error and requires restart

See mysociety/popit#719 for some details on the original site.

This doesn't happen immediately but after a while the proxy starts returning 504 errors and then continues to do so until the node process is restarted.

This only seems to happen for images that are hosted on the same app as the proxy is included in so I'm assuming it's something in the interaction that's causing this. The images are served fine when not called via the proxy.

I'm mostly raising this in case someone else has come across similar behaviour.

“Deploy to Heroku” not working

When I hit the Deploy to Heroku button, it says:

There was an issue building your app. This can mean your app.json's project is not a valid Heroku application. Please ensure your app is deployable to Heroku and try again.

Here's the build log:

-----> Fetching set buildpack https://github.com/ddollar/heroku-buildpack-multi... done
-----> Multipack app detected
WARNING: This buildpack is no longer maintained.
Please choose a different buildpack or go to https://github.com/ddollar/heroku-buildpack-multi
and fork it to your own account.
This buildpack will cease to function at the stroke of midnight on January 1, 2017.
=====> Downloading Buildpack: https://github.com/ello/heroku-buildpack-imagemagick-cedar-14
=====> Detected Framework: 
       Get:1 http://us.archive.ubuntu.com vivid InRelease [218 kB]
       Get:2 http://us.archive.ubuntu.com vivid-updates InRelease [65.9 kB]
       Get:3 http://us.archive.ubuntu.com vivid-backports InRelease [64.5 kB]
       Get:4 http://security.ubuntu.com vivid-security InRelease [65.9 kB]
       Get:5 http://us.archive.ubuntu.com vivid/main Sources [1,077 kB]
       Get:6 http://us.archive.ubuntu.com vivid/restricted Sources [7,162 B]
       Get:7 http://us.archive.ubuntu.com vivid/universe Sources [7,017 kB]
       Get:8 http://us.archive.ubuntu.com vivid/multiverse Sources [174 kB]
       Get:9 http://us.archive.ubuntu.com vivid/main amd64 Packages [1,364 kB]
       Get:10 http://us.archive.ubuntu.com vivid/restricted amd64 Packages [15.4 kB]
       Get:11 http://us.archive.ubuntu.com vivid/universe amd64 Packages [6,485 kB]
       Get:12 http://us.archive.ubuntu.com vivid/multiverse amd64 Packages [134 kB]
       Get:13 http://us.archive.ubuntu.com vivid/main Translation-en [793 kB]
       Get:14 http://us.archive.ubuntu.com vivid/multiverse Translation-en [103 kB]
       Get:15 http://us.archive.ubuntu.com vivid/restricted Translation-en [4,228 B]
       Get:16 http://us.archive.ubuntu.com vivid/universe Translation-en [4,456 kB]
       Get:17 http://us.archive.ubuntu.com vivid-updates/main Sources [112 kB]
       Get:18 http://us.archive.ubuntu.com vivid-updates/restricted Sources [4,351 B]
       Get:19 http://us.archive.ubuntu.com vivid-updates/universe Sources [51.7 kB]
       Get:20 http://us.archive.ubuntu.com vivid-updates/multiverse Sources [2,406 B]
       Get:21 http://us.archive.ubuntu.com vivid-updates/main amd64 Packages [289 kB]
       Get:22 http://us.archive.ubuntu.com vivid-updates/restricted amd64 Packages [13.6 kB]
       Get:23 http://us.archive.ubuntu.com vivid-updates/universe amd64 Packages [138 kB]
       Get:24 http://us.archive.ubuntu.com vivid-updates/multiverse amd64 Packages [6,028 B]
       Get:25 http://us.archive.ubuntu.com vivid-updates/main Translation-en [137 kB]
       Get:26 http://us.archive.ubuntu.com vivid-updates/multiverse Translation-en [3,120 B]
       Get:27 http://us.archive.ubuntu.com vivid-updates/restricted Translation-en [3,112 B]
       Get:28 http://us.archive.ubuntu.com vivid-updates/universe Translation-en [81.6 kB]
       Get:29 http://us.archive.ubuntu.com vivid-backports/main Sources [751 B]
       Get:30 http://us.archive.ubuntu.com vivid-backports/restricted Sources [28 B]
       Get:31 http://us.archive.ubuntu.com vivid-backports/universe Sources [6,489 B]
       Get:32 http://us.archive.ubuntu.com vivid-backports/multiverse Sources [28 B]
       Get:33 http://us.archive.ubuntu.com vivid-backports/main amd64 Packages [612 B]
       Get:34 http://us.archive.ubuntu.com vivid-backports/restricted amd64 Packages [28 B]
       Get:35 http://us.archive.ubuntu.com vivid-backports/universe amd64 Packages [5,116 B]
       Get:36 http://us.archive.ubuntu.com vivid-backports/multiverse amd64 Packages [28 B]
       Get:37 http://us.archive.ubuntu.com vivid-backports/main Translation-en [486 B]
       Get:38 http://us.archive.ubuntu.com vivid-backports/multiverse Translation-en [14 B]
       Get:39 http://us.archive.ubuntu.com vivid-backports/restricted Translation-en [14 B]
       Get:40 http://us.archive.ubuntu.com vivid-backports/universe Translation-en [5,249 B]
       Get:41 http://security.ubuntu.com vivid-security/main Sources [59.7 kB]
       Get:42 http://security.ubuntu.com vivid-security/restricted Sources [3,418 B]
       Get:43 http://security.ubuntu.com vivid-security/universe Sources [24.3 kB]
       Get:44 http://security.ubuntu.com vivid-security/multiverse Sources [1,942 B]
       Get:45 http://security.ubuntu.com vivid-security/main amd64 Packages [198 kB]
       Get:46 http://security.ubuntu.com vivid-security/restricted amd64 Packages [11.2 kB]
       Get:47 http://security.ubuntu.com vivid-security/universe amd64 Packages [77.0 kB]
       Get:48 http://security.ubuntu.com vivid-security/multiverse amd64 Packages [6,028 B]
       Get:49 http://security.ubuntu.com vivid-security/main Translation-en [95.9 kB]
       Get:50 http://security.ubuntu.com vivid-security/multiverse Translation-en [2,743 B]
       Get:51 http://security.ubuntu.com vivid-security/restricted Translation-en [2,774 B]
       Get:52 http://security.ubuntu.com vivid-security/universe Translation-en [48.1 kB]
       Fetched 23.4 MB in 9s (2,448 kB/s)
       Reading package lists...
W: Unknown Multi-Arch type 'no' for package 'compiz-core'
W: Unknown Multi-Arch type 'no' for package 'compiz-gnome'
W: Unknown Multi-Arch type 'no' for package 'compiz-core'
W: Unknown Multi-Arch type 'no' for package 'compiz-gnome'
W: You may want to run apt-get update to correct these problems
-----> Installing ImageMagick 8:6.8.9.9-5
       Reading package lists...
       Building dependency tree...
       Some packages could not be installed. This may mean that you have
       requested an impossible situation or if you are using the unstable
       distribution that some required packages have not yet been created
       or been moved out of Incoming.
       The following information may help to resolve the situation:

       The following packages have unmet dependencies:
        imagemagick : Depends: imagemagick-6.q16 (= 8:6.8.9.9-5) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
ls: cannot access /app/tmp/cache/apt/cache/archives/*.deb: No such file or directory
-----> Linking binaries
/tmp/buildpackHfzvL/bin/compile: line 44: cd: /tmp/build_4b4e0b960d42ad56ec71e78d9ab438cd/jpmckinney-image-proxy-39c855c/.apt/usr/bin: No such file or directory
 !     Push rejected, failed to compile Multipack app

return fallback image on error