Giter Site home page Giter Site logo

nginx-modules / ngx_cache_purge Goto Github PK

View Code? Open in Web Editor NEW

This project forked from frickle/ngx_cache_purge

206.0 18.0 33.0 154 KB

nginx module which support to purge ngx_http_(fastcgi|proxy|scgi|uwsgi)_module cache backend

License: Other

C 99.31% Shell 0.69%
nginx module cache purge fastcgi proxy scgi uwsgi

ngx_cache_purge's Introduction

About

ngx_cache_purge is nginx module which adds ability to purge content from FastCGI, proxy, SCGI and uWSGI caches. A purge operation removes the content with the same cache key as the purge request has.

Sponsors

Work on the original patch was fully funded by yo.se.

Status

This module is production-ready.

Configuration directives (same location syntax)

fastcgi_cache_purge

  • syntax: fastcgi_cache_purge on|off|<method> [purge_all] [from all|<ip> [.. <ip>]]
  • default: none
  • context: http, server, location

Allow purging of selected pages from FastCGI's cache.

proxy_cache_purge

  • syntax: proxy_cache_purge on|off|<method> [purge_all] [from all|<ip> [.. <ip>]]
  • default: none
  • context: http, server, location

Allow purging of selected pages from proxy's cache.

scgi_cache_purge

  • syntax: scgi_cache_purge on|off|<method> [purge_all] [from all|<ip> [.. <ip>]]
  • default: none
  • context: http, server, location

Allow purging of selected pages from SCGI's cache.

uwsgi_cache_purge

  • syntax: uwsgi_cache_purge on|off|<method> [purge_all] [from all|<ip> [.. <ip>]]
  • default: none
  • context: http, server, location

Allow purging of selected pages from uWSGI's cache.

Configuration directives (separate location syntax)

fastcgi_cache_purge

  • syntax: fastcgi_cache_purge zone_name key
  • default: none
  • context: location

Sets area and key used for purging selected pages from FastCGI's cache.

proxy_cache_purge

  • syntax: proxy_cache_purge zone_name key
  • default: none
  • context: location

Sets area and key used for purging selected pages from proxy's cache.

scgi_cache_purge

  • syntax: scgi_cache_purge zone_name key
  • default: none
  • context: location

Sets area and key used for purging selected pages from SCGI's cache.

uwsgi_cache_purge

  • syntax: uwsgi_cache_purge zone_name key
  • default: none
  • context: location

Sets area and key used for purging selected pages from uWSGI's cache.

Configuration directives (Optional)

cache_purge_response_type

  • syntax: cache_purge_response_type html|json|xml|text
  • default: html
  • context: http, server, location

Sets a response type of purging result.

Partial Keys

Sometimes it's not possible to pass the exact key cache to purge a page. For example; when the content of a cookie or the params are part of the key. You can specify a partial key adding an asterisk at the end of the URL.

curl -X PURGE /page*

The asterisk must be the last character of the key, so you must put the $uri variable at the end.

Sample configuration (same location syntax)

http {
    proxy_cache_path  /tmp/cache  keys_zone=tmpcache:10m;

    server {
        location / {
            proxy_pass         http://127.0.0.1:8000;
            proxy_cache        tmpcache;
            proxy_cache_key    "$uri$is_args$args";
            proxy_cache_purge  PURGE from 127.0.0.1;
        }
    }
}

Sample configuration (same location syntax - purge all cached files)

http {
    proxy_cache_path  /tmp/cache  keys_zone=tmpcache:10m;

    server {
        location / {
            proxy_pass         http://127.0.0.1:8000;
            proxy_cache        tmpcache;
            proxy_cache_key    "$uri$is_args$args";
            proxy_cache_purge  PURGE purge_all from 127.0.0.1 192.168.0.0/8;
        }
    }
}

Sample configuration (separate location syntax)

http {
    proxy_cache_path  /tmp/cache  keys_zone=tmpcache:10m;

    server {
        location / {
            proxy_pass         http://127.0.0.1:8000;
            proxy_cache        tmpcache;
            proxy_cache_key    "$uri$is_args$args";
        }

        location ~ /purge(/.*) {
            allow              127.0.0.1;
            deny               all;
            proxy_cache        tmpcache;
            proxy_cache_key    "$1$is_args$args";
        }
    }
}

Sample configuration (Optional)

http {
    proxy_cache_path  /tmp/cache  keys_zone=tmpcache:10m;

    cache_purge_response_type text;

    server {

        cache_purge_response_type json;

        location / { #json
            proxy_pass         http://127.0.0.1:8000;
            proxy_cache        tmpcache;
            proxy_cache_key    "$uri$is_args$args";
        }

        location ~ /purge(/.*) { #xml
            allow              127.0.0.1;
            deny               all;
            proxy_cache        tmpcache;
            proxy_cache_key    "$1$is_args$args";
            cache_purge_response_type xml;
        }

        location ~ /purge2(/.*) { # json
            allow              127.0.0.1;
            deny               all;
            proxy_cache        tmpcache;
            proxy_cache_key    "$1$is_args$args";
        }
    }

    server {

        location / { #text
            proxy_pass         http://127.0.0.1:8000;
            proxy_cache        tmpcache;
            proxy_cache_key    "$uri$is_args$args";
        }

        location ~ /purge(/.*) { #text
            allow              127.0.0.1;
            deny               all;
            proxy_cache        tmpcache;
            proxy_cache_key    "$1$is_args$args";
        }

        location ~ /purge2(/.*) { #html
            allow              127.0.0.1;
            deny               all;
            proxy_cache        tmpcache;
            proxy_cache_key    "$1$is_args$args";
            cache_purge_response_type html;
        }
    }
}

Solve problems

  • Enabling gzip_vary can lead to different results when clearing, when enabling it, you may have problems clearing the cache. For reliable operation, you can disable gzip_vary inside the location #20.

Testing

ngx_cache_purge comes with complete test suite based on Test::Nginx.

You can test it by running:

$ prove

License

Copyright (c) 2009-2014, FRiCKLE <[email protected]>
Copyright (c) 2009-2014, Piotr Sikora <[email protected]>
All rights reserved.

This project was fully funded by yo.se.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

See also

ngx_cache_purge's People

Contributors

amirhosseinvz94 avatar dbu avatar denji avatar fdintino avatar federicoheichou avatar hnakamur avatar itpp16 avatar novashdima avatar piotrsikora avatar szepeviktor avatar torden avatar tugrul avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ngx_cache_purge's Issues

Wildcard Purge not working on special variables

Hi

my config is

server {
   proxy_cache_purge PURGE from 127.0.0.1;
   server_name a.com;

   location / {
      ...
   }
}

server {
   proxy_cache_purge PURGE from 127.0.0.1;
   server_name b.com;

   location / {
      ...
   }
}

I'm purging the content by
curl -I -XPURGE -H "Host: a.com" 127.0.0.1/images/test.jpg
and is all ok

however when I tried to purge by
curl -I -XPURGE -H "Host: a.com" 127.0.0.1/images/test.*
It always giving 412

Please advice

purge with wildcard return always 200

Hi

i tried to purge content with /images.jpg*

But it's always return 200 instead of 412 if the cache is not available

is it expected? Or is there any way to achieve that just like normal cache?

"fastcgi_cache_purge" directive is incompatible with "fastcgi_cache"

I've added the following to my nginx config:

  location ~ /purge(/.*) {
    allow 127.0.0.1;
    allow 10.10.0.0/16;
    include includes/public-cache.conf;
    fastcgi_cache_purge CACHE_PUB $public_cache_key;
  }

The .conf included in the above loads config such as:

fastcgi_cache CACHE_PUB;

fastcgi_cache_methods GET HEAD POST;

map $request_uri $public_cache_key {
    default $scheme$request_method$host$request_uri;
}
...<other params truncated>...

fastcgi_cache_path /var/www/site/cache/nginx levels=1:2 keys_zone=CACHE_PUB:200m inactive=168h;

When starting nginx:

nginx: [emerg] "fastcgi_cache_purge" directive is incompatible with "fastcgi_cache" in /etc/nginx/conf.d/site.conf:12

# Line 12 being " fastcgi_cache_purge" line. 

nginx -V output

nginx version: nginx/1.14.2
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
built with OpenSSL 1.1.0f  25 May 2017 (running with OpenSSL 1.1.0j  20 Nov 2018)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.14.2/debian/debuild-base/nginx-1.14.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

412 Precondition Failed please help me

my config is like this

fastcgi_cache_path /etc/nginx-cache levels=1:2 keys_zone=WORDPRESS:1400m inactive=1300m;
fastcgi_cache_key "$request_uri$mobile_request";

fastcgi_cache_purge PURGE from 127.0.0.1;

location ~ /purga(/.)?. {
fastcgi_cache_purge WORDPRESS "$1mobileversion";

}
curl -X GET example.com/purga/example

its work but

curl -X PURGE example.com/example

412 Precondition Failed

I've been trying to solve it for over a week, please help. No matter what I do I can't solve the problem. please help please

How are vary headers treated?

I have a scenario where the upstream server (in this case Django) may set a few vary headers. I do ignore Set-Cookie in the Nginx configuration.

  1. Visit /path/ from a web browser -> creates a file in the cache directory.
  2. Visit same /path/ with curl -X GET from a command line on the webserver -> creates another file in the cache directory.

I assume this is because of different Accept-Encoding's and I understand the reasoning.

Now, when I issue curl -X PURGE http://localhost/path/ only the cached file from (2) is deleted.

Is there a way to tell the module to consider only the cache key, no other headers, when determining which cached files to delete?

Crash purge page, "server_tokens" not "on"

Hi, I have NGINX compiled with the --add-module=/tmp/naxsi/naxsi_src --add-module=/tmp/ngx_cache_purge module options (placed first in the ./configure order, if that matters). All of this stuff is running in Docker on a Kubernetes cluster with PHP FPM.

I'm on a single site setup using this configuration. I have the fastcgi_cache_path, fastcgi_cache_key, fastcgi_cache_use_stale, and fastcgi_ignore_headers directives in the http {} context.

I can't get any of the cache files to purge using either the "Purge Entire Cache" button or the PURGE/url method. Neither does the automatic purging work (I'm verifying by watching the /var/run/nginx-cache directory).

When I try the PURGE/url method I get an NGINX error in the log - [alert] 1#1: worker process 114 exited on signal 11.

NGINX version:

nginx version: nginx/1.12.0
built by gcc 6.3.0 20170415 (Debian 6.3.0-14) 
built with OpenSSL 1.1.0e  16 Feb 2017
TLS SNI support enabled

Failed to purge a cached POST method by key

That is what I have to initiate cache:

uwsgi_cache vsp-cache;
uwsgi_cache_lock on;
uwsgi_cache_lock_timeout 5s;
uwsgi_cache_valid 15m;
uwsgi_cache_key "$role:$proxy_host$uri$is_args$args:$request_body_md5";
uwsgi_cache_methods GET HEAD POST;

As it is shown I cache not only GET and HEAD methods but POST methods as well.

Purging is realized in a separate location with a key being passed within the url path:

location ~ ^/purge/develop/(.*) { #text
    allow              127.0.0.1;
    uwsgi_cache_purge  vsp-cache $1;
}

When I try to purge a cached GET request with the key associated with that request everything works just fine and the cached request is removed. But the same logic doesn't work when I try to purge a cached POST request. I get a response with 412 status code which means the key is not found I suppose.

The question is how to purge a cached POST request from NGINX cache with a key passed in the url path?

Update Releases

Last release from 2017.02.21
Last change on master - few days ago. In between were merged several bug fixes.
please, keep releases up to date.

ngx_cache_purge falls when nginx compiled with module aio

  1. This construction with AIO breaks down when requested: curl -IXPURGE localhost/favicon.*
nginx version: nginx/1.11.10
built by gcc 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) 
built with OpenSSL 1.1.0e  16 Feb 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-openssl=openssl-1.1.0e --with-http_ssl_module --add-module=ngx_cache_purge --with-file-aio --with-cc-opt=-g --with-debug

$ curl -IXGET localhost/favicon.ico
HTTP/1.1 200 OK
Server: nginx/1.11.10
Date: Wed, 22 Feb 2017 15:32:08 GMT
Content-Type: image/x-icon
Content-Length: 1150
Connection: keep-alive
Last-Modified: Tue, 21 Feb 2017 11:26:54 GMT
ETag: "58ac23fe-47e"
X-Cache-Status: HIT
Accept-Ranges: bytes

$ curl -IXPURGE localhost/favicon.*
curl: (52) Empty reply from server

Output of gdb contains in attached file
gdb-core-dump.txt


  1. In construction without module aio all is well:
nginx version: nginx/1.11.10
built by gcc 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) 
built with OpenSSL 1.1.0e  16 Feb 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-openssl=openssl-1.1.0e --with-http_ssl_module --add-module=ngx_cache_purge --with-cc-opt=-g --with-debug

$ curl -IXGET localhost/favicon.ico
HTTP/1.1 200 OK
Server: nginx/1.11.10
Date: Wed, 22 Feb 2017 16:34:07 GMT
Content-Type: image/x-icon
Content-Length: 1150
Connection: keep-alive
Last-Modified: Tue, 21 Feb 2017 11:26:54 GMT
ETag: "58ac23fe-47e"
X-Cache-Status: MISS
Accept-Ranges: bytes

$ curl -IXPURGE localhost/favicon.*
HTTP/1.1 200 OK
Server: nginx/1.11.10
Date: Wed, 22 Feb 2017 16:34:08 GMT
Content-Type: text/html
Content-Length: 214
Connection: keep-alive

Attached config-file: nginx.conf.txt

Got 'Empty reply from server' while trying to PURGE cache.

Everything worked fine with nginx_1.11.1 + FRiCKLE/ngx_cache_purge bundle. Then, in order to implement APLN for http2 support, I recompiled nginx with openssl-1.1.0c and nginx-modules/ngx_cache_purge.
After that I started getting Empty reply from server on my purge cache requests.

# curl -X PURGE http://127.0.0.1/test
curl: (52) Empty reply from server

nginx -V:

nginx version: nginx/1.11.6
built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04) 
built with OpenSSL 1.1.0c  10 Nov 2016
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' --with-openssl=/home/wizard/dist/openssl/openssl-1.1.0c --add-dynamic-module=/home/wizard/dist/ngx_cache_purge

nginx -T configuration file /etc/nginx/nginx.conf:

user  nginx;
worker_processes  1;

load_module "modules/ngx_http_cache_purge_module.so";

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}





http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    error_log  /var/log/nginx/error.log  debug;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

# configuration file /etc/nginx/mime.types:

types {
    text/html                             html htm shtml;
    text/css                              css;
    text/xml                              xml;
    image/gif                             gif;
    image/jpeg                            jpeg jpg;
    application/javascript                js;
    application/atom+xml                  atom;
    application/rss+xml                   rss;

    text/mathml                           mml;
    text/plain                            txt;
    text/vnd.sun.j2me.app-descriptor      jad;
    text/vnd.wap.wml                      wml;
    text/x-component                      htc;

    image/png                             png;
    image/tiff                            tif tiff;
    image/vnd.wap.wbmp                    wbmp;
    image/x-icon                          ico;
    image/x-jng                           jng;
    image/x-ms-bmp                        bmp;
    image/svg+xml                         svg svgz;
    image/webp                            webp;

    application/font-woff                 woff;
    application/java-archive              jar war ear;
    application/json                      json;
    application/mac-binhex40              hqx;
    application/msword                    doc;
    application/pdf                       pdf;
    application/postscript                ps eps ai;
    application/rtf                       rtf;
    application/vnd.apple.mpegurl         m3u8;
    application/vnd.ms-excel              xls;
    application/vnd.ms-fontobject         eot;
    application/vnd.ms-powerpoint         ppt;
    application/vnd.wap.wmlc              wmlc;
    application/vnd.google-earth.kml+xml  kml;
    application/vnd.google-earth.kmz      kmz;
    application/x-7z-compressed           7z;
    application/x-cocoa                   cco;
    application/x-java-archive-diff       jardiff;
    application/x-java-jnlp-file          jnlp;
    application/x-makeself                run;
    application/x-perl                    pl pm;
    application/x-pilot                   prc pdb;
    application/x-rar-compressed          rar;
    application/x-redhat-package-manager  rpm;
    application/x-sea                     sea;
    application/x-shockwave-flash         swf;
    application/x-stuffit                 sit;
    application/x-tcl                     tcl tk;
    application/x-x509-ca-cert            der pem crt;
    application/x-xpinstall               xpi;
    application/xhtml+xml                 xhtml;
    application/xspf+xml                  xspf;
    application/zip                       zip;

    application/octet-stream              bin exe dll;
    application/octet-stream              deb;
    application/octet-stream              dmg;
    application/octet-stream              iso img;
    application/octet-stream              msi msp msm;

    application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
    application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

    audio/midi                            mid midi kar;
    audio/mpeg                            mp3;
    audio/ogg                             ogg;
    audio/x-m4a                           m4a;
    audio/x-realaudio                     ra;

    video/3gpp                            3gpp 3gp;
    video/mp2t                            ts;
    video/mp4                             mp4;
    video/mpeg                            mpeg mpg;
    video/quicktime                       mov;
    video/webm                            webm;
    video/x-flv                           flv;
    video/x-m4v                           m4v;
    video/x-mng                           mng;
    video/x-ms-asf                        asx asf;
    video/x-ms-wmv                        wmv;
    video/x-msvideo                       avi;
}

# configuration file /etc/nginx/conf.d/default.conf:
proxy_cache_path /data/nginx/img_cache levels=1:2 keys_zone=img_cache:10m max_size=100m inactive=90d;

server {
    listen       80;
    server_name  localhost;

    resolver 8.8.8.8 ipv6=off;

    #charset koi8-r;
    #access_log  /var/log/nginx/log/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    location /test {
        proxy_intercept_errors on;

        proxy_cache img_cache;
        proxy_cache_key $uri;

        proxy_cache_lock on;
        proxy_cache_lock_timeout 3s;
        proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
        proxy_ignore_headers "Set-Cookie";
        proxy_cache_valid 90d;

        proxy_cache_purge PURGE from all;

        proxy_pass http://nginx.org/nginx.png;
    }

    location /purge {
        proxy_cache_purge img_cache /test;
    }

}

Debug log:


2016/12/12 13:40:12 [debug] 31487#31487: accept on 0.0.0.0:80, ready: 0
2016/12/12 13:40:12 [debug] 31487#31487: posix_memalign: 0000000000E27B30:512 @16
2016/12/12 13:40:12 [debug] 31487#31487: *2 accept: 127.0.0.1:32966 fd:12
2016/12/12 13:40:12 [debug] 31487#31487: *2 event timer add: 12: 60000:1481550072300
2016/12/12 13:40:12 [debug] 31487#31487: *2 reusable connection: 1
2016/12/12 13:40:12 [debug] 31487#31487: *2 epoll add event: fd:12 op:1 ev:80002001
2016/12/12 13:40:12 [debug] 31487#31487: *2 http wait request handler
2016/12/12 13:40:12 [debug] 31487#31487: *2 malloc: 0000000000E27D40:1024
2016/12/12 13:40:12 [debug] 31487#31487: *2 recv: eof:0, avail:1
2016/12/12 13:40:12 [debug] 31487#31487: *2 recv: fd:12 79 of 1024
2016/12/12 13:40:12 [debug] 31487#31487: *2 reusable connection: 0
2016/12/12 13:40:12 [debug] 31487#31487: *2 posix_memalign: 0000000000E17D00:4096 @16
2016/12/12 13:40:12 [debug] 31487#31487: *2 http process request line
2016/12/12 13:40:12 [debug] 31487#31487: *2 http request line: "PURGE /test HTTP/1.1"
2016/12/12 13:40:12 [debug] 31487#31487: *2 http uri: "/test"
2016/12/12 13:40:12 [debug] 31487#31487: *2 http args: ""
2016/12/12 13:40:12 [debug] 31487#31487: *2 http exten: ""
2016/12/12 13:40:12 [debug] 31487#31487: *2 http process request header line
2016/12/12 13:40:12 [debug] 31487#31487: *2 http header: "User-Agent: curl/7.35.0"
2016/12/12 13:40:12 [debug] 31487#31487: *2 http header: "Host: 127.0.0.1"
2016/12/12 13:40:12 [debug] 31487#31487: *2 http header: "Accept: */*"
2016/12/12 13:40:12 [debug] 31487#31487: *2 http header done
2016/12/12 13:40:12 [debug] 31487#31487: *2 event timer del: 12: 1481550072300
2016/12/12 13:40:12 [debug] 31487#31487: *2 generic phase: 0
2016/12/12 13:40:12 [debug] 31487#31487: *2 rewrite phase: 1
2016/12/12 13:40:12 [debug] 31487#31487: *2 test location: "/"
2016/12/12 13:40:12 [debug] 31487#31487: *2 test location: "purge"
2016/12/12 13:40:12 [debug] 31487#31487: *2 test location: "test"
2016/12/12 13:40:12 [debug] 31487#31487: *2 using configuration "/test"
2016/12/12 13:40:12 [debug] 31487#31487: *2 http cl:-1 max:1048576
2016/12/12 13:40:12 [debug] 31487#31487: *2 rewrite phase: 3
2016/12/12 13:40:12 [debug] 31487#31487: *2 post rewrite phase: 4
2016/12/12 13:40:12 [debug] 31487#31487: *2 generic phase: 5
2016/12/12 13:40:12 [debug] 31487#31487: *2 generic phase: 6
2016/12/12 13:40:12 [debug] 31487#31487: *2 generic phase: 7
2016/12/12 13:40:12 [debug] 31487#31487: *2 access phase: 8
2016/12/12 13:40:12 [debug] 31487#31487: *2 access phase: 9
2016/12/12 13:40:12 [debug] 31487#31487: *2 access phase: 10
2016/12/12 13:40:12 [debug] 31487#31487: *2 post access phase: 11
2016/12/12 13:40:12 [debug] 31487#31487: *2 posix_memalign: 0000000000E0A140:4096 @16
2016/12/12 13:40:12 [debug] 31487#31487: *2 http set discard body
2016/12/12 13:40:12 [alert] 31485#31485: worker process 31487 exited on signal 11 (core dumped)
2016/12/12 13:40:12 [debug] 31496#31496: epoll add event: fd:6 op:1 ev:00002001

purge returns 404

This may not an issue with the purge module itself, but i am looking for an explanation. My settings are quite simple. Just as the example but with a different cache key and I have tried the same example but still getting same result.

In vhost.conf

  location ~ /purge(/.*) {
            proxy_cache_purge  cache_proxy $scheme$request_method$host$request_uri;
  }

In nginx.conf

 proxy_cache_path /usr/local/nginx/proxy levels=1:2 keys_zone=cache_proxy:15m inactive=4h max_size=1000m use_temp_path=off;
    proxy_cache_key $scheme$request_method$host$request_uri;
    proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504 http_403 http_404;
    proxy_cache_valid  any 1m;

When I try to purge the HTTP status is 404. I always make sure that X-Proxy-Status: HIT when requesting something like: curl -I http://example.com/test.php and trying to purge it with curl -I http://example.com/purge/test.php -XPURGE

Could example to me what is the wrong with my configuration ?
I am using nginx 10.x with this module and also tried the original one but still same result

Not working with DELETE?

I use proxy_cache to cache S3 bucket contents.

http {
    proxy_cache_path  /tmp/cache  keys_zone=tmp_cache:10m;

    # KEY will be '/bucket/file.ext"
    map $request_uri $request_path {
        "~^(?P<path>[^?]*)(\?.*)?$" $path;
    }

    server {
        location / {

        proxy_redirect              off;

        proxy_http_version          1.1;

        proxy_set_header            Host $http_host;
        proxy_set_header            Authorization $http_authorization;
        proxy_set_header            Connection "";
        proxy_set_header            Proxy "";
        proxy_set_header            Accept-Encoding "";

        proxy_cache                 tmp_cache;
        proxy_cache_methods         GET HEAD;
        proxy_cache_key             $request_path;
        proxy_cache_purge           DELETE from all;
        proxy_cache_valid           200 30d;
        proxy_cache_lock            on;
        proxy_cache_convert_head    off;
        proxy_cache_use_stale       error timeout;
        }
    }
}

Uploading a 2.0M file to the bucket

curl -X PUT file.pdf "http://127.0.0.1:8000/bucket"

Download it

curl "http://127.0.0.1:8000/bucket/file.pdf" -o file.pdf

Check the cache folder

du -sh /tmp/cache
2.0M

Delete the file (should delete the cache too)

curl -X DELETE http://127.0.0.1:8000/bucket/file

Check the cache folder

du -sh /var/cache
2.0M

partial purge with ngx_http_slice_module returns 404

slice              1m;
proxy_cache_key    $host$uri$is_args$args$slice_range;
proxy_set_header   Range $slice_range;

After several test, I found when the proxy_cache_key ends with $slice_range which is a key config of ngx_http_slice_module, the partial purge will return 404.

The partial purge can work after I remove $slice_range from the cache key.

With my test, the purge_all function can work well under this situation.

Fixed URI sometimes not Deleted

I've implemented #36 however even tough I've fixed uri such as /images/image.jpg, cache doesn't removed

On debug log, sometimes I saw http file cache purge with partial enabled running even I loop the same URI without wildcard

Onother weird debug log that I found key is not deleted is like below

2021/06/03 08:30:58 [debug] 9#9: http cache file "/data/nginx/1a33c4ca21ce8/9/3f/0d8c2799f208c5e0781d98945e6043f9" key read: "1a33c4ca21ce8/assets/web_1.0/fonts/roboto-v18-latin-300.woff2-https://andy.handsome.local2òØœŽ–´¤kp�*��Ô/•üÌ¢�f�"£°÷�[��îx�z�´�·ÙïÖ´úK�ç¬�xJ
UÎÀv�±¨ñ��6�žþܳ�Ë�>#ì×h¬
˜w�Vâšõˆ
]»ÊŸ³¸êµø®�¹È­$�˜_=u�é*Ñá�d_‹7ÁjUeTQñ3_yRŸ�W�*“�cýÚ§ã)Ó©‹3 ©Ð|ÿ"

Purge pages with arguments /purge/?*

I have some pages like / and /example and using proxy_cache_key $uri$is_args$args;, so I have some cached pages with /?a=1. So I don't wanna purge the entire cache with /purge/*, I want just to purge cached pages with additional params, like this /purge/?*. That request get endless and Nginx give me that log:

nginx_1   | 2020/07/26 04:32:04 [alert] 1#1: worker process 3321 exited on signal 11
nginx_1   | 2020/07/26 04:32:04 [alert] 1#1: worker process 3322 exited on signal 11
nginx_1   | 2020/07/26 04:32:04 [alert] 1#1: worker process 3323 exited on signal 11
nginx_1   | 2020/07/26 04:32:04 [alert] 1#1: worker process 3324 exited on signal 11

How to purge any Accept-Encoding cache ?

find /data/proxy_cache/ -type f |wc -l
0

192.168.60.99 m.xx.com - - [10/Aug/2018:01:35:47 +0800] "GET /purge/ask/118060.html HTTP/1.1" 412 186 - "-" "python-requests/2.19.1" "-" "-""0.000" "-" ""
192.168.60.99 m.xx.com - - [10/Aug/2018:01:35:47 +0800] "GET /purge/ask/118060.html HTTP/1.1" 412 186 - "-" "python-requests/2.19.1" "-" "-""0.000" "-" "gzip"
192.168.60.99 m.xx.com - - [10/Aug/2018:01:35:47 +0800] "GET /ask/118060.html HTTP/1.1" 200 23294 - "-" "python-requests/2.19.1" "-" "-""0.001" "MISS" ""
192.168.60.99 m.xx.com - - [10/Aug/2018:01:35:47 +0800] "GET /ask/118060.html HTTP/1.1" 200 5863 - "-" "python-requests/2.19.1" "-" "-""0.001" "MISS" "gzip"

find /data/proxy_cache/ -type f |wc -l
2

192.168.60.99 m.xx.com - - [10/Aug/2018:01:36:55 +0800] "GET /purge/ask/118060.html HTTP/1.1" 200 241 - "-" "python-requests/2.19.1" "-" "-""0.000" "-" ""
192.168.60.99 m.xx.com - - [10/Aug/2018:01:36:55 +0800] "GET /purge/ask/118060.html HTTP/1.1" 412 186 - "-" "python-requests/2.19.1" "-" "-""0.000" "-" "gzip"

find /data/proxy_cache/ -type f |wc -l
1

Unable to clear cache Accept-Encoding: gzip
/proxy_cache/ad/47/720ed608237b61e85ceac7d253e447ad

fastcgi_cache_purge not working

Here is my setup:

at http level:

fastcgi_cache_path /tmp/nginx-cache levels=1:2 keys_zone=WORDPRESS:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
fastcgi_cache_use_stale error timeout invalid_header http_500;

at server level:

location ~* /purge(/.*) {
fastcgi_cache_purge WORDPRESS "$scheme$request_method$host$1";
}

No matter what purge does not work.

I enabled debug logs on nginx and here is what i see:

2020/07/30 10:18:05 [debug] 44967#44967: *7 http script var: "http"
2020/07/30 10:18:05 [debug] 44967#44967: *7 http script var: "GET"
2020/07/30 10:18:05 [debug] 44967#44967: *7 http script var: "localhost"
2020/07/30 10:18:05 [debug] 44967#44967: *7 http script capture: "/"
2020/07/30 10:18:05 [debug] 44967#44967: *7 http cache key: "httpGETlocalhost/"
2020/07/30 10:18:05 [debug] 44967#44967: *7 add cleanup: 000055DEB1BAA658
2020/07/30 10:18:05 [debug] 44967#44967: *7 http file cache exists: 0 e:1
2020/07/30 10:18:05 [debug] 44967#44967: *7 cache file: "/tmp/nginx-cache/8/60/e59c4686e1048ac50e848367eccc6608"
2020/07/30 10:18:05 [debug] 44967#44967: *7 add cleanup: 000055DEB1BAA6A8
2020/07/30 10:18:05 [debug] 44967#44967: *7 http file cache fd: 12
2020/07/30 10:18:05 [debug] 44967#44967: *7 thread read: 12, 000055DEB1BAA728, 515, 0
2020/07/30 10:18:05 [debug] 44967#44967: *7 http file cache purge: -1, "/tmp/nginx-cache/8/60/e59c4686e1048ac50e848367eccc6608"
2020/07/30 10:18:05 [debug] 44967#44967: *7 http finalize request: 500, "/purge/?" a:1, c:2
2020/07/30 10:18:05 [debug] 44967#44967: *7 http special response: 500, "/purge/?"
2020/07/30 10:18:05 [debug] 44967#44967: *7 http set discard body
2020/07/30 10:18:05 [debug] 44967#44967: *7 posix_memalign: 000055DEB1BA02B0:4096 @16
2020/07/30 10:18:05 [debug] 44967#44967: *7 HTTP/1.1 500 Internal Server Error
Server: nginx
Date: Thu, 30 Jul 2020 10:18:05 GMT
Content-Type: text/html
Content-Length: 572
Connection: close
X-Xss-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff

not work

I see a transaction successful message when I visit www.xxx.com/purge/*

The files in /var/cache/nginx/domain.com_proxy/ remain unchanged and the site comes from the cache again.

nginx: [emerg] invalid parameter "purge_all"

When I try to save:

location ~ /purgeall {
fastcgi_pass localhost:8001;
fastcgi_cache xyz.com;
fastcgi_cache_key $uri$is_args$args;
fastcgi_cache_purge PURGE purge_all from 127.0.0.1;
}

Failed to save configuration file : Configuration is invalid : nginx: [emerg] invalid parameter "purge_all", expected "from" keyword in /etc/nginx/sites-enabled/xyz.com.conf:49 nginx: configuration file /etc/nginx/nginx.conf test failed

nginx version: nginx/1.14.1

Has anyone had the same issue?

Using optional configuration fails with `unknown directive "cache_purge_response_type"`

Whenever I try to use the optional configuration directive cache_purge_response_type NGINX fails with:

unknown directive "cache_purge_response_type" in [...]

Here is what I got:

  • Build configuration: --conf-path=/etc/nginx/nginx.conf --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --with-http_ssl_module --with-http_v2_module --add-dynamic-module=../ngx_http_geoip2_module-2.0 --add-dynamic-module=../ngx_http_cache_purge_module-2.4.2
  • NGINX version: 1.12.2
  • OS: Ubuntu 16.04.5

I've tried every context without success. Beside that it works! :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.