Giter Site home page Giter Site logo

spiderlabs / modsecurity-nginx Goto Github PK

View Code? Open in Web Editor NEW
1.5K 86.0 271.0 279 KB

ModSecurity v3 Nginx Connector

License: Apache License 2.0

C 42.57% Perl 48.31% Shell 5.62% Dockerfile 3.09% Batchfile 0.41%
modsecurity nginx modsecurity-nginx nginx-connector waf libmodsecurity

modsecurity-nginx's Introduction

Build Status

The ModSecurity-nginx connector is the connection point between nginx and libmodsecurity (ModSecurity v3). Said another way, this project provides a communication channel between nginx and libmodsecurity. This connector is required to use LibModSecurity with nginx.

The ModSecurity-nginx connector takes the form of an nginx module. The module simply serves as a layer of communication between nginx and ModSecurity.

Notice that this project depends on libmodsecurity rather than ModSecurity (version 2.9 or less).

What is the difference between this project and the old ModSecurity add-on for nginx?

The old version uses ModSecurity standalone, which is a wrapper for Apache internals to link ModSecurity to nginx. This current version is closer to nginx, consuming the new libmodsecurity which is no longer dependent on Apache. As a result, this current version has less dependencies, fewer bugs, and is faster. In addition, some new functionality is also provided - such as the possibility of use of global rules configuration with per directory/location customizations (e.g. SecRuleRemoveById).

Compilation

Before compile this software make sure that you have libmodsecurity installed. You can download it from the ModSecurity git repository. For information pertaining to the compilation and installation of libmodsecurity please consult the documentation provided along with it.

With libmodsecurity installed, you can proceed with the installation of the ModSecurity-nginx connector, which follows the nginx third-party module installation procedure. From the nginx source directory:

./configure --add-module=/path/to/ModSecurity-nginx

Or, to build a dynamic module:

./configure --add-dynamic-module=/path/to/ModSecurity-nginx --with-compat

Note that when building a dynamic module, your nginx source version needs to match the version of nginx you're compiling this for.

Further information about nginx third-party add-ons support are available here: http://wiki.nginx.org/3rdPartyModules

Usage

ModSecurity for nginx extends your nginx configuration directives. It adds four new directives and they are:

modsecurity

syntax: modsecurity on | off

context: http, server, location

default: off

Turns on or off ModSecurity functionality. Note that this configuration directive is no longer related to the SecRule state. Instead, it now serves solely as an nginx flag to enable or disable the module.

modsecurity_rules_file

syntax: modsecurity_rules_file <path to rules file>

context: http, server, location

default: no

Specifies the location of the modsecurity configuration file, e.g.:

server {
    modsecurity on;
    location / {
        root /var/www/html;
        modsecurity_rules_file /etc/my_modsecurity_rules.conf;
    }
}

modsecurity_rules_remote

syntax: modsecurity_rules_remote <key> <URL to rules>

context: http, server, location

default: no

Specifies from where (on the internet) a modsecurity configuration file will be downloaded. It also specifies the key that will be used to authenticate to that server:

server {
    modsecurity on;
    location / {
        root /var/www/html;
        modsecurity_rules_remote my-server-key https://my-own-server/rules/download;
    }
}

modsecurity_rules

syntax: modsecurity_rules <modsecurity rule>

context: http, server, location

default: no

Allows for the direct inclusion of a ModSecurity rule into the nginx configuration. The following example is loading rules from a file and injecting specific configurations per directory/alias:

server {
    modsecurity on;
    location / {
        root /var/www/html;
        modsecurity_rules_file /etc/my_modsecurity_rules.conf;
    }
    location /ops {
        root /var/www/html/opts;
        modsecurity_rules '
          SecRuleEngine On
          SecDebugLog /tmp/modsec_debug.log
          SecDebugLogLevel 9
          SecRuleRemoveById 10
        ';
    }
}

modsecurity_transaction_id

syntax: modsecurity_transaction_id string

context: http, server, location

default: no

Allows to pass transaction ID from nginx instead of generating it in the library. This can be useful for tracing purposes, e.g. consider this configuration:

log_format extended '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" $request_id';

server {
    server_name host1;
    modsecurity on;
    modsecurity_transaction_id "host1-$request_id";
    access_log logs/host1-access.log extended;
    error_log logs/host1-error.log;
    location / {
        ...
    }
}

server {
    server_name host2;
    modsecurity on;
    modsecurity_transaction_id "host2-$request_id";
    access_log logs/host2-access.log extended;
    error_log logs/host2-error.log;
    location / {
        ...
    }
}

Using a combination of log_format and modsecurity_transaction_id you will be able to find correlations between access log and error log entries using the same unique identificator.

String can contain variables.

Contributing

As an open source project we invite (and encourage) anyone from the community to contribute to our project. This may take the form of: new functionality, bug fixes, bug reports, beginners user support, and anything else that you are willing to help with. Thank you.

Providing Patches

We prefer to have your patch within the GitHub infrastructure to facilitate our review work, and our QA integration. GitHub provides an excellent documentation on how to perform “Pull Requests”. More information available here: https://help.github.com/articles/using-pull-requests/

Please respect the coding style in use. Pull requests can include various commits, so provide one fix or one functionality per commit. Do not change anything outside the scope of your target work (e.g. coding style in a function that you have passed by).

Don’t know where to start?

Within our code there are various items marked as TODO or FIXME that may need your attention. Check the list of items by performing a grep:

$ cd /path/to/modsecurity-nginx
$ egrep -Rin "TODO|FIXME" -R *

You may also take a look at recent bug reports and open issues to get an idea of what kind of help we are looking for.

Testing your patch

Along with the manual testing, we strongly recommend that you to use the nginx test utility to make sure that you patch does not adversely affect the behavior or performance of nginx.

The nginx tests are available on: http://hg.nginx.org/nginx-tests/

To use those tests, make sure you have the Perl utility prove (part of Perl 5) and proceed with the following commands:

$ cp /path/to/ModSecurity-nginx/tests/* /path/to/nginx/test/repository
$ cd /path/to/nginx/test/repository
$ TEST_NGINX_BINARY=/path/to/your/nginx prove .

If you are facing problems getting your added functionality to pass all the nginx tests, feel free to contact us or the nginx mailing list at: http://nginx.org/en/support.html

Debugging

We respect the nginx debugging schema. By using the configuration option "--with-debug" during the nginx configuration you will also be enabling the connector's debug messages. Core dumps and crashes are expected to be debugged in the same fashion that is used to debug nginx. For further information, please check the nginx debugging information: http://wiki.nginx.org/Debugging

Reporting Issues

If you are facing a configuration issue or if something is not working as you expect it to be, please use ModSecurity user’s mailing list. Issues on GitHub are also welcome, but we prefer to have users question on the mailing list first, where you can reach an entire community. Also don’t forget to look for an existing issue before opening a new one.

Lastly, If you are planning to open an issue on GitHub, please don’t forget to tell us the version of your libmodsecurity and the version of the nginx connector you are running.

Security issue

Please do not publicly report any security issue. Instead, contact us at: [email protected] to report the issue. Once the problem is fixed we will provide you with credit for the discovery.

Feature Request

We would love to discuss any ideas that you may have for a new feature. Please keep in mind this is a community driven project so be sure to contact the community via the mailing list to get feedback first. Alternatively, feel free to open GitHub issues requesting for new features. Before opening a new issue, please check if there is an existing feature request for the desired functionality.

Packaging

Having our packages in distros on time is something we highly desire. Let us know if there is anything we can do to facilitate your work as a packager.

modsecurity-nginx's People

Contributors

airisx avatar airween avatar brandonpayton avatar defanator avatar eduar-hte avatar fpnuseis avatar linuxjedi avatar martinhsv avatar nikolas avatar p0pr0ck5 avatar phantom-az avatar synchro avatar turchanov avatar victorhora avatar zimmerle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

modsecurity-nginx's Issues

nginx test cases failed

Hello,
I have to installed nginx through source code.
hg clone http://hg.ginx.org/nginx
then ls
auto conf contrib docs misc src
there was no configure file to ran ./configure command
I had ran it through auto folder which contains configure .
then make
make install
output of ./configure
checking for OS

  • Linux 3.16.0-23-generic x86_64
    checking for C compiler ... found
  • using GNU C compiler
  • gcc version: 4.9.1 (Ubuntu 4.9.1-16ubuntu6)
    checking for gcc -pipe switch ... found
    checking for gcc builtin atomic operations ... found
    checking for C99 variadic macros ... found
    checking for gcc variadic macros ... found
    checking for unistd.h ... found
    checking for inttypes.h ... found
    checking for limits.h ... found
    checking for sys/filio.h ... not found
    checking for sys/param.h ... found
    checking for sys/mount.h ... found
    checking for sys/statvfs.h ... found
    checking for crypt.h ... found
    checking for Linux specific features
    checking for epoll ... found
    checking for EPOLLRDHUP ... found
    checking for O_PATH ... found
    checking for sendfile() ... found
    checking for sendfile64() ... found
    checking for sys/prctl.h ... found
    checking for prctl(PR_SET_DUMPABLE) ... found
    checking for sched_setaffinity() ... found
    checking for crypt_r() ... found
    checking for sys/vfs.h ... found
    checking for nobody group ... not found
    checking for nogroup group ... found
    checking for poll() ... found
    checking for /dev/poll ... not found
    checking for kqueue ... not found
    checking for crypt() ... not found
    checking for crypt() in libcrypt ... found
    checking for F_READAHEAD ... not found
    checking for posix_fadvise() ... found
    checking for O_DIRECT ... found
    checking for F_NOCACHE ... not found
    checking for directio() ... not found
    checking for statfs() ... found
    checking for statvfs() ... found
    checking for dlopen() ... not found
    checking for dlopen() in libdl ... found
    checking for sched_yield() ... found
    checking for SO_SETFIB ... not found
    checking for SO_REUSEPORT ... found
    checking for SO_ACCEPTFILTER ... not found
    checking for TCP_DEFER_ACCEPT ... found
    checking for TCP_KEEPIDLE ... found
    checking for TCP_FASTOPEN ... found
    checking for TCP_INFO ... found
    checking for accept4() ... found
    checking for eventfd() ... found
    checking for int size ... 4 bytes
    checking for long size ... 8 bytes
    checking for long long size ... 8 bytes
    checking for void * size ... 8 bytes
    checking for uint64_t ... found
    checking for sig_atomic_t ... found
    checking for sig_atomic_t size ... 4 bytes
    checking for socklen_t ... found
    checking for in_addr_t ... found
    checking for in_port_t ... found
    checking for rlim_t ... found
    checking for uintptr_t ... uintptr_t found
    checking for system byte ordering ... little endian
    checking for size_t size ... 8 bytes
    checking for off_t size ... 8 bytes
    checking for time_t size ... 8 bytes
    checking for setproctitle() ... not found
    checking for pread() ... found
    checking for pwrite() ... found
    checking for pwritev() ... found
    checking for sys_nerr ... found
    checking for localtime_r() ... found
    checking for posix_memalign() ... found
    checking for memalign() ... found
    checking for mmap(MAP_ANON|MAP_SHARED) ... found
    checking for mmap("/dev/zero", MAP_SHARED) ... found
    checking for System V shared memory ... found
    checking for POSIX semaphores ... not found
    checking for POSIX semaphores in libpthread ... found
    checking for struct msghdr.msg_control ... found
    checking for ioctl(FIONBIO) ... found
    checking for struct tm.tm_gmtoff ... found
    checking for struct dirent.d_namlen ... not found
    checking for struct dirent.d_type ... found
    checking for sysconf(_SC_NPROCESSORS_ONLN) ... found
    checking for openat(), fstatat() ... found
    checking for getaddrinfo() ... found
    checking for PCRE library ... found
    checking for PCRE JIT support ... found
    checking for md5 in system md library ... not found
    checking for md5 in system md5 library ... not found
    checking for md5 in system OpenSSL crypto library ... found
    checking for sha1 in system md library ... not found
    checking for sha1 in system OpenSSL crypto library ... found
    checking for zlib library ... found
    creating objs/Makefile

Configuration summary

  • using system PCRE library

  • OpenSSL library is not used

  • md5: using system crypto library

  • sha1: using system crypto library

  • using system zlib library

    nginx path prefix: "/usr/local/nginx"
    nginx binary file: "/usr/local/nginx/sbin/nginx"
    nginx configuration prefix: "/usr/local/nginx/conf"
    nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
    nginx pid file: "/usr/local/nginx/logs/nginx.pid"
    nginx error log file: "/usr/local/nginx/logs/error.log"
    nginx http access log file: "/usr/local/nginx/logs/access.log"
    nginx http client request body temporary files: "client_body_temp"
    nginx http proxy temporary files: "proxy_temp"
    nginx http fastcgi temporary files: "fastcgi_temp"
    nginx http uwsgi temporary files: "uwsgi_temp"
    nginx http scgi temporary files: "scgi_temp"

then I have ran separate test case folder an ran it

TEST_NGINX_BINARY=/path/to/nginx/objs/nginx prove . > test_status

output was test_status

./access.t ............................. skipped: no ipv6 available
./access_log.t .........................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/10 subtests
./access_log_variables.t ............... ok
./addition.t ........................... ok
./auth_basic.t .........................
Dubious, test returned 14 (wstat 3584, 0xe00)
Failed 14/22 subtests
./auth_request.t .......................
Dubious, test returned 8 (wstat 2048, 0x800)
Failed 8/20 subtests
(less 2 skipped subtests: 10 okay)
./auth_request_satisfy.t ...............
Dubious, test returned 6 (wstat 1536, 0x600)
Failed 6/19 subtests
./auth_request_set.t ...................
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/7 subtests
./autoindex.t ..........................
Dubious, test returned 15 (wstat 3840, 0xf00)
Failed 15/17 subtests
./autoindex_format.t ...................
Dubious, test returned 33 (wstat 8448, 0x2100)
Failed 33/38 subtests
./body.t ...............................
Dubious, test returned 6 (wstat 1536, 0x600)
Failed 6/14 subtests
./body_chunked.t .......................
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/11 subtests
./charset.t ............................
Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/8 subtests
./charset_gzip_static.t ................ skipped: no gzip_static available
./config_dump.t ........................ ok
./dav.t ................................
Dubious, test returned 14 (wstat 3584, 0xe00)
Failed 14/16 subtests
./dav_chunked.t ........................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 6/7 subtests
./debug_connection.t ................... skipped: no --with-debug available
./debug_connection_syslog.t ............ skipped: no --with-debug available
./empty_gif.t .......................... ok
./error_log.t .......................... ok
./fastcgi.t ............................ skipped: FCGI not installed
./fastcgi_body.t ....................... skipped: FCGI not installed
./fastcgi_body2.t ...................... skipped: FCGI not installed
./fastcgi_buffering.t .................. skipped: FCGI not installed
./fastcgi_cache.t ...................... skipped: FCGI not installed

./fastcgi_header_params.t .............. skipped: FCGI not installed
./fastcgi_keepalive.t .................. ok
./fastcgi_merge_params.t ............... skipped: FCGI not installed
./fastcgi_merge_params2.t .............. skipped: FCGI not installed
./fastcgi_request_buffering.t .......... skipped: FCGI not installed
./fastcgi_request_buffering_chunked.t .. skipped: FCGI not installed
./fastcgi_split.t ...................... skipped: FCGI not installed
./fastcgi_variables.t .................. skipped: FCGI not installed
./geo.t ................................ skipped: no 127.0.0.1 on host
./gunzip.t ............................. skipped: no gzip_static available
./gunzip_memcached.t ................... skipped: Cache::Memcached not installed
./gunzip_perl.t ........................ ok
./gunzip_ssi.t ......................... skipped: no gzip_static available
./gunzip_static.t ...................... skipped: no gzip_static available
./gzip.t ...............................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/9 subtests
./gzip_flush.t ......................... ok
./h2.t ................................. skipped: IO::Socket::SSL not installed
./headers.t ............................
Dubious, test returned 16 (wstat 4096, 0x1000)
Failed 16/26 subtests
./http_disable_symlinks.t .............. skipped: no external file found
./http_error_page.t ....................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/8 subtests
./http_expect_100_continue.t ........... ok
./http_host.t .......................... ok
./http_location.t ...................... ok
./http_location_auto.t ................. ok
./http_location_win32.t ................ skipped: not win32
./http_resolver.t ...................... ok
./http_resolver_aaaa.t ................. skipped: no ipv6 available
./http_server_name.t ...................
Dubious, test returned 19 (wstat 4864, 0x1300)
Failed 19/21 subtests
./http_try_files.t .....................
Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/11 subtests
./http_variables.t ..................... ok
./image_filter.t ....................... skipped: GD not installed
./image_filter_finalize.t ..............
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/4 subtests
./index.t ..............................
Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/8 subtests
./js.t ................................. skipped: no njs available
./limit_conn.t .........................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests

Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/8 subtests
./js.t ................................. skipped: no njs available
./limit_conn.t .........................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests
./limit_conn_complex.t ................. ok
./limit_req.t ..........................
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/6 subtests
./mail_error_log.t ..................... ok
./mail_imap.t .......................... ok
./mail_imap_ssl.t ...................... skipped: IO::Socket::SSL not installed
./mail_pop3.t .......................... ok
./mail_resolver.t ...................... ok
./mail_smtp.t .......................... ok
./mail_smtp_greeting_delay.t ........... ok
./mail_smtp_xclient.t .................. ok
./map.t ................................ ok
./memcached.t .......................... skipped: Cache::Memcached not installed
./memcached_fake.t .....................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/4 subtests
./memcached_keepalive.t ................ skipped: Cache::Memcached not installed
./memcached_keepalive_stale.t .......... skipped: Cache::Memcached not installed
./mp4.t ................................ skipped: no mp4 available
./not_modified.t .......................
Dubious, test returned 13 (wstat 3328, 0xd00)
Failed 13/16 subtests
(less 2 skipped subtests: 1 okay)
./not_modified_proxy.t .................
Dubious, test returned 12 (wstat 3072, 0xc00)
Failed 12/13 subtests
./perl.t ............................... ok
./perl_gzip.t .......................... ok
./post_action.t ........................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/6 subtests
./proxy.t .............................. ok
./proxy_bind.t .........................
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/5 subtests
./proxy_cache.t ........................
Dubious, test returned 11 (wstat 2816, 0xb00)
Failed 11/14 subtests
./proxy_cache_chunked.t ................
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/3 subtests
./proxy_cache_convert_head.t ...........

./range_charset.t ......................
Dubious, test returned 10 (wstat 2560, 0xa00)
Failed 10/11 subtests
./range_flv.t ..........................
Dubious, test returned 12 (wstat 3072, 0xc00)
Failed 12/13 subtests
./range_if_range.t .....................
Dubious, test returned 7 (wstat 1792, 0x700)
Failed 7/9 subtests
./range_mp4.t .......................... skipped: no mp4 available
./realip.t ............................. skipped: no 127.0.0.1 on host
./realip_remote_addr.t ................. skipped: no 127.0.0.1 on host
./referer.t ............................ ok
./rewrite.t ............................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/23 subtests
./rewrite_set.t ........................ ok
./rewrite_unescape.t ................... ok
./scgi.t ............................... skipped: SCGI not installed
./scgi_body.t .......................... skipped: SCGI not installed
./scgi_cache.t ......................... skipped: SCGI not installed
./scgi_gzip.t .......................... skipped: SCGI not installed
./scgi_merge_params.t .................. skipped: SCGI not installed
./secure_link.t ........................
Dubious, test returned 6 (wstat 1536, 0x600)
Failed 6/11 subtests
./spdy.t ............................... skipped: no spdy available
./split_clients.t ......................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests
./ssi.t ................................
Dubious, test returned 19 (wstat 4864, 0x1300)
Failed 19/28 subtests
./ssi_if.t .............................
Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run
./ssi_include_big.t ....................
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/9 subtests
./ssi_waited.t .........................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests
./ssl.t ................................ skipped: IO::Socket::SSL not installed
./ssl_certificate_chain.t .............. skipped: IO::Socket::SSL not installed
./ssl_engine_keys.t .................... skipped: may not work, leaves coredump
./ssl_password_file.t .................. skipped: IO::Socket::SSL not installed
./ssl_proxy_protocol.t ................. skipped: IO::Socket::SSL not installed
./ssl_proxy_upgrade.t .................. skipped: IO::Socket::SSL not installed
./ssl_sni.t ............................ skipped: IO::Socket::SSL version >= 1.56 required

Test Summary Report

./access_log.t (Wstat: 512 Tests: 8 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 10 tests but ran 8.
./auth_basic.t (Wstat: 3584 Tests: 22 Failed: 14)
Failed tests: 2, 4, 6-10, 15-21
Non-zero exit status: 14
./auth_request.t (Wstat: 2048 Tests: 20 Failed: 8)
Failed tests: 1, 5-6, 8, 12, 15-17
Non-zero exit status: 8
./auth_request_satisfy.t (Wstat: 1536 Tests: 19 Failed: 6)
Failed tests: 1, 10-13, 16
Non-zero exit status: 6
./auth_request_set.t (Wstat: 1024 Tests: 7 Failed: 4)
Failed tests: 1-4
Non-zero exit status: 4
./autoindex.t (Wstat: 3840 Tests: 17 Failed: 15)
Failed tests: 1-4, 6-16
Non-zero exit status: 15
./autoindex_format.t (Wstat: 8448 Tests: 38 Failed: 33)
Failed tests: 1-8, 10, 12-24, 26-27, 29-37
Non-zero exit status: 33
./body.t (Wstat: 1536 Tests: 14 Failed: 6)
Failed tests: 4-5, 7-8, 11-12
Non-zero exit status: 6
./body_chunked.t (Wstat: 1024 Tests: 11 Failed: 4)
Failed tests: 3-4, 6-7
Non-zero exit status: 4
./charset.t (Wstat: 1280 Tests: 8 Failed: 5)
Failed tests: 2-4, 6-7
Non-zero exit status: 5
./dav.t (Wstat: 3584 Tests: 16 Failed: 14)
Failed tests: 1-7, 9-15
Non-zero exit status: 14
./dav_chunked.t (Wstat: 512 Tests: 2 Failed: 1)
Failed test: 1
Non-zero exit status: 2
Parse errors: Bad plan. You planned 7 tests but ran 2.
./gzip.t (Wstat: 512 Tests: 9 Failed: 2)
Failed tests: 2, 4
Non-zero exit status: 2
./headers.t (Wstat: 4096 Tests: 26 Failed: 16)
Failed tests: 1-2, 7-8, 10-16, 18-22
Non-zero exit status: 16
./http_error_page.t (Wstat: 256 Tests: 8 Failed: 1)
Failed test: 6
Non-zero exit status: 1
./http_server_name.t (Wstat: 4864 Tests: 21 Failed: 19)
Failed tests: 1, 3-20
Non-zero exit status: 19

Memory issue with code base of Feb24

Hi,

I just ran Nginx+ with the code base of Feb24 on my backup machine. It just listen on port 80 for one IP and not in production. Now my monitoring system told me about full memory and indeed, all my 64GB RAM were gone and nginx restarted itself:

2017/03/02 14:29:49 [notice] 738#738: signal 17 (SIGCHLD) received
2017/03/02 14:29:49 [alert] 738#738: worker process 22623 exited on signal 9
2017/03/02 14:29:49 [notice] 738#738: start worker process 26684
2017/03/02 14:29:49 [notice] 738#738: signal 29 (SIGIO) received
2017/03/02 14:29:49 [notice] 738#738: signal 17 (SIGCHLD) received
2017/03/02 14:29:49 [alert] 738#738: worker process 22624 exited on signal 9
2017/03/02 14:29:49 [alert] 738#738: worker process 22625 exited on signal 9
2017/03/02 14:29:49 [alert] 738#738: worker process 22627 exited on signal 9
2017/03/02 14:29:49 [notice] 738#738: start worker process 26685
2017/03/02 14:29:49 [notice] 738#738: start worker process 26686
2017/03/02 14:29:49 [notice] 738#738: start worker process 26687
2017/03/02 14:29:49 [notice] 738#738: signal 29 (SIGIO) received
2017/03/02 14:29:51 [notice] 738#738: signal 17 (SIGCHLD) received
2017/03/02 14:29:51 [alert] 738#738: worker process 22643 exited on signal 9
2017/03/02 14:29:51 [notice] 738#738: start worker process 26688
2017/03/02 14:29:51 [notice] 738#738: signal 29 (SIGIO) received

I grabbed the logs of nginx but the last connect was at 11:39:02.

I had the same issue two days ago after benchmarking with "ab". I thought this was the cause so I restarted nginx, did a benchmark again and waited. Now after two days I have the same memory error again.

Sadly I cannot reproduce it.

Someone running the same code base and seeing this issue?

ModSecurity-nginx support SecAuditLogFormat JSON ?

ModSecurity-nginx support SecAuditLogFormat JSON ?

ModSecurity configure

./configure --with-yajl=../lloyd-yajl-66cb08c/build/yajl-2.1.0/

ModSecurity - v3.0.0+b58f713 for Linux

Mandatory dependencies

  • libInjection ....v2.9.0-799-gb58f713
  • SecLang tests ....b58f713

Optional dependencies

  • GeoIP ....not found
  • LibCURL ....not found
  • YAJL ....found
    -lyajl, -DWITH_YAJL -I../lloyd-yajl-66cb08c/build/yajl-2.1.0//include
  • LMDB ....not found
  • LibXML2 ....found v2.9.1
    -lxml2 -lz -lm -ldl, -I/usr/include/libxml2 -DWITH_LIBXML2

Other Options

  • Test Utilities ....enabled
  • SecDebugLog ....enabled
  • afl fuzzer ....disabled
  • library examples ....enabled
  • Building parser ....disabled

nginx.conf

modsecurity on;
modsecurity_rules_file /usr/local/nginx/conf/modsecurity.conf

modsecurity.conf

SecAuditLogFormat JSON

nginx start error

nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /usr/local/nginx/conf/modsecurity.conf. Line: 256. Column: 23. Invalid input: SecAuditLogFormat JSON in /usr/local/nginx/conf/nginx.conf:118

Website Firewall Not Found on Sucuri SiteCheck test

Hello,
I have used Compilation recipes at https://github.com/SpiderLabs/ModSecurity/wiki/Compilation-recipes

but still I can see Website Firewall Not Found

sucuri

My actions step by step on CentOS 7

NGINX

yum install gcc-c++ flex bison yajl yajl-devel curl-devel curl GeoIP-devel doxygen zlib-devel libtool lmdb-devel -y

yum install ftp://195.220.108.108/linux/fedora/linux/updates/23/x86_64/b/bison-3.0.4-3.fc23.x86_64.rpm -y

cd /usr/local/src/
wget http://nginx.org/download/nginx-1.11.9.tar.gz
tar zxvf nginx-1.11.9.tar.gz
mv /usr/local/src/nginx-1.11.9 /usr/local/src/nginx

wget https://www.openssl.org/source/openssl-1.0.2k.tar.gz
tar -zxf openssl-1.0.2k.tar.gz
mv /usr/local/src/openssl-1.0.2k /usr/local/src/nginx_modules/openssl

curl -L -O http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
tar zxvf ./ngx_cache_purge-2.3.tar.gz
mv /usr/local/src/ngx_cache_purge-2.3 /usr/local/src/nginx_modules/ngx_cache_purge

curl -L -O http://hg.nginx.org/njs/archive/tip.tar.gz
tar zxvf ./tip.tar.gz
mv /usr/local/src/njs-214afa2466a0 /usr/local/src/nginx_modules/njs

git clone https://github.com/google/ngx_brotli.git
mv /usr/local/src/ngx_brotli /usr/local/src/nginx_modules/ngx_brotli
cd /usr/local/src/nginx_modules/ngx_brotli && git submodule update --init && cd /usr/local/src/nginx

bash <(curl -f -L -sS https://ngxpagespeed.com/install) \

mv /userme/ngx_pagespeed-latest-stable /usr/local/src/nginx_modules/ngx_pagespeed

================= libModSecurity ===============

cd /usr/local/src/nginx_modules/
git clone https://github.com/SpiderLabs/ModSecurity

cd /usr/local/src/nginx_modules/ModSecurity
git checkout -b v3/master origin/v3/master
sh build.sh
git submodule init
git submodule update

./configure

make
make install

================= nginx connector =================

export MODSECURITY_INC="/usr/local/src/nginx_modules/ModSecurity/headers/"
export MODSECURITY_LIB="/usr/local/src/nginx_modules/ModSecurity/src/.libs/"

cd /usr/local/src/
git clone https://github.com/SpiderLabs/ModSecurity-nginx
mv /usr/local/src/ModSecurity-nginx /usr/local/src/nginx_modules/ModSecurity-nginx

cd /usr/local/src/nginx_modules/ModSecurity-nginx
git checkout -b experimental origin/experimental

================= nginx ==========================

cd /usr/local/src/nginx
/bin/cp -f /usr/sbin/nginx /usr/sbin/nginx_original_bkp

./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --add-dynamic-module=/usr/local/src/nginx_modules/njs/nginx --with-openssl=/usr/local/src/nginx_modules/openssl --add-module=/usr/local/src/nginx_modules/ngx_cache_purge --add-module=/usr/local/src/nginx_modules/ngx_brotli --add-module=/usr/local/src/nginx_modules/ngx_pagespeed --add-module=/usr/local/src/nginx_modules/ModSecurity-nginx

make
make install

================= nginx_host.conf =======================

And use this directive rules

	modsecurity on;
	location /	{
	modsecurity_rules '
	SecRuleEngine On
	SecDebugLog /tmp/modsec_debug.log
	SecDebugLogLevel 9
	SecRule ARGS "@contains test" "id:1,phase:2,t:trim,block"
  ';
	
	try_files	$uri	$uri/	/index.php?$args;
				}

Invalid input: SecDisableBackendCompression On

SecDisableBackendCompression directive doesn't work with Libmodsecurity and ModSecurity-nginx.

Here my error :

nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /etc/nginx/modsecurity.conf. Line: 209. Column: 32. Invalid input:  SecDisableBackendCompression On in /etc/nginx/nginx.conf:16

I can't read /var/log/modsec_audit.log file, output is encoding.

header already sent while sending response to client

nginx -V

nginx version: nginx/1.11.3
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --add-dynamic-module=njs-0.1.0/nginx --with-threads --with-stream --with-stream_ssl_module --with-stream_geoip_module=dynamic --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_v2_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic'

nginx.conf

user nginx;
worker_processes auto;
pid /var/run/nginx.pid;

load_module "modules/ngx_http_modsecurity.so";
load_module "modules/ngx_http_image_filter_module.so";

events {
        worker_connections 4096;
        multi_accept on;
}

http {
...

site.com.conf

server {
        listen 80;

        server_name site.com;
        root /var/www/site.com;

        modsecurity on;

        location / {
                modsecurity_rules '
                SecRuleEngine On
                SecDebugLog /tmp/modsec_debug.log
                SecDebugLogLevel 9
                SecRule ARGS "@contains test" "id:1,phase:2,t:trim,block"
                ';
        }

access.log

ip - - [time] "GET /robots.txt HTTP/1.1" 500 0 "-" "user agent"

error.log

time [alert] num: *num header already sent, client: ip, server: site.com, request: "GET /robots.txt HTTP/1.1", host: "site.com"

nginx.spec

        --add-dynamic-module=%{_topdir}/SOURCES/ModSecurity-nginx \

/opt/ModSecurity
git status

# On branch libmodsecurity

git log -n 1

commit 0e5f72977ee1026ae41317881f34c8c290d97b16
Author: Felipe Zimmerle <[email protected]>
Date:   Fri Jul 29 10:40:45 2016 -0700

    Changes MATCHED_VAR behaviour

~/rpmbuild/SOURCES/ModSecurity-nginx
git status

# On branch master

git log -n 1

commit ed57c81f005a539a7334ebdeb8cee87c362d6c26
Author: Alexey Zelkin <[email protected]>
Date:   Tue Jul 12 20:22:28 2016 +0000

    Correct mistake in module name

Issues while using error_page and location

According to the original report (Ehsan Mahdavi) the audit logs are not working for custom error pages (using: error_page or location). In that scenario the logs are not written.

Ehsan Mahdavi also said that in case of SecRuleEngine is set to DetectionOnly everything works like expected.

Best Way to Include modsecurity.conf / OWASP Rules

Hello,

As suggested here, I have now configured libModSecurity (aka v3) and nginx connector on CentOS 7.

I am trying to figure out a neat way of including all parameters (in modsecurity.conf).

This is what is working for me now.

server {
    listen       80 default_server;
    server_name  localhost;

    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsec/modsec_rules_base.conf;

    location / {
        root   /etc/nginx/html;
        index  index.html index.htm;
		modsecurity_rules '
                       # Paramaters of modsecurity.conf
                       # Can I include these in a file somewhere?
			SecRuleEngine On
			SecRequestBodyAccess On
			SecRequestBodyLimit 13107200
			SecRequestBodyNoFilesLimit 131072
			SecRequestBodyInMemoryLimit 131072
			SecRequestBodyLimitAction Reject
			SecPcreMatchLimit 1000
			SecPcreMatchLimitRecursion 1000
			SecResponseBodyAccess On
			SecResponseBodyMimeType text/plain text/html text/xml
			SecResponseBodyLimit 524288
			SecResponseBodyLimitAction ProcessPartial
			SecTmpDir /tmp/
			SecDataDir /tmp/
			SecAuditEngine RelevantOnly
			SecAuditLogRelevantStatus "^(?:5|4(?!04))"
			SecAuditLogParts ABIJDEFHZ
			SecAuditLogType Serial
			SecAuditLog /var/log/nginx/modsec_audit.log
			SecArgumentSeparator &
			SecCookieFormat 0
			SecUnicodeMapFile unicode.mapping 20127
			SecStatusEngine On
			SecDefaultAction "phase:1,deny,log"
  		';
    }
...
...
}
# cat /etc/nginx/modsec/modsec_rules_base.conf
# OWASP CRS v3 rules
Include /usr/local/owasp-modsecurity-crs/crs-setup.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-901-INITIALIZATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-903.9001-DRUPAL-EXCLUSION-RULES.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-903.9002-WORDPRESS-EXCLUSION-RULES.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-905-COMMON-EXCEPTIONS.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-910-IP-REPUTATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-911-METHOD-ENFORCEMENT.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-912-DOS-PROTECTION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-913-SCANNER-DETECTION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-921-PROTOCOL-ATTACK.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-931-APPLICATION-ATTACK-RFI.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-933-APPLICATION-ATTACK-PHP.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-950-DATA-LEAKAGES.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-951-DATA-LEAKAGES-SQL.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-952-DATA-LEAKAGES-JAVA.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-953-DATA-LEAKAGES-PHP.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-954-DATA-LEAKAGES-IIS.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf

It would be great to include a sample file somewhere in the documentation.

SecRuleRemoveByTag

Hi, May I know whether the connector currently accepts SecRuleRemoveByTag?

With the current configuration,

modsecurity on;
modsecurity_rules_file /etc/nginx/modsecurity.conf;
modsecurity_rules '
SecRuleRemoveByTag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"
';

I get this error

# nginx -t -c /etc/nginx/nginx.conf
nginx: [emerg] "modsecurity_rules" directive Rules error. File: <<reference missing or not informed>>. Line: 2. Column: 55. syntax error, unexpected QUOTATION_MARK  in /etc/nginx/nginx.conf:26
nginx: configuration file /etc/nginx/nginx.conf test failed

Line 26 refers to the SecRuleRemoveByTag line, and column 55 is the last character of the line, which happens to be end quote.

Error to run config

Hi,

I'm trying to compile it to use as a dynamic module but I'm getting these errors running ./config:

./config: line 23: [: =: unary operator expected
./config: line 25: [: =: unary operator expected
./config: line 31: auto/feature: No such file or directory
./config: line 33: [: =: unary operator expected
./config: line 64: [: =: unary operator expected

I'm using Ubuntu 14.04. I couldn't find any documentation explaining how to do it.

Cheers,

Nginx Restart with loaded module cause segfaults (maybe problem for ModSec v3 and not connector itself)

Hi Devs, i noticed that latest master of connector (maybe issue not in connector exactly) causing nginx segfaults during its restart.
Reload or Stop then Start do not cause such segfaults.
Also by default files for shared collection dropped to root folder '/'

ls -l / | grep shared
-rw-r--r--   1 root root 1048576 Jan 13 17:03 modsec-shared-collections
-rw-r--r--   1 root root    8192 Jan 13 17:05 modsec-shared-collections-lock

But, during restart Nginx with init/init.d scripts those files created in directory where cli command was used.

Jan 13 17:03:07 abyss kernel: [15515838.077384] nginx[7190]: segfault at 7f9548807000 ip 00007fbe74c5b9da sp 00007ffcde6b9208 error 4
Jan 13 17:03:07 abyss kernel: [15515838.077395] nginx[7192]: segfault at 7f9548807000 ip 00007fbe74c5b9da sp 00007ffcde6b9208 error 4
Jan 13 17:03:07 abyss kernel: [15515838.077398] nginx[7186]: segfault at 7f9548807000 ip 00007fbe74c5b9da sp 00007ffcde6b9208 error 4
Jan 13 17:03:07 abyss kernel: [15515838.077400] nginx[7191]: segfault at 7f9548807000 ip 00007fbe74c5b9da sp 00007ffcde6b9208 error 4
Jan 13 17:03:07 abyss kernel: [15515838.077401]  in libc-2.19.so[7fbe74bd3000+1ba000]
Jan 13 17:03:07 abyss kernel: [15515838.077402] nginx[7188]: segfault at 7f9548807000 ip 00007fbe74c5b9da sp 00007ffcde6b9208 error 4
Jan 13 17:03:07 abyss kernel: [15515838.077403]  in libc-2.19.so[7fbe74bd3000+1ba000]
Jan 13 17:03:07 abyss kernel: [15515838.077404]
Jan 13 17:03:07 abyss kernel: [15515838.077405]  in libc-2.19.so[7fbe74bd3000+1ba000]
Jan 13 17:03:07 abyss kernel: [15515838.077405]
Jan 13 17:03:07 abyss kernel: [15515838.077406]
Jan 13 17:03:07 abyss kernel: [15515838.077407]  in libc-2.19.so[7fbe74bd3000+1ba000]
Jan 13 17:03:07 abyss kernel: [15515838.077414]  in libc-2.19.so[7fbe74bd3000+1ba000]
Jan 13 17:03:07 abyss kernel: [15515838.080151] nginx[7185]: segfault at 7f9548807000 ip 00007fbe74c5b9da sp 00007ffcde6b92e8 error 4 in libc-2.19.so[7fbe74bd3000+1ba000]




Jan 13 17:03:24 abyss kernel: [15515854.689932] nginx[7689]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954ecc8 error 4
Jan 13 17:03:24 abyss kernel: [15515854.689934] nginx[7687]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954ecc8 error 4 in libc-2.19.so[7fea863e5000+1ba000]
Jan 13 17:03:24 abyss kernel: [15515854.689936]  in libc-2.19.so[7fea863e5000+1ba000]
Jan 13 17:03:24 abyss kernel: [15515854.689960] nginx[7686]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954ecc8 error 4 in libc-2.19.so[7fea863e5000+1ba000]
Jan 13 17:03:24 abyss kernel: [15515854.689978] nginx[7690]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954ecc8 error 4 in libc-2.19.so[7fea863e5000+1ba000]
Jan 13 17:03:24 abyss kernel: [15515854.689983] nginx[7691]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954ecc8 error 4 in libc-2.19.so[7fea863e5000+1ba000]
Jan 13 17:03:24 abyss kernel: [15515854.690068] nginx[7688]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954ecc8 error 4 in libc-2.19.so[7fea863e5000+1ba000]
Jan 13 17:03:24 abyss kernel: [15515854.692350] nginx[7685]: segfault at 7fc7b8ed5000 ip 00007fea8646d9da sp 00007ffe7954eda8 error 4 in libc-2.19.so[7fea863e5000+1ba000]

All messages produced by libmodsecurity are being printed with the "info" level

There is no way to control error log level for messages from libmodsecurity: all messages appear at the "info" level.

Probably there should be a way to set desired log level in nginx configuration.

Perhaps it's also worth to put some important messages, like those produced by "disruptive" actions, at higher level ("warn"?).

UPDATE: elaborating on thoughts of having different log levels for different actions, I'd like to demonstrate how it would look like, with the help of syslog "severities":

       Numerical         Severity
         Code

          0       Emergency: system is unusable
          1       Alert: action must be taken immediately
          2       Critical: critical conditions
          3       Error: error conditions
          4       Warning: warning conditions
          5       Notice: normal but significant condition
          6       Informational: informational messages
          7       Debug: debug-level messages

          Table 2. Syslog Message Severities

From https://tools.ietf.org/html/rfc5424#section-6.2.1

So, in terms of libmodsecurity it may be something like:

action                          severity
rule1 matched some pattern      6
rule2 matched some pattern      6
...
rule3 matched some pattern      6
anomaly score > threshold       5
request is blocked              4
some internal error             3

A "log severity" could be passed as an additional parameter from logger function in libmodsecurity to configured logger callback, where it may be used to determine which log level should be used to log a particular message.

From my understanding, this could be useful for other connectors as well (Apache, IIS, etc).

tests failing

The libmodsecurity tests pass, and the tests from http://hg.nginx.org/nginx-tests/ pass, but the ModSecurtiy-nginx tests fail for me.

I know the product is in alpha (and I am using the experimental branch), but when I run the tests I get the following results:

$ prove modsecurity*
modsecurity-config-auditlog.t .. nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRule ARGS "@Streq root" "id:1,phase:1,auditlog,status:302,redirect:http://www.modsecurity.org"
SecDebugLog /tmp/nginx-test-_hnbNFuBXw/auditlog-debug-root.txt
SecDebugLogLevel 9
SecAuditLogParts AB
SecAuditLog /tmp/nginx-test-_hnbNFuBXw/auditlog-root.txt
SecAuditLogType Serial
SecAuditLogStorageDir /tmp/nginx-test-_hnbNFuBXw/
. Line: 2. Column: 455. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-_hnbNFuBXw/nginx.conf:91
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity-config-auditlog.t .. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
modsecurity-config-debuglog.t .. nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRule ARGS "@Streq root" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
SecDebugLog /tmp/nginx-test-W5Hd5Xk3DI/debuglog-root.txt
SecDebugLogLevel 9
. Line: 2. Column: 226. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-W5Hd5Xk3DI/nginx.conf:55
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity-config-debuglog.t .. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
modsecurity-config.t ........... nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRule ARGS "@Streq root" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
. Line: 2. Column: 118. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-Fm9IIs6mvi/nginx.conf:49
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity-config.t ........... Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
modsecurity-proxy.t ............ nginx: [emerg] "proxy_read_timeout" directive Rules error. File:
SecRuleEngine On
SecRule ARGS "@Streq redirect301" "id:1,phase:1,status:301,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq redirect302" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq block401" "id:11,phase:1,status:401,block"
SecRule ARGS "@Streq block403" "id:11,phase:1,status:403,block"
. Line: 3. Column: 397. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-ndF1spIzZj/nginx.conf:79
Can't start nginx at lib/Test/Nginx.pm line 274.
(in cleanup) Can't open error.log: No such file or directory at lib/Test/Nginx.pm line 378.

Looks like your test exited with 2 before it could output anything.

modsecurity-proxy.t ............ Dubious, test returned 2 (wstat 512, 0x200)
Failed 24/24 subtests
modsecurity.t .................. nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRuleEngine On
SecRule ARGS "@Streq redirect301" "id:1,phase:1,status:301,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq redirect302" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq block401" "id:11,phase:1,status:401,block"
SecRule ARGS "@Streq block403" "id:11,phase:1,status:403,block"
. Line: 3. Column: 397. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-IRILiSUeGJ/nginx.conf:74
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity.t .................. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run

Test Summary Report

modsecurity-config-auditlog.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
modsecurity-config-debuglog.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
modsecurity-config.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
modsecurity-proxy.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 24 tests but ran 0.
modsecurity.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
Files=5, Tests=0, 2 wallclock secs ( 0.03 usr 0.00 sys + 0.21 cusr 0.07 csys = 0.31 CPU)
Result: FAIL

SELinux & modsecurity-nginx

Hello,

Hoping you are able to shed a little light on some difficulty I'm experiencing currently with SELinux & ModSecurity-nginx:

type=AVC msg=audit(1489419584.341:40788): avc: denied { execmem } for pid=37367 comm="nginx" scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:system_r:httpd_t:s0 tclass=process

Which, whilst it is a straightforward boolean adjustment required for the httpd_execmem context, I do not wish to enable this without justification.

Which part of ModSecurity-nginx module is requiring this feature? As this is preventing nginx to start, whilst ModSecurity is enabled:

"2017/03/13 15:39:44 [emerg] 37367#37367: "log_format" directive Failed to allocate shared memory (1): Permission denied in /etc/nginx/nginx.conf:48"

^ as an example error log entry as being denied by SELinux.

The reason is not static to "log_format" this changes, but does not occur when modsecurity is not enabled. I hadn't had this issue when using the nginx_refactoring branch of Modsecurity 2.9.

Any help / information would be useful!

For verbosity, the versions being used are as follows:

Master branch of ModSecurity-nginx,
v3/Master for libmodsecurity,
nginx version: nginx/1.10.3
CentOS Linux release 7.3.1611 (Core)

We can create a custom policy to resolve this, but I would like to know what is causing this so I can justify the change,

Thanks,

David

RESPONSE_BODY in ModSecurity V3 +Nginx doesn't work

@igge2015 commented on Tue Aug 01 2017

RESPONSE_BODY in ModSecurity V3 + Nginx doesn't work
Audit Log:
---wUYho2Gz---D--

---wUYho2Gz---F--
HTTP/1.1 304
Server: nginx
Date: Wed, 02 Aug 2017 02:21:10 GMT
Keep-Alive: timeout=60
Connection: keep-alive
ETag: "3-555a579fb4dfb"

---wUYho2Gz---H--

---wUYho2Gz---I--

---wUYho2Gz---J--

---wUYho2Gz---Z--

The conf:
modsecurity on;
modsecurity_rules '
SecRuleEngine On
SecResponseBodyAccess On
SecResponseBodyLimit 128
SecRule RESPONSE_BODY "@contains php" "id:11,phase:response,deny,log,status:403"
SecRuleEngine On
SecDebugLog /tmp/modsec_debug.log
SecDebugLogLevel 9
SecRuleRemoveById 10
SecAuditEngine On
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecAuditLog /var/log/modsec_audit.log
';

Debug log:
[4] Initializing transaction
[4] Transaction context created.
[4] Starting phase CONNECTION. (SecRules 0)
[9] This phase consists of 0 rule(s).
[4] Starting phase URI. (SecRules 0 + 1/2)
[4] Starting phase REQUEST_HEADERS. (SecRules 1)
[9] This phase consists of 1 rule(s).
[4] (Rule: 999323) Executing operator "Contains" with param "1" against RESPONSE_BODY.
[9] Target value: "" (Variable: RESPONSE_BODY)
[4] Rule returned 0.
[9] Matched vars cleaned.
[4] Starting phase REQUEST_BODY. (SecRules 2)
[4] Request body processing is disabled
[9] This phase consists of 0 rule(s).
[4] Starting phase RESPONSE_HEADERS. (SecRules 3)
[9] This phase consists of 0 rule(s).
[9] Appending response body: 0 bytes. Limit set to: 0.000000
[4] Starting phase RESPONSE_BODY. (SecRules 4)
[9] This phase consists of 0 rule(s).
[4] Starting phase LOGGING. (SecRules 5)
[9] This phase consists of 0 rule(s).
[8] Checking if this request is suitable to be saved as an audit log.
[8] Checking if this request is relevant to be part of the audit logs.
[5] Saving this request as part of the audit logs.
[8] Request was relevant to be saved. Parts: 6014
[4] Initializing transaction
[4] Transaction context created.
[4] Starting phase CONNECTION. (SecRules 0)
[9] This phase consists of 0 rule(s).
[4] Starting phase URI. (SecRules 0 + 1/2)
[4] Starting phase REQUEST_HEADERS. (SecRules 1)
[9] This phase consists of 0 rule(s).
[4] Starting phase REQUEST_BODY. (SecRules 2)
[4] Request body processing is disabled
[9] This phase consists of 0 rule(s).
[4] Starting phase RESPONSE_HEADERS. (SecRules 3)
[9] This phase consists of 0 rule(s).
[9] Appending response body: 0 bytes. Limit set to: 128.000000
[4] Starting phase RESPONSE_BODY. (SecRules 4)
[9] This phase consists of 1 rule(s).
[4] (Rule: 11) Executing operator "Contains" with param "php" against RESPONSE_BODY.
[9] Target value: "" (Variable: RESPONSE_BODY)
[4] Rule returned 0.
[9] Matched vars cleaned.
[4] Starting phase LOGGING. (SecRules 5)
[9] This phase consists of 0 rule(s).
[8] Checking if this request is suitable to be saved as an audit log.
[8] Checking if this request is relevant to be part of the audit logs.
[5] Saving this request as part of the audit logs.
[8] Request was relevant to be saved. Parts: 6006

The HTML Text
php


@igge2015 commented on Tue Aug 01 2017

It doesn't work.


@lkarsten commented on Wed Aug 02 2017

This has not been my experience, although I agree that debugging is not straight forward.

I've written (modified) a test case for libmodsecurity that test this, and as far as I can tell the library itself does what I expect it to.

Anecdotally, an extensive set of 403 Forbidden logged for all directory listings when I have CRS3 loaded on my modsecurity-nginx enabled host also indicate that response body matching is working.

Test case:

[
  {
    "enabled":1,
    "version_min":300000,
    "title":"Testing Variables :: RESPONSE_BODY",
    "client": {
      "ip":"200.249.12.31"
    },
    "request": {
      "headers": {
        "Host":"localhost"
      },
      "uri":"/foo",
      "http_version":1.1
    },
    "response":{
      "headers":{
        "Date":"Mon, 13 Jul 2015 20:02:41 GMT",
        "Last-Modified":"Sun, 26 Oct 2014 22:33:37 GMT",
        "Content-Type":"text/plain"
      },
      "body":[
        "denystring"
      ]
    },
    "expected":{
      "http_code": 403
    },
    "rules":[
      "SecRuleEngine On",
      "SecRule RESPONSE_BODY \"@contains denystring\" \"id:1,phase:4,deny\""
    ]
  }
]

Log output:

$ ./regression_tests test-cases/regression/variable-RESPONSE_BODY.json
ModSecurity 3.0.0-alpha - tests
(options are not available -- missing GetOpt)

  # File Name                                         Test Name                                                             Passed?   
--- ---------                                         ---------                                                             -------   
  1 variable-RESPONSE_BODY.json                       Testing Variables :: RESPONSE_BODY                                    passed!

Ran a total of: 1 regression tests - All tests passed.

@igge2015 commented on Wed Aug 02 2017

Can you give me your nginx.conf so that I can take it as an example?


@lkarsten commented on Wed Aug 02 2017

Sorry, I can't share that verbatim. In /etc/nginx/sites-enabled/example.com I have:

modsecurity on;
modsecurity_rules_file /opt/nginxmodsec/conf/modsecurity.conf;
modsecurity_rules_file /opt/nginxmodsec/conf/crs-setup.conf;
modsecurity_rules_file /opt/nginxmodsec/conf/rules/all.conf;

all.conf is cat rules/*conf > all.conf since I couldn't get the parser to accept a directory. This is for CRS3 with some modifications. libmodsecurity from head of v3/master, modsecurity-nginx from head of master as well.


@igge2015 commented on Thu Aug 03 2017

What's your Nginx Version?


@lkarsten commented on Thu Aug 03 2017

@igge2015 I'm running nginx 1.13.2.


@igge2015 commented on Thu Aug 03 2017

@lkarsten Does your auditlog have response_body? There's no response_body. in my auditlog


@lkarsten commented on Thu Aug 03 2017

@igge2015 That is out of scope for this issue. See #1519 for some background.


@victorhora commented on Sun Aug 06 2017

Hi @igge2015,

I have loaded your configuration on the latest code from master for libModSecurity + nginx Connector and this is what I get on my debug logs:

[4] Initializing transaction
[4] Transaction context created.
[4] Starting phase CONNECTION. (SecRules 0)
[9] This phase consists of 0 rule(s).
[4] Starting phase URI. (SecRules 0 + 1/2)
[4] Starting phase REQUEST_HEADERS.  (SecRules 1)
[9] This phase consists of 0 rule(s).
[4] Starting phase REQUEST_BODY. (SecRules 2)
[4] Request body processing is disabled
[9] This phase consists of 0 rule(s).
[4] Starting phase RESPONSE_HEADERS. (SecRules 3)
[9] This phase consists of 0 rule(s).
[9] Appending response body: 6 bytes. Limit set to: 0.000000
[4] Starting phase RESPONSE_BODY. (SecRules 4)
[9] This phase consists of 1 rule(s).
[4] (Rule: 11) Executing operator "Contains" with param "php" against RESPONSE_BODY.
[9] Target value: "\x0a" (Variable: RESPONSE_BODY)
[4] Matched vars updated.
[4] Running [I] (_non_ disruptive) action: log
[9] Saving transaction to logs
[4] Rule returned 1.
[4] Running (disruptive) action: deny
[8] Running action deny
[4] Running [I] (_non_ disruptive) action: status
[8] Skipping this phase as this request was already intercepted.

The logs suggests that the rule have triggered and indeed I get a disruptive action (although not so sure if all is right here).

But I find interesting that your debug log entry in regards to buffered data on response body:

[9] Appending response body: 0 bytes. Limit set to: 128.000000
[4] Starting phase RESPONSE_BODY. (SecRules 4)
[9] This phase consists of 1 rule(s).
[4] (Rule: 11) Executing operator "Contains" with param "php" against RESPONSE_BODY.
[9] Target value: "" (Variable: RESPONSE_BODY)
[4] Rule returned 0.

I wonder why your response body is appended with 0 bytes. Maybe that's why ModSecurity can't see any data on the server response to match on.

Can you make sure you're running the latest code from master? Can you provide further detailed reproduction steps? Like: "Saving a html file with x content server and then running wget to trigger a server response"

Also, maybe a good idea to increase SecResponseBodyLimit. If I'm not mistaken you're setting that to 128 bytes which might too little for any normal PHP script, unless you're matching against a very small subset of test data.


@theMiddleBlue commented on Wed Aug 09 2017

Hi,

same problem for my configuration (nginx+libmodsecurity). I've created a Rule like the following:

SecRule RESPONSE_BODY "my password is iloveyou" \
        "phase:response,\
        block,\
        msg:'Leakage',\
        id:9001"

Reading the modsec_debug.log, it seems that the RESPONSE_BODY is empty...

[4] (Rule: 9001) Executing operator "Rx" with param "my password is iloveyou" against RESPONSE_BODY.
[9] Target value: "" (Variable: RESPONSE_BODY)
[4] Rule returned 0.
[4] Matched vars cleaned.

I don't know if I'm doing something wrong...


@friparia commented on Thu Aug 10 2017

same problem with me. The configuration is as follows:

        modsecurity on;
        modsecurity_rules_file /etc/nginx/modsecurity/modsec_includes.conf;
        location / {
            modsecurity_rules 'SecRule RESPONSE_BODY "2017" "phase:4,id:1000001,t:none,deny"';
            proxy_pass http://10.3.19.25;
            proxy_set_header X-Forwarded-For $remote_addr;
            proxy_set_header REMOTE_ADDR $remote_addr;
            proxy_set_header HOST $host;
        }

@friparia commented on Fri Aug 11 2017

Hey guys, i found the reason, let's see the code in this line

However, when using proxies the code cannot define it fully loaded, the mod security code will not be executed. So we delete the code of checking buffer_fully_loaded to fix the problem. But we don't know the purpose of buffer_fully_loaded code, so we don't know the side effect of this fixture.

The fixture repo is here


@theMiddleBlue commented on Fri Aug 11 2017

😭


@igge2015 commented on Sat Aug 12 2017

@friparia Thank you.But there is still an error.When I hit the rule,there is a error log like

2017/08/12 18:09:43 [alert] 41571#41571: *11 header already sent while sending to client, client: 117.82.45.62, server: dev4.08tk.cn, request: "GET /1.php HTTP/1.1", upstream: "http://175.6.244.139:8080/1.php", host: "dev4.08tk.cn"

Need help on Logging and Testing in Modsecurity-nginx + Libmodsecurity

Hi,

I have compiled libmodsecurity with ModSecurity-Nginx connector over Nginx-1.10.3 correctly without any issues. Now im looking at usage example given in wiki of the Modsecurity-nginx project. How do i add logging and do actual tests if the modsecurity is working or not in this setup ? Because im not seeing any difference in requests so far, so any info on how to add logs for this, May be an usage example would be great.

AuditLog does not contains msg

Hi,

i've the following modsecurity configuration:

SecAuditLogType Concurrent
SecAuditLogStorageDir /usr/local/openresty/nginx/logs/modsecurity/

and all audit logs seems to be without messages and tags:

{
    "transaction": {
        "client_ip": "1.2.3.4",
        "time_stamp": "Sat Dec  3 22:36:29 2016",
        "server_id": "4d3b7d07f855b347f9fb2799eef4b6ca7cee51c9",
        "client_port": 57030,
        "host_ip": "1.2.3.4",
        "host_port": 80,
        "id": "148080458934.713966",
        "request": {
            "protocol": "GET",
            "http_version": 1.1,
            "uri": "/?a=<script>alert(123);</script>",
            "headers": {
                "REQUEST_HEADERS:Host": "xy",
                "REQUEST_HEADERS:User-Agent": "curl/7.49.1",
                "REQUEST_HEADERS:Accept": "*/*"
            }
        },
        "response": {
            "http_code": 403,
            "headers": {
                "RESPONSE_HEADERS:Connection": "keep-alive",
                "RESPONSE_HEADERS:Server": "openresty/1.11.2.2",
                "RESPONSE_HEADERS:Date": "Sat, 03 Dec 2016 22:36:29 GMT",
                "RESPONSE_HEADERS:Content-Length": "175",
                "RESPONSE_HEADERS:Content-Type": "text/html"
            }
        },
        "producer": {
            "modsecurity": "ModSecurity v3.0.0-alpha (Linux)",
            "connector": "ModSecurity-nginx v0.1.1-beta",
            "secrules_engine": "Enabled",
            "components": [
                "OWASP_CRS/3.0.0\""
            ]
        },
        "messages": [

        ]
    }
}

is possible to configure the native audit log file? (not JSON)

thank you!

nginx -t failing for higher number of domains

I am testing ModSecurity-nginx and i have

server{
modsecurity on;
location / {
modsecurity_rules_file /etc/nginx/conf.d/zz_modsecurity.conf;
}
}

Towards the end of /etc/nginx/conf.d/zz_modsecurity.conf file i added

# Include OWASP crs core ruleset
Include /etc/nginx/owasp-modsecurity-crs/crs-setup.conf
Include /etc/nginx/owasp-modsecurity-crs/rules/*.conf

I am not sure if this is how it should be done . Because this works fine for a few domains (3-4 nos). But when we add say like 30 domains.

nginx -t itself is failing and taking long time to complete and consume all memory on low memory server

# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: [alert] mmap(MAP_ANON|MAP_SHARED, 335544320) failed (12: Cannot allocate memory)
nginx: configuration file /etc/nginx/nginx.conf test failed

What am I doing wrong? . If I am doing everything fine then I believe the large file inclusion is making it harder for config test and loading of config file for nginx . even with somthing like 30-40 domains added.

compilation failure

Hi guys,

Using the nginx source : https://nginx.org/download/nginx-1.12.1.tar.gz as well as the current master in ModSecurity-nginx, I'm currently experiencing the following compilation error;

/root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_module.c:588:5: error: too many arguments to function 'msc_rules_merge'
     rules = msc_rules_merge(c->rules_set, p->rules_set, &error);
     ^
In file included from /root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_common.h:27:0,
                 from /root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_module.c:21:
/usr/include/modsecurity/rules.h:102:5: note: declared here
 int msc_rules_merge(Rules *rules_dst, Rules *rules_from);
     ^
/root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_module.c: In function 'ngx_http_modsecurity_merge_loc_conf':
/root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_module.c:630:5: error: too many arguments to function 'msc_rules_merge'
     rules = msc_rules_merge(c->rules_set, p->rules_set, &error);
     ^
In file included from /root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_common.h:27:0,
                 from /root/rpmbuild/BUILD/ModSecurity-nginx-master/src/ngx_http_modsecurity_module.c:21:
/usr/include/modsecurity/rules.h:102:5: note: declared here
 int msc_rules_merge(Rules *rules_dst, Rules *rules_from);
     ^
make[1]: *** [objs/addon/src/ngx_http_modsecurity_module.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/root/rpmbuild/BUILD/nginx-1.12.0'
make: *** [build] Error 2

I have replicated this with fresh downloads of libmodsecurity, nginx and ModSecurity-nginx on CentOS 7 & Fedora 25.

Any help would be greatly appreciated!

David

assembly problem through rpmbuild on centos

We are observing the assembly problem through RPM build. Please tell me pless what's wrong?
ERROR 0002: file '/usr/sbin/nginx' contains an invalid rpath '/usr/local/modsecurity/lib' in [/usr/local/modsecurity/lib]
ERROR 0002: file '/usr/sbin/nginx.debug' contains an invalid rpath '/usr/local/modsecurity/lib' in [/usr/local/modsecurity/lib]

SecDebugLog does not work.

Has this in my site vhost config file for Nginx. By some reason debug log does not work ... although audit one works. What could be the reason? how to troubleshoot such issues(( It looks like a bug

  modsecurity on;

  location / {
_##  root /var/www/html;
##  modsecurity_rules_file /etc/nginx/modsecurity-3/nginx-modsecurity-complete.conf;
##   modsecurity_rules_file /etc/nginx/modsec3/modsecurity.conf;_
  modsecurity_rules '
    SecRuleEngine On
    SecDebugLog /tmp/modsec_debug.log
    SecDebugLogLevel 9
    SecRule ARGS "@contains test" "id:1,phase:2,t:trim,block"
  ';
  }

Error occur randomly on images

net::ERR_INCOMPLETE_CHUNKED_ENCODING and net::ERR_CONTENT_LENGTH_MISMATCH occur randomly on images served via proxy_pass to a static site. Didn't even activate modsecurity in config.

Build:

(commit hashes are all their branches' HEAD atm, listing them down just in case)
Ubuntu 14.04
nginx-1.9.5
libinject - copied from Modsecurity's fecefbe8b4fc628dfef9ebff0317bfb9fd871078 (apache2/libinject directory)
libmodsecurity - 283c8c818db1e11fb496064b6cc1158ebe947d58
ModSecurity-nginx - 1713be9e7ca5c95d5c09df59e3c62e548072af0d

nginx.conf

user www-data;
worker_processes 2;
pid /run/nginx.pid;

events {
        worker_connections 2048 ;
        # multi_accept on;
}

http {
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;

        include mime.types;
        default_type application/octet-stream;
        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        gzip on;
        gzip_disable "msie6";

        server {
                listen       80;
                server_name  <my server name>;
                location / {
                        proxy_pass http://<my ip>;
                }
        }
}

It happens at random intervals - reloading may make the image load again. Also when encountering the wrong CONTENT_LENGTH_MISMATCH error, Content-Length header was correct, so it's probably the image's body returned that's wrong.

Jump on uninitialized values

The data element of the string variables in the configuration is unset at initialize time. This can cause startup failure in NGINX for certain allocators. You can reproduce this with the following when daemon and master_process is disabled in NGINX:

valgrind --malloc-fill=A5 --free-fill=DE sbin/nginx

I've attached a patch for this
conf_fix.diff.zip

centos6.5 install modv3 error

when i run build.sh, report this err msg:

`
bash build.sh

libtoolize: putting auxiliary files in .'. libtoolize: copying file ./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, build'. libtoolize: copying file build/libtool.m4'
libtoolize: copying file build/ltoptions.m4' libtoolize: copying file build/ltsugar.m4'
libtoolize: copying file build/ltversion.m4' libtoolize: copying file build/lt~obsolete.m4'
configure.ac:44: warning: macro AM_PROG_AR' not found in library configure.ac:44: warning: macro AM_PROG_AR' not found in library
configure.ac:44: error: possibly undefined macro: AM_PROG_AR
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure:19307: error: possibly undefined macro: AM_DEFAULT_VERBOSITY
autoreconf: /usr/bin/autoconf failed with exit status: 1
configure.ac:44: error: possibly undefined macro: AM_PROG_AR
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure:19307: error: possibly undefined macro: AM_DEFAULT_VERBOSITY
`

autoconf : 2.63
automake: 1.11.1
libtool: 2.4

[1] XML: No XML document found, returning

The message

[1] XML: No XML document found, returning

is getting flooded in the modsec_debug_log

# -- Debug log configuration -------------------------------------------------

# The default debug log configuration is to duplicate the error, warning
# and notice messages from the error log.
#
SecDebugLog /var/log/nginx/modsec_debug_log
SecDebugLogLevel 1


# -- Audit log configuration -------------------------------------------------

# Log the transactions that are marked by a rule, as well as those that
# trigger a server error (determined by a 5xx or 4xx, excluding 404,
# level response status codes).
#
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"

# Log everything we know about a transaction.
SecAuditLogParts ABIJDFHZ

# Use a single file for logging. This is much easier to look at, but
# assumes that you will use the audit log only ocassionally.
#
SecAuditLogType Concurrent
#SecAuditLog /var/log/nginx/modsec_audit_log

# Specify the path for concurrent audit logging.
SecAuditLogStorageDir /var/log/nginx/modsecurity/

About modsecurity-nginx issue

When I use modsecurity-nginx,if I have many crs rules like "REQUEST-901-INITIALIZATION.conf REQUEST-903.9001-DRUPAL-EXCLUSION-RULES.conf REQUEST-903.9002-WORDPRESS-EXCLUSION-RULES.conf
REQUEST-905-COMMON-EXCEPTIONS.conf REQUEST-910-IP-REPUTATION.conf REQUEST-911-METHOD-ENFORCEMENT.conf
REQUEST-912-DOS-PROTECTION.conf REQUEST-913-SCANNER-DETECTION.conf REQUEST-920-PROTOCOL-ENFORCEMENT.conf
REQUEST-921-PROTOCOL-ATTACK.conf REQUEST-930-APPLICATION-ATTACK-LFI.conf REQUEST-931-APPLICATION-ATTACK-RFI.conf
REQUEST-932-APPLICATION-ATTACK-RCE.conf REQUEST-933-APPLICATION-ATTACK-PHP.conf REQUEST-941-APPLICATION-ATTACK-XSS.conf
REQUEST-942-APPLICATION-ATTACK-SQLI.conf REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf REQUEST-949-BLOCKING-EVALUATION.conf
RESPONSE-950-DATA-LEAKAGES.conf RESPONSE-951-DATA-LEAKAGES-SQL.conf RESPONSE-952-DATA-LEAKAGES-JAVA.conf
RESPONSE-953-DATA-LEAKAGES-PHP.conf RESPONSE-954-DATA-LEAKAGES-IIS.conf RESPONSE-959-BLOCKING-EVALUATION.conf
RESPONSE-980-CORRELATION.conf" so I can use "modsecurity_rules_file /etc/nginx/owasp-modsecurity-crs/rules/*.conf"???????? If I use modsecurity-nginx so I don't use the modsecurity.conf and crs-setup.conf and unicode.mapping file ??????

Which branch to use?

Hi,

I'm not sure which branch to use right now? My last info was the experimental branch but it seems outdated againt master. Shall we switch to master? I've several segfaults with N+ and searching for a solution (also on modsec_dev list)

Thanks!

"root" directive Rules error

I followed test/benchmark test example to include CRS v3, but when reloading nginx, it always return such error:

sudo service nginx reload
nginx: [emerg] "root" directive Rules error. File: /usr/local/nginx/conf/crs-setup.conf. Line: 117. Column: 40. SecDefaultActions can only be placed once per phase and configuration context. Phase 1 was informed already.  in /usr/local/nginx/conf/nginx.conf:119
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed

Softwares version I'm using:

vagrant@waf:/usr/local/nginx/conf$ nginx -V
nginx version: nginx/1.9.15
built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) 
configure arguments: --add-module=../ModSecurity-nginx
vagrant@waf:/usr/local/nginx/conf$ 

And configuration files are attached.
config.zip

no message in log and auditlog

hi
I'm using libmodsecurity-3.0.0 + modsecurity-nginx-0.0.1 + nginx-1.12.0. I could not find message in the log file even if rule has been triggered.
and here is my nginx configuration

location / {
              modsecurity on;
              modsecurity_rules_file /etc/nginx/modsecurity/modsec_includes.conf;
              root   /usr/share/nginx/html;
              index  index.html index.htm;
              proxy_set_header X-Forwarded-For $remote_addr;
              proxy_set_header REMOTE_ADDR $remote_addr;
              proxy_set_header HOST $host;
          }

and this is modsec_includes.conf

include modsecurity.conf
include crs-setup.conf
include rules/*.conf

SecRule ARGS "test" "msg:'trigger the rule',id:'44444',log,auditlog,deny" #this is the rule i used to test

and in modsecurity.conf, I define the log and audit log path

SecDebugLogLevel 3
SecDebugLog /var/log/nginx/debug.log

SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecAuditLog /var/log/modsec_audit.log

Even if the rule(id:44444) is triggered , I cannot find any message in the debug.log or modsec_audit.log
I really don't know why this happened and please correct me if my configuration is wrong.

Sharp increase in memory usage

I thought mod_security would increase nginx CPU usage on the contrary ..it is having more impact on memory usage and less impact on CPU.

Is this how it should be

Audit log E block contains binary data

Hi.

Issue: The E segment of serial logging contains binary data, possibly of uninitialized memory.
Expected: Contents of the response body, mostly HTML and other human readable responses.

While developing rules I'm using the Serial audit log format, since it is easy to tail -f and truncate. The output of the E block of an audit entry looks suspicious:

---9vOuhfZZ---A--
[28/Jul/2017:09:34:02 +0200] 150122724251.842785 127.0.0.1 52804 127.0.0.1 8085
---9vOuhfZZ---B--
GET /foo?file=/../../etc/passwd HTTP/1.1
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: HTTPie/0.9.8
X-Forwarded-For: x.x.x.x
host: example.com
---9vOuhfZZ---D--

---9vOuhfZZ---E--
<B3><C9>(<C9>ͱ<E3><E5><B2><C9>HML<B1><B3>)<C9>,<C9>I<B5>310Vp<CB>/J<CA>LIIͳ<DA>胕^@<95>&<E5><A7>T*$<A5>'<E7><E7><E4>^W<D9>*<95>gd<96><A4>*<81>
<8C>HN<CD>+I-<B2><B3><C9>0D7^A(b<A3>^O<95>^F<D9>^ET^D<E5><E5><A5>g<E6>U<E8>ESC<EA>^Y^Z<EB>^Y!+<D1>^GY^B2T^_<EA>@^@a^Qs<8F><A9>^@^@^@<FF><FF>
<FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF>
<FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF>
<FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF><FF>
[cut]

(The large number of here is because my nginx was just restarted, later requests have more entropy)

According to https://github.com/SpiderLabs/ModSecurity/wiki/ModSecurity-2-Data-Formats#intended-response-body-e the E block is the intended response body.

The response body for the /foo URI is the stock nginx 403 Forbidden page. I don't think the binary representation above is of that HTML page.

To me it looks like uninitialized memory is being logged. If that is the case, it can be both confusing and downright misleading to read, depending on what the allocation heap was used for last time.

Suggested: If the E block data is not available inside the nginx runtime, the E block should be logged as empty.

I'm running abbf2c4 with libmodsecurity from v3/master (currently 02426466).

Won't compile with nginx 1.12.0

I can't seem to get this connector to compile with nginx 1.12.0

I have installed modsecurity lib but when I compile nginx, it claims it can't find it.

Output:

adding module in /opt/ModSecurity-nginx
checking for ModSecurity library ... not found
checking for ModSecurity library in /usr/local/modsecurity ... not found
 ./configure: error: ngx_http_modsecurity_module requires the ModSecurity library.

tree command of /usr/local/modsecurity:

/usr/local/modsecurity/
├── bin
│   ├── mlogc
│   ├── mlogc-batch-load.pl
│   └── rules-updater.pl
└── lib
└── mod_security2.so

Did I not compile the branch for the modsecurity library correctly? Perhaps I missed something.

EDIT: For instructions on the library compilation, I went here (however I'm using different VPS and ArchLinux):
https://help.dreamhost.com/hc/en-us/articles/223608748-How-to-Install-libmodsecurity-Nginx-on-Ubuntu-14-04

ngx_http_modsecurity_module requires the ModSecurity library

I am trying to build custom deb package with modsecurity and pagespeed but getting this error

hecking for sysconf(_SC_NPROCESSORS_ONLN) ... found
checking for openat(), fstatat() ... found
checking for getaddrinfo() ... found
configuring additional modules
adding module in /root/rebuildnginx/modsecurity
checking for ModSecurity library ... not found
checking for ModSecurity library in /usr/local/modsecurity ... not found
 ./configure: error: ngx_http_modsecurity_module requires the ModSecurity library.
debian/rules:86: recipe for target 'configure_debug' failed
make[1]: *** [configure_debug] Error 1
make[1]: Leaving directory '/root/rebuildnginx/nginx-1.11.4'
debian/rules:64: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

Following this tutorial but with latest nginx and pagespeed modules versions

https://serversforhackers.com/compiling-third-party-modules-into-nginx

root@host:/rebuildnginx# ll
total 1084
drwxr-xr-x 5 root root 4096 Sep 14 14:20 ./
drwx------ 13 root root 4096 Sep 14 14:21 ../
drwxr-xr-x 4 root root 4096 Aug 22 16:20 modsecurity/
drwxr-xr-x 11 root root 4096 Sep 14 14:22 nginx-1.11.4/
-rw-r--r-- 1 root root 133672 Sep 13 12:47 nginx_1.11.4-1
xenial.debian.tar.xz
-rw-r--r-- 1 root root 1843 Sep 13 12:47 nginx_1.11.4-1~xenial.dsc
-rw-r--r-- 1 root root 949793 Sep 13 12:46 nginx_1.11.4.orig.tar.gz
drwxr-xr-x 6 root root 4096 Sep 14 14:19 pagespeed/

Gzip encoded data

Ok so I was getting:

---deISzRSW---B--
GET /wp-content/uploads/2012/03/satan%E2%80%99s-malicious-agenda-300x200.jpg HTTP/1.1
Content-Length: 0
X-Forwarded-For: 51.255.81.66
CF-IPCountry: FR
Host: www.danielsblog.org
CF-Origin-IP: 174.138.119.56
Accept-Encoding: gzip
X-Forwarded-Proto: https
CF-Origin-Https: on
User-Agent: Toweya.com bot; report abuse to [email protected]
Connection: Keep-Alive
CF-RAY: 3734527bd93168f6-CDG
Accept: /
CF-Visitor: {"scheme":"https"}

(binary data here)

After reading through docs I find the directive I need:

SecDisableBackendCompression On

.....but it don't work....Nginx won't start with it in my modsecurity.conf file:

2017/06/23 03:52:08 [emerg] 5146#5146: "modsecurity_rules_file" directive Rules error. File: /etc/nginx/snippets/modsecurity.conf. Line: 9. Column: 32. Invalid input: SecDisableBackendCompression On in /etc/nginx/conf.d/allcapa.org.conf:21

Poor performance with modsecurity enabled

As suggested within issues owasp-modsecurity/ModSecurity#1318 I'll open a new one here. With MS enabled with the current codebase the performance test with "ab" from apache2-utils is not going over 300req/s.

With a virtual machine on Virtualbox with 4 cores and 4gb ram:

root@nginx:~# ab -n 10000 -c 20 -k http://127.0.0.1:80/
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.11.9
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /
Document Length:        612 bytes

Concurrency Level:      20
Time taken for tests:   73.444 seconds
Complete requests:      10000
Failed requests:        0
Keep-Alive requests:    9909
Total transferred:      8499545 bytes
HTML transferred:       6120000 bytes
Requests per second:    136.16 [#/sec] (mean)
Time per request:       146.889 [ms] (mean)
Time per request:       7.344 [ms] (mean, across all concurrent requests)
Transfer rate:          113.02 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    14  147  31.7    141    1308
Waiting:       14  147  31.7    141    1308
Total:         14  147  31.7    141    1308

Percentage of the requests served within a certain time (ms)
  50%    141
  66%    147
  75%    151
  80%    154
  90%    166
  95%    188
  98%    207
  99%    232
 100%   1308 (longest request)

And without:

root@nginx:~# /opt/nginx/sbin/nginx
root@nginx:~# ab -n 10000 -c 20 -k http://127.0.0.1:80/
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.11.9
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /
Document Length:        612 bytes

Concurrency Level:      20
Time taken for tests:   0.197 seconds
Complete requests:      10000
Failed requests:        0
Keep-Alive requests:    9909
Total transferred:      8499545 bytes
HTML transferred:       6120000 bytes
Requests per second:    50755.75 [#/sec] (mean)
Time per request:       0.394 [ms] (mean)
Time per request:       0.020 [ms] (mean, across all concurrent requests)
Transfer rate:          42128.99 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.3      0       5
Waiting:        0    0   0.3      0       5
Total:          0    0   0.3      0       5

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      1
  98%      1
  99%      2
 100%      5 (longest request)

I tested this also with Nginx+ but then the tool stucks a 90%, no mather how many requests (20 cores and 64GB ram):

This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 999 requests completed

But with just 100 it's ok:

Server Software:        nginx
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /
Document Length:        162 bytes

Concurrency Level:      20
Time taken for tests:   0.243 seconds
Complete requests:      100
Failed requests:        0
Non-2xx responses:      100
Keep-Alive requests:    100
Total transferred:      31000 bytes
HTML transferred:       16200 bytes
Requests per second:    410.71 [#/sec] (mean)
Time per request:       48.696 [ms] (mean)
Time per request:       2.435 [ms] (mean, across all concurrent requests)
Transfer rate:          124.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.7      0       3
Processing:     3   42  34.6     58      99
Waiting:        3   42  34.6     58      99
Total:          3   42  34.6     60      99

Percentage of the requests served within a certain time (ms)
  50%     60
  66%     66
  75%     70
  80%     74
  90%     86
  95%     92
  98%     97
  99%     99
 100%     99 (longest request)

Perhaps others can post their results too.

Ah, I also tried to disable logging but it doens't affect the performance at all.

Worker Process Crashes on POST Request Only

Greetings,

I am using Openresty 1.11.2.4 together with Libmodsecurity + Nginx Connector current master. Here is my setup together with compiled dependencies:

Compilation options for Modsecurity3:

./configure --with-yajl=/usr/local --with-geoip=yes --with-lmdb=yes

Recognised Deps:

ModSecurity - v3.0.0+e14dc60 for Linux

 Mandatory dependencies
   + libInjection                                  ....v2.9.0-839-ge14dc60
   + SecLang tests                                 ....e14dc60

 Optional dependencies
   + GeoIP                                         ....found
      /usr/lib/x86_64-linux-gnu//libGeoIP.so, /usr/include
   + LibCURL                                       ....found v7.38.0
      -L/usr/lib/x86_64-linux-gnu -lcurl,  -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....found
      -lyajl, -DWITH_YAJL -I/usr/local/include
   + LMDB                                          ....found
      -llmdb, -DWITH_LMDB -I/usr/include
   + LibXML2                                       ....found v2.9.1
      -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2

 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled

When sending POST requests

Nginx Debug Log snippet:

6 @16
2017/07/17 09:21:54 [debug] 597#0: *42 rewrite phase: 5
2017/07/17 09:21:54 [debug] 597#0: *42 rewrite phase: 6
2017/07/17 09:21:54 [debug] 597#0: *42 http script value: "duedil"
2017/07/17 09:21:54 [debug] 597#0: *42 http script set $log_format
2017/07/17 09:21:54 [debug] 597#0: *42 post rewrite phase: 7
2017/07/17 09:21:54 [debug] 597#0: *42 generic phase: 8
2017/07/17 09:21:55 [debug] 598#0: timer delta: 501
2017/07/17 09:21:55 [debug] 598#0: worker cycle
2017/07/17 09:21:55 [debug] 598#0: accept mutex locked
2017/07/17 09:21:55 [debug] 598#0: epoll add event: fd:20 op:1 ev:00002001
2017/07/17 09:21:55 [debug] 598#0: epoll timer: -1
2017/07/17 09:21:55 [notice] 1#0: signal 17 (SIGCHLD) received
2017/07/17 09:21:55 [alert] 1#0: worker process 597 exited on signal 11 (core dumped)

The core dump reads:

Reading symbols from /opt/openresty/nginx/sbin/nginx...done.
[New LWP 587]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `nginx: worker process                                                         '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000000004f4b38 in ngx_http_modsecurity_pre_access_handler (r=0x2b97398)
    at /root/install/lib/ModSecurity-nginx-master/src/ngx_http_modsecurity_pre_access.c:134
134	        ngx_chain_t *chain = r->request_body->bufs;

Please let me know if you require any more information. Many thanks in advance.

header already sent while sending response to client

I think issue #14 is not fixed

I am getting this error

# nginx -V
nginx version: nginx/1.11.12
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) 
built with LibreSSL 2.5.1
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/etc/nginx/modules --with-pcre=./pcre-8.40 --with-pcre-jit --with-zlib=./zlib-1.2.11 --with-openssl=./libressl-2.5.1 --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --http-log-path=/var/log/nginx/access_log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nobody --group=nobody --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --add-dynamic-module=naxsi-http2/naxsi_src --with-file-aio --with-threads --with-stream --with-stream_ssl_module --with-http_slice_module --with-compat --with-http_v2_module --with-http_geoip_module=dynamic --add-dynamic-module=ngx_pagespeed-release-1.11.33.4-beta --add-dynamic-module=/usr/local/rvm/gems/ruby-2.3.1/gems/passenger-5.1.2/src/nginx_module --add-dynamic-module=ngx_brotli --add-dynamic-module=echo-nginx-module-0.60 --add-dynamic-module=headers-more-nginx-module-0.32 --add-dynamic-module=ngx_http_redis-0.3.8 --add-dynamic-module=redis2-nginx-module --add-dynamic-module=srcache-nginx-module-0.31 --add-dynamic-module=ngx_devel_kit-0.3.0 --add-dynamic-module=set-misc-nginx-module-0.31 --add-dynamic-module=ModSecurity-nginx --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' --with-ld-opt=-Wl,-E

I was trying to do a ab benchmark which is flooding audit log with


---8Tko7yAx---F--
X-Page-Speed: 1.11.33.4-0
Expires: Fri, 31 Mar 2017 14:10:46 GMT
Vary: Accept-Encoding
Cache-Control: max-age=0, no-cache
Cache-Control: max-age=0, no-cache
Connection: keep-alive
Content-Type: text/html;charset=ISO-8859-1
Date: Fri, 31 Mar 2017 14:10:46 GMT
Date: Fri, 31 Mar 2017 14:10:46 GMT
Server: XtendWeb-nginx
Server: XtendWeb-nginx

---8Tko7yAx---H--
ModSecurity: Warning. Matched "Operator `Eq' with parameter `0' against variable `TX' (Value: `0' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-912-DOS-PROTECTION.conf"] [line "80"] [id "912100"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [ref ""]
ModSecurity: Warning. Matched "Operator `Eq' with parameter `0' against variable `TX' (Value: `0' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-910-IP-REPUTATION.conf"] [line "119"] [id "910130"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [ref ""]
ModSecurity: Warning. Matched "Operator `Rx' with parameter `(?:<(?:TITLE>Index of.*?<H|title>Index of.*?<h)1>Index of|>\[To Parent Directory\]<\/[Aa]><br>)' against variable `RESPONSE_BODY' (Value: `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\x0a<html>\x0a <head>\x0a  <title>Index of /< (565 characters omitted)' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-950-DATA-LEAKAGES.conf"] [line "22"] [id "950130"] [rev "2"] [msg "Directory Listing"] [data "Matched Data: <title>Index of /</title>\x0a </head>\x0a <body>\x0a<h1>Index of found within RESPONSE_BODY: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\x0a<html>\x0a <head>\x0a  <title>Index of /</title>\x0a </head>\x0a <body>\x0a<h1>Index of /</h1>\x0a  <table>\x0a   <tr><th valign="top">&nbsp;</th><th><a href="?C=N;O=D">Name</a></th><th><a href="?C=M;O=A">Last modified</a></th><th><a href="?C=S;O=A">Size</a></th><th><a href="?C=D;O=A">Description</a></th></tr>\x0a   <tr><th colspan="5"><hr></th></tr>\x0a<tr><td valign="top">&nbsp;</td><td><a href="cgi-bin/">cgi-bin/</a>               </td><td align="right">2017-03-31 11:50  </td><td align="right">  - </td><td>&nbsp;</td></tr>\x0a   <tr><th colspan="5"><hr></th></tr>\x0a</table>\x0a</body></html>\x0a"] [severity "3"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-disclosure"] [tag "OWASP_CRS/LEAKAGE/INFO_DIRECTORY_LISTING"] [tag "WASCTC/WASC-13"] [tag "OWASP_TOP_10/A6"] [tag "PCI/6.5.6"] [ref "o73,55v92,623"]
ModSecurity: Warning. Matched "Operator `Ge' with parameter `%{tx.outbound_anomaly_score_threshold}' against variable `TX:OUTBOUND_ANOMALY_SCORE' (Value: `4' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf"] [line "164"] [id "959100"] [rev ""] [msg "Outbound Anomaly Score Exceeded (Total Score: 4)"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [tag "anomaly-evaluation"] [ref ""]
ModSecurity: Warning. Matched "Operator `Eq' with parameter `0' against variable `TX' (Value: `0' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-912-DOS-PROTECTION.conf"] [line "74"] [id "912110"] [rev ""] [msg ""] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [ref ""]
ModSecurity: Warning. Matched "Operator `Ge' with parameter `%{tx.outbound_anomaly_score_threshold}' against variable `TX:OUTBOUND_ANOMALY_SCORE' (Value: `4' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf"] [line "69"] [id "980140"] [rev ""] [msg "Outbound Anomaly Score Exceeded (score 4): Directory Listing'"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [tag "event-correlation"] [ref ""]

---8Tko7yAx---I--

And the nginx error log is filled with

2017/03/31 14:12:20 [alert] 2518#2518: *30195 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30196 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30197 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30198 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30199 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30200 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30201 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30202 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30203 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30204 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30205 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30206 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30207 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"
2017/03/31 14:12:20 [alert] 2518#2518: *30208 header already sent while sending to client, client: xx.xx.xx.xx, server: domain.com, request: "GET / HTTP/1.0", upstream: "http://yy.yy.yy.yy:9999/", host: "domain.com"

Segfault while reloading/stopping nginx

This problem was originally reported by @defanator. @defanator also investigated the issue, nailing down the problem to the exactly point where the code is crashing.

Bellow is a simple configuration that will lead to the crash:

    server {
        listen 80;

        modsecurity on;
        modsecurity_rules '
           SecDefaultAction "phase:2,deny"
        ';

        location / {
            proxy_pass http://backend;
        }

        location /another/one {
            proxy_pass http://backend;
        }

        location /and/third/one/ {
            proxy_pass http://backend;
        }
    }

Here crash stack trace:

#0  0x0000000000000061 in ?? ()
#1  0x00007fbeb7867376 in modsecurity::Rules::~Rules (this=0x23f4440, __in_chrg=<optimized out>) at rules.cc:97
#2  0x00007fbeb786739e in modsecurity::msc_rules_cleanup (rules=0x23f4440) at rules.cc:357
#3  0x00007fbeb7b45645 in ngx_http_modsecurity_config_cleanup (data=0x23f1438) at ../ModSecurity-nginx/src/ngx_http_modsecurity_module.c:611
#4  0x000000000040df2c in ngx_destroy_pool (pool=0x236c4f0) at src/core/ngx_palloc.c:57
#5  0x000000000043031a in ngx_worker_process_exit (cycle=cycle@entry=0x236c540) at src/os/unix/ngx_process_cycle.c:1001
#6  0x0000000000430407 in ngx_worker_process_cycle (cycle=cycle@entry=0x236c540, data=data@entry=0x0) at src/os/unix/ngx_process_cycle.c:758
#7  0x000000000042ee60 in ngx_spawn_process (cycle=cycle@entry=0x236c540, proc=proc@entry=0x430352 <ngx_worker_process_cycle>, data=data@entry=0x0, 
    name=name@entry=0x481297 "worker process", respawn=respawn@entry=-3) at src/os/unix/ngx_process.c:198
#8  0x000000000043053b in ngx_start_worker_processes (cycle=cycle@entry=0x236c540, n=8, type=type@entry=-3) at src/os/unix/ngx_process_cycle.c:358
#9  0x0000000000430e0f in ngx_master_process_cycle (cycle=cycle@entry=0x236c540) at src/os/unix/ngx_process_cycle.c:130
#10 0x000000000040c94e in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:367

modsecurity not working with srcache module

If
modsecurity on;

is done in a server context that has

srcache_fetch GET /redis-fetch $key;
srcache_store PUT /redis-store key=$escaped_key;

Nothing is stored in the redis sever!

Have ModSecurity-nginx variable in nginx?

Hello.
Have ModSecurity-nginx variable in nginx?
Can i used this code?
if ($modsecurity variable) {
modsecurity_rules '
SecRuleRemoveById 10 ';
}

if ($modsecurity variable) {
modsecurity on;
modsecurity_rules_file /etc/nginx/main.conf;
}

If nginx made with modsecurity than block modsecurity_rules work.
If nginx made without modsecurity than block modsecurity_rules does not work.

Can you help me?

Milestone

Understand that libmodsecurity is under heavy development. How close are we to at least an alpha for testing this module?

Memory leak

I haven't fully diagnosed the leak yet but I've managed to get some readouts in the attached massif output in the hope it will aid you in finding the route cause.

The leak appears to happen when using this:

SecRule ARGS "@contains test" "id:50,auditlog,phase:2,t:trim,block"

As part of the following:

  modsecurity_rules '
    SecRuleEngine On
    SecAuditEngine On
    SecAuditLogParts ABCIFHZ
    SecAuditLogType Serial
    SecAuditLog /tmp/audit.log
    SecRule ARGS "@contains test" "id:50,auditlog,phase:2,t:trim,block"
    SecRule ARGS "@streq block403" "id:11,phase:1,status:403,block"
    SecRule ARGS "@streq redirect302" "id:3,phase:1,status:302,redirect:http://www.modsecurity.org"

  ';

Several KB appear to be leaked on every request which significantly grows over time.

massif.out.11899.zip

dpkg-shlibdeps: error: no dependency information found for /usr/local/modsecurity/lib/libmodsecurity.so.3

Hmm looks like incompatibility with Ubuntu16 =(( not sure how to work around.

I am rebuild nginx with mod_security and nginx.

root@host:/rebuildnginx/nginx-1.11.4ll ../
total 1084
drwxr-xr-x 5 root root 4096 Sep 14 14:32 ./
drwx------ 13 root root 4096 Sep 14 14:34 ../
drwxr-xr-x 3 root root 4096 Sep 14 13:35 modsec/
drwxr-xr-x 11 root root 4096 Sep 14 15:07 nginx-1.11.4/
-rw-r--r-- 1 root root 133672 Sep 13 12:47 nginx_1.11.4-1
xenial.debian.tar.xz
-rw-r--r-- 1 root root 1843 Sep 13 12:47 nginx_1.11.4-1~xenial.dsc
-rw-r--r-- 1 root root 949793 Sep 13 12:46 nginx_1.11.4.orig.tar.gz
drwxr-xr-x 3 root root 4096 Sep 14 13:32 pagespeed/

dh_compress
dh_fixperms
debian/rules override_dh_strip
make[1]: Entering directory '/root/rebuildnginx/nginx-1.11.4'
dh_strip --dbg-package=nginx-dbg
make[1]: Leaving directory '/root/rebuildnginx/nginx-1.11.4'
dh_makeshlibs
dh_shlibdeps
dpkg-shlibdeps: error: no dependency information found for /usr/local/modsecurity/lib/libmodsecurity.so.3 (used by debian/nginx/usr/sbin/nginx-debug)
Hint: check if the library actually comes from a package.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/nginx.substvars debian/nginx/usr/sbin/nginx debian/nginx/usr/sbin/nginx-debug returned exit code 2
debian/rules:64: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: error: debian/rules binary gave error exit status 2

JSON enabled but 'Failed to parse request body' in logs

Environment

I have configured libModSecurity (aka v3) and nginx connector on CentOS 7.

Is this the correct to check whether it has been compiled properly?

# ldd /usr/local/modsecurity/lib/libmodsecurity.so.3 | grep -i yajl
	libyajl.so.2 => /lib64/libyajl.so.2 (0x00007f66478ea000)

Problem

Failed to parse request body in log and below error in UI

2017-06-21_19-55-58

Error in /var/log/nginx/error.log

2017/06/21 09:57:34 [warn] 61#61: *2114 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/token"] [unique_id "149803905416.910867"] [ref "v475,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/token HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 62#62: *2116 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/schema"] [unique_id "149803905474.710536"] [ref "v476,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/schema HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 62#62: *2117 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/projects"] [unique_id "149803905493.197393"] [ref "v506,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/projects?all=true&limit=-1&sort=name HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 62#62: *2118 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/userpreferences"] [unique_id "149803905499.256323"] [ref "v504,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/userpreferences?limit=-1&sort=name HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 62#62: *2119 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/setting"] [unique_id "149803905487.495233"] [ref "v506,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/setting?all=false&limit=-1&sort=name HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 62#62: *2120 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/schema"] [unique_id "149803905497.996843"] [ref "v476,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/schema HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 62#62: *2121 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/projectTemplates"] [unique_id "149803905492.131429"] [ref "v505,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/projectTemplates?limit=-1&sort=name HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"
2017/06/21 09:57:34 [warn] 61#61: *2122 [client 10.61.76.80] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.61.76.80"] [uri "/v2-beta/projects"] [unique_id "149803905474.799709"] [ref "v506,1"], client: 10.61.76.80, server: mysite.xyz.com, request: "GET /v2-beta/projects?all=true&limit=-1&sort=name HTTP/1.1", host: "mysite.xyz.com", referrer: "https://mysite.xyz.com/"

Error in /var/log/nginx/modsec_audit.log

---hAifXuzH---H--
ModSecurity: Warning. Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsec_rules.conf"] [line "11"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data "JSON parsing error: parse error: premature EOF\x0a"] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [ref "v493,1"]

Config Files

/etc/nginx/modsec/modsec_rules.conf
SecRuleEngine On
SecRequestBodyAccess On
SecRule REQUEST_HEADERS:Content-Type "(?:application(?:/soap\+|/)|text/)xml" "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"
SecRule REQUEST_HEADERS:Content-Type "application/json" "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecRequestBodyInMemoryLimit 131072
SecRequestBodyLimitAction Reject
SecRule REQBODY_ERROR "!@eq 0" "id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"
SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:'200003',phase:2,t:none,log,deny,status:400, msg:'Multipart request body failed strict validation: PE %{REQBODY_PROCESSOR_ERROR}, BQ %{MULTIPART_BOUNDARY_QUOTED}, BW %{MULTIPART_BOUNDARY_WHITESPACE}, DB %{MULTIPART_DATA_BEFORE}, DA %{MULTIPART_DATA_AFTER}, HF %{MULTIPART_HEADER_FOLDING}, LF %{MULTIPART_LF_LINE}, SM %{MULTIPART_MISSING_SEMICOLON}, IQ %{MULTIPART_INVALID_QUOTING}, IP %{MULTIPART_INVALID_PART}, IH %{MULTIPART_INVALID_HEADER_FOLDING}, FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"
SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000
SecRule TX:/^MSC_/ "!@streq 0" "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"
SecResponseBodyAccess On
SecResponseBodyMimeType text/plain text/html text/xml application/json
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial
SecTmpDir /tmp/
SecDataDir /tmp/
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecAuditLog /var/log/nginx/modsec_audit.log
SecArgumentSeparator &
SecCookieFormat 0
SecUnicodeMapFile unicode.mapping 20127
SecStatusEngine On

# OWASP CRS v3 rules
Include /usr/local/owasp-modsecurity-crs/crs-setup.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-901-INITIALIZATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-903.9001-DRUPAL-EXCLUSION-RULES.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-903.9002-WORDPRESS-EXCLUSION-RULES.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-905-COMMON-EXCEPTIONS.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-910-IP-REPUTATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-911-METHOD-ENFORCEMENT.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-912-DOS-PROTECTION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-913-SCANNER-DETECTION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-921-PROTOCOL-ATTACK.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-931-APPLICATION-ATTACK-RFI.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-933-APPLICATION-ATTACK-PHP.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-950-DATA-LEAKAGES.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-951-DATA-LEAKAGES-SQL.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-952-DATA-LEAKAGES-JAVA.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-953-DATA-LEAKAGES-PHP.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-954-DATA-LEAKAGES-IIS.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf
Include /usr/local/owasp-modsecurity-crs/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf

RESPONSE_BODY variable is unfilled

It seems that ngx_http_modsecurity_body_filter.c makes some assumptions about the ngx_chain_t struct passed to ngx_http_modsecurity_body_filter. Examine the following:

Given the following nginx 1.8.1 build:

poprocks@soter:~/code/C/ModSecurity-nginx/src$ /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.8.1
built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04) 
configure arguments: --with-debug --add-module=/home/poprocks/code/C/ModSecurity-nginx

And given the following inline modsecurity config: https://gist.github.com/p0pr0ck5/3864f5f18ae5ba5cb7aa

Given this diff for debug purposes - https://gist.github.com/p0pr0ck5/a24b2f8f8cd00997df41 -

Given a request to localhost/index.html (which will read the file from disk) we will find ourselves stepping through to here:

ngx_http_modsecurity_body_filter (r=0xfefee0, in=0x7ffff21ab810)
    at /home/poprocks/code/C/ModSecurity-nginx/src/ngx_http_modsecurity_body_filter.c:53
53      if (ctx == NULL) {
(gdb) 
59          dd("checking chain %p", chain);
(gdb) 
61          if (chain->buf->last_buf)
(gdb) 
63              buffer_fully_loadead = 1;
(gdb) 
57      for (; chain != NULL; chain = chain->next)
(gdb) 
67      if (buffer_fully_loadead == 1)
(gdb) 
73              dd("checking chain %p", chain);
(gdb) 
75              u_char *data = chain->buf->start;
(gdb) 
77              dd("data length is %ld", chain->buf->end - data);

We will see the following in the error log:

modsec *** ngx_http_modsecurity_body_filter: body filter, recovering ctx: 0xff0e50 at /home/poprocks/code/C/ModSecurity-nginx/src/ngx_http_modsecurity_body_filter.c line 51.
modsec *** ngx_http_modsecurity_body_filter: checking chain 0x7ffff21ab810 at /home/poprocks/code/C/ModSecurity-nginx/src/ngx_http_modsecurity_body_filter.c line 59.
modsec *** ngx_http_modsecurity_body_filter: checking chain 0x7ffff21ab810 at /home/poprocks/code/C/ModSecurity-nginx/src/ngx_http_modsecurity_body_filter.c line 73.

Examining the chain:

(gdb) print *(ngx_chain_t *) 0x7ffff21ab810
$1 = {buf = 0x1064250, next = 0x0}

Okay, we have a single buffer. The buffers contents:

(gdb) print *(ngx_buf_t *) 0x1064250
$2 = {pos = 0x0, last = 0x0, file_pos = 0, file_last = 612, start = 0x0, end = 0x0, tag = 0x0, file = 0x10642a0, shadow = 0x0, 
  temporary = 0, memory = 0, mmap = 0, recycled = 0, in_file = 1, flush = 0, sync = 0, last_buf = 1, last_in_chain = 1, 
  last_shadow = 0, temp_file = 0, num = 0}

So there is no buffer to pass to msc_append_response_body since the contents come from disk.

How to include all CRS rules?

I have successfully built a nginx+libmodescuryt+ModSecurity-nginx WAF, and it works when I manully injected a rule in a / url. But when I try to include owasp-crs, seems it can not work. And below are my configures:

nginx.conf

server {
        ...
        location / {
            modsecurity on;
            modsecurity_rules_file /usr/local/nginx/conf/modsecurity.conf;
            root   html;
            index  index.html index.htm;
        }
}

modsecurity.conf

SecRuleEngine On
....
SecStatusEngine On

Include crs-setup.conf
Include rules/*.conf

After I reload the nginx, it can not block malicious attack which can be blocked by the nginx+2.9modscurity.

So can someone tell me how to configure?

Negation on operators not working

Hi,

iam tried to use negation on operator eg. !@ipmatch, but not work.
If i use this rule
SecRule REMOTE_ADDR "!@ipmatch 172.16.71.132" \ "msg:'DENY TEST',\ id:1,\ t:none,\ deny"
but in debug log operator starting without !
[4] (Rule: 1) Executing operator "@ipmatch" with param "172.16.71.132" against REMOTE_ADDR.

And question is, negation is not implemented?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.