Comments (7)
Hi @theseion,
thanks for this report. We're going to check this issue soon.
from modsecurity.
I have looked into this issue and I believe that this behavior is caused by Nginx, not libmodsecurity3.
Libmodsecurity3 has a great testing framework - anyone can write tests and check the behavior of the library, without connectors and other components. This framework is base of the regression test, also uses the CI pipeline.
I have created this test case:
[
{
"enabled":1,
"version_min":300000,
"title":"Testing % character in URI - issue 3235",
"expected":{
"http_code": 403
},
"client":{
"ip":"200.249.12.31",
"port":123
},
"request":{
"headers":{
"User-Agent":"OWASP CRS test agent",
"Host":"localhost",
"Accept":"text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"
},
"uri":"/?test=%uff0F",
"method":"GET",
"body": [ ]
},
"server":{
"ip":"200.249.12.31",
"port":80
},
"rules":[
"SecRuleEngine On",
"SecRule REQUEST_URI|REQUEST_BODY \"@rx (?i)%uff[0-9a-f]{2}\" \"id:920260,phase:2,deny,t:none,msg:'Unicode Full/Half Width Abuse Attack Attempt',logdata:'%{MATCHED_VAR_NAME}=%{MATCHED_VAR}'\""
]
}
]
Key parts:
- under the
expected
I mark that the library must return with 403 - check the
request
.headers
for headers, those are the same as in the mentioned CRS test file - the
request
.uri
is also the same as in the test file - in the
rules
section I copied the simple form of original rule 920620.
When I run this test, I got the expected 403 result, and the test passes:
$ ./regression_tests test-cases/regression/issue-3135.json
ModSecurity 3.0.12 - tests
(options are not available -- missing GetOpt)
# File Name Test Name Passed?
--- --------- --------- -------
1 issue-3135.json Testing % character in URI - issue 3235 passed!
Ran a total of: 1 regression tests - All tests passed. 0 skipped test(s). 0 disabled test(s).
If I change the expected
result to 200
, I got the detailed errors:
Test failed. From: test-cases/regression/issue-3135.json.
Test name: Testing % character in URI - issue 3235.
Reason:
HTTP code mismatch. expecting: 200 got: 403
Debug log:
[17150244124.274430] [] [4] Initializing transaction
[17150244124.274430] [] [4] Transaction context created.
[17150244124.274430] [] [4] Starting phase CONNECTION. (SecRules 0)
[17150244124.274430] [] [9] This phase consists of 0 rule(s).
[17150244124.274430] [] [4] Starting phase URI. (SecRules 0 + 1/2)
[17150244124.274430] [/?test=%uff0F] [4] Adding request argument (GET): name "test", value "%uff0F"
[17150244124.274430] [/?test=%uff0F] [4] Starting phase REQUEST_HEADERS. (SecRules 1)
[17150244124.274430] [/?test=%uff0F] [9] This phase consists of 0 rule(s).
[17150244124.274430] [/?test=%uff0F] [9] Appending request body: 0 bytes. Limit set to: 0.000000
[17150244124.274430] [/?test=%uff0F] [4] Starting phase REQUEST_BODY. (SecRules 2)
[17150244124.274430] [/?test=%uff0F] [9] This phase consists of 1 rule(s).
[17150244124.274430] [/?test=%uff0F] [4] (Rule: 920260) Executing operator "Rx" with param "(?i)%uff[0-9a-f]{2}" against REQUEST_URI|REQUEST_BODY.
[17150244124.274430] [/?test=%uff0F] [9] Target value: "/?test=%uff0F" (Variable: REQUEST_URI)
[17150244124.274430] [/?test=%uff0F] [9] Matched vars updated.
[17150244124.274430] [/?test=%uff0F] [9] Target value: "" (Variable: REQUEST_BODY)
[17150244124.274430] [/?test=%uff0F] [4] Rule returned 1.
[17150244124.274430] [/?test=%uff0F] [9] Saving msg: Unicode Full/Half Width Abuse Attack Attempt
[17150244124.274430] [/?test=%uff0F] [4] Running (disruptive) action: deny.
[17150244124.274430] [/?test=%uff0F] [8] Running action deny
[17150244124.274430] [/?test=%uff0F] [8] Skipping this phase as this request was already intercepted.
[17150244124.274430] [/?test=%uff0F] [4] Starting phase RESPONSE_HEADERS. (SecRules 3)
[17150244124.274430] [/?test=%uff0F] [9] This phase consists of 0 rule(s).
[17150244124.274430] [/?test=%uff0F] [9] Appending response body: 0 bytes. Limit set to: 0.000000
[17150244124.274430] [/?test=%uff0F] [4] Starting phase RESPONSE_BODY. (SecRules 4)
[17150244124.274430] [/?test=%uff0F] [4] Response body is disabled, returning... 2
[17150244124.274430] [/?test=%uff0F] [4] Starting phase LOGGING. (SecRules 5)
[17150244124.274430] [/?test=%uff0F] [9] This phase consists of 0 rule(s).
[17150244124.274430] [/?test=%uff0F] [8] Checking if this request is suitable to be saved as an audit log.
[17150244124.274430] [/?test=%uff0F] [8] Checking if this request is relevant to be part of the audit logs.
[17150244124.274430] [/?test=%uff0F] [5] Audit log engine was not set.
[17150244124.274430] [/?test=%uff0F] [8] Request was relevant to be saved. Parts: 4430
Error log:
[client 200.249.12.31] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Rx' with parameter `(?i)%uff[0-9a-f]{2}' against variable `REQUEST_URI' (Value: `/?test=%uff0F' ) [file "issue-3135.json"] [line "2"] [id "920260"] [rev ""] [msg "Unicode Full/Half Width Abuse Attack Attempt"] [data "REQUEST_URI=/?test=%uff0F"] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "200.249.12.31"] [uri "/"] [unique_id "17150244124.274430"] [ref "o7,6v4,13"]
I think I have to insect the connector - if there is no any invalid decoding, then this behavior by Nginx, not the library nor connector.
Please take a look at the test and correct me, may be I'm wrong.
from modsecurity.
Thanks, I'll check.
from modsecurity.
Just for sure: I checked with the example you given - here is the test case:
"uri":"/?test=%ua",
"method":"GET",
"body": [ ]
and the result:
[171502559910.049186] [/?test=%ua] [4] Adding request argument (GET): name "test", value "%ua"
[171502559910.049186] [/?test=%ua] [4] Starting phase REQUEST_HEADERS. (SecRules 1)
[171502559910.049186] [/?test=%ua] [9] This phase consists of 0 rule(s).
[171502559910.049186] [/?test=%ua] [9] Appending request body: 0 bytes. Limit set to: 0.000000
[171502559910.049186] [/?test=%ua] [4] Starting phase REQUEST_BODY. (SecRules 2)
[171502559910.049186] [/?test=%ua] [9] This phase consists of 1 rule(s).
[171502559910.049186] [/?test=%ua] [4] (Rule: 920260) Executing operator "Rx" with param "(?i)%uff[0-9a-f]{2}" against REQUEST_URI|REQUEST_BODY.
[171502559910.049186] [/?test=%ua] [9] Target value: "/?test=%ua" (Variable: REQUEST_URI)
So the library does not encode the pattern %ua
to \xfa
.
from modsecurity.
I just checked with Nginx and libmodsecurity3 - seems to work as you expect:
curl -v "http://localhost/?test=%ua"
result:
[171510021893.966265] [/?test=%ua] [4] (Rule: 920260) Executing operator "Rx" with param "(?i)%uff[0-9a-f]{2}" against REQUEST_URI|REQUEST_BODY.
[171510021893.966265] [/?test=%ua] [9] Target value: "/?test=%ua" (Variable: REQUEST_URI)
[171510021893.966265] [/?test=%ua] [9] Target value: "" (Variable: REQUEST_BODY)
[171510021893.966265] [/?test=%ua] [4] Rule returned 0.
from modsecurity.
This is the same issue as #3016.
Unfortunately, the patch somehow never made it into the release branch:
ModSecurity/src/utils/decode.cc
Line 102 in 5f44383
from modsecurity.
Yes, the release (3.0.12) was on 30th of January, and #3016 was merged in March. Finally, I'm happy we found the root cause.
from modsecurity.
Related Issues (20)
- Annoying DNS queries with @rbl operator HOT 18
- Feature request: Limit the number of rules processed per request HOT 3
- SecAuditLogPart 'E' is logged even if it is not configured HOT 7
- Error: Could not set variable "ip.brute_force_counter" and Could not set variable "ip.xmlrpc_counter" as the collection does not exist. HOT 6
- Is it possible to change the SecAuditLogStorageDir variable so that the logs are sorted by vhost?
- Lua installed, but Modsecurity still dont work with it HOT 2
- Phasing out SecStatusEngine HOT 3
- Regular Expression Failure Triggers `!@rx` HOT 1
- Incorrect utf8toUnicode transformation for 00xx
- docs, contributing: shorten description to improve flow for GitHub contributors, rewrite for owasp HOT 6
- @rbl operator does not support IPv6
- [Idea] Add variable support for SecAuditLog HOT 2
- Update link on Reference Manual v3 wiki page HOT 2
- Ignore SecStatusEngine directive in v2 HOT 1
- Encountering SIGSEGV when parsing multiple rule sets in parallel HOT 4
- is sanitiseArgs not supported in libmodsecurity3 version 3.0.12 HOT 5
- AH00526: Syntax error on line 93 of /etc/apache2/modsec/owasp-modsecurity-crs/rules/REQUEST-903.9004-DOKUWIKI-EXCLUSION-RULES.conf: Error parsing actions: Unknown action: \\ Action 'configtest' failed. The Apache error log may have more information. HOT 4
- Apache: Short Lingering Close
- How to disable audit logging of some HTTP_CODE? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from modsecurity.