Comments (3)
Hi,
That's wierd, I cannot reproduce your issue.
Can you send me how you use grok?
I tried this:
Grok g = new Grok();
g.addPatternFromFile("patterns/base");
g.compile("%{APACHE}");
Match gm = g.match("10.192.1.47 - - [23/May/2013:10:47:40] \"GET /flower1_store/category1.screen?category_id1=FLOWERS HTTP/1.1\" 200 10577 \"http://mystore.abc.com/flower1_store/main.screen&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 3823 404");
gm.captures();
//See the result
System.out.println(gm.toJson());
I also change patterns/base with:
HTTPDATE1 %{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME}
APACHE_2 %{IPORHOST} %{USER:hyphen} %{USER:user} \[%{HTTPDATE1:timestamp}\] %{QUOTEDSTRING:query} %{NUMBER:response} (?:%{NUMBER:bytes}|-)(?: %{QUOTEDSTRING:Referer} %{QUOTEDSTRING:agent})?
And I get:
{
"APACHE": "10.192.1.47 - - [23/May/2013:10:47:40] \"GET /flower1_store/category1.screen?category_id1\u003dFLOWERS HTTP/1.1\" 200 10577 \"http://mystore.abc.com/flower1_store/main.screen\u0026JSESSIONID\u003dSD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\"",
"BASE10NUM": 10577,
"HOSTNAME": "10.192.1.47",
"HOUR": 10,
"IPORHOST": "10.192.1.47",
"MINUTE": 47,
"MONTH": "May",
"MONTHDAY": 23,
"Referer": "http://mystore.abc.com/flower1_store/main.screen\u0026JSESSIONID\u003dSD1SL10FF3ADFF3",
"SECOND": 40,
"TIME": "10:47:40",
"USERNAME": "-",
"YEAR": 2013,
"agent": "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10",
"bytes": 10577,
"hyphen": "-",
"query": "GET /flower1_store/category1.screen?category_id1\u003dFLOWERS HTTP/1.1",
"response": 200,
"timestamp": "23/May/2013:10:47:40",
"user": "-"
}
Also what version of java are you using?
from java-grok.
1 I do as your steps and it is ok now, thanks. Version of grok is "Grok-0.0.3.1-SNAPSHOT.jar".
2 I know the reason: I do not add " APACHE_2 %{IPORHOST} %{USER:hyphen} %{USER:user} [%{HTTPDATE1:timestamp}] %{QUOTEDSTRING:query} %{NUMBER:response} (?:%{NUMBER:bytes}|-)(?: %{QUOTEDSTRING:Referer} %{QUOTEDSTRING:agent})?"
3 If I parse a new type of log, I have to add a new pattern, otherwise errors may occurs. In my case, I need to parse types of logs, some types of them I do not know before my program runs. So I want to ask you wehether I can parse a log that only parts of this log are fit to some attom patterns?
for example:
log is "55.3.244.1 GET /index.html 15824 0.043". I use function g.discover(log)
and get the pattern of this log : %{URIHOST} GET %{URIPATHPARAM} 15824 0.043 . If I use g.match(log), I will get errors:
java.lang.IndexOutOfBoundsException: No group 11
at java.util.regex.Matcher.group(Matcher.java:470)
at com.google.code.regexp.Matcher.namedGroups(Unknown Source)
at com.nflabs.Grok.Match.captures(Match.java:75)
at com.nflabs.Grok.LzbGrokeTest.testGrok(LzbGrokeTest.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Can I get some fields of this log based on the gained pattern "%{URIHOST} GET %{URIPATHPARAM} 15824 0.043"??
from java-grok.
Okay.
for your point 3, let me fix it
from java-grok.
Related Issues (20)
- NamedRegexCollection with namedOnly HOT 11
- There should be a way to validate that a grok expression is correct syntactically
- GROK Multiline log parsing HOT 7
- Java 7 version?
- Feature request: Maintain order for captures? HOT 3
- Grok compiler
- Java stack exception HOT 1
- Dropped support for named group captures with underscore HOT 6
- Documentation site down HOT 1
- Travis is broken
- Documentation link isn't available. HOT 2
- Issue with HTTPDATE pattern.
- "Duplicate key" error when using repeated matches with the same name and type
- Publish a new release (version `0.1.10`) to maven central
- Matched json is blank in case any of one regex out multiple does not match HOT 1
- Grok is truncating data after new line
- Parsed log is getting duplicated or repeated content HOT 1
- GrokException or IOException or UncheckedIOException
- RuntimeException or GrokException in Match HOT 1
- Custom date format doesn't work with commas
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from java-grok.