mike-mosher / aws-la Goto Github PK
View Code? Open in Web Editor NEWAWS Log Analyzer -- Send AWS logs to ELK running locally in Docker containers
License: MIT License
AWS Log Analyzer -- Send AWS logs to ELK running locally in Docker containers
License: MIT License
I am following the guide at Readme.md on MAC, but failed at import log step:
$ python importLogs.py --logtype vpc --logdir /Users/wumark/workspace/temp/vpclog/
Beginning import process
Creating mapping in ES for index: vpc_flowlogs
Creating Ingest Pipeline for index: vpc_flowlogs
Creating new index-pattern in .kibana index
Setting formatted fields on index-pattern
Setting index-pattern as default index
Deleting useless index-patterns in .kibana index
Deleting index-pattern: .ml-anomalies-*
Deleting index-pattern: .ml-notifications
importing saved objects into Kibana
Begin importing log files
File: .DS_Store is not the correct format. File need to end with *.gz
Importing log file: /Users/wumark/workspace/temp/vpclog//667164967571_vpcflowlogs_ap-northeast-1_fl-09efe29fb030b37b0_20190225T0645Z_9ba3c655.log.gz
Traceback (most recent call last):
File "importLogs.py", line 356, in
loadFiles()
File "importLogs.py", line 211, in loadFiles
processFiles(f)
File "importLogs.py", line 168, in processFiles
for i in status:
File "/usr/local/aws/lib/python2.7/site-packages/elasticsearch/helpers/init.py", line 306, in parallel_bulk
_chunk_actions(actions, chunk_size, max_chunk_bytes, client.transport.serializer)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 668, in next
raise value
elasticsearch.helpers.BulkIndexError: (u'500 document(s) failed to index.', [{u'index': {u'status': 500, u'_type': u'vpc_flowlogs', u'_index': u'vpc_flowlogs', u'error': {u'caused_by': {u'caused_by': {u'reason': u'Provided Grok expressions do not match field value: [version account-id interface-id srcaddr dstaddr sr (Skip)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.