handsetdetection/logstash-filter-handsetdetection

A logstash filter for Handset Detection API v4. Enrich your logs with all sorts of device information. :-)

Handset Detection for Logstash

This is the Handset Detection plugin for Logstash.

Example

Extract the User-Agent header from an Apache log with Grok, and then enrich the log with Headset Detection data:

    filter { 
        grok { 
            match => {  
                "message" => "%{COMBINEDAPACHELOG}" 
            } 
        } 
        handsetdetection {
            username   => "xxxxxxxxxx" 
            password   => "xxxxxxxxxxxxxxxx"
            site_id    => 000000
            detection_type   => "ultimate"
            match      => { 
                "agent" => "user-agent" 
            }
            filter     => [
                "general_vendor", "general_model", "general_type"
            ]
        } 
    }

Configuration Fields

Field	Description	Default
detection_type	Set to cloud in order to do online lookups against the Handset Detection API. Set to ultimate to use Handset Detection Ultimate Edition. Set to community to use Handset Detection Community Edition	cloud
username	Your Handset Detection API username
password	Your Handset Detection API password
site_id	The Handset Detection API site ID to use
db_refresh_days	How often to update the locally stored database, in days (for Handset Detection Ultimate Edition and Community Edition)	10
match	An associative array mapping input field names to header names used for handset detection. For example: Extract the user-agent header from the agent input field.	{ "agent" => "user-agent" }
filter	Optionally, define an array of the handset spec properties to be included in the output. By default, all properties are included in the output.	[]
use_proxy	Set to true if accessing the web through a proxy	false
proxy_server	The proxy server address, if using a proxy
proxy_port	The proxy server port, if using a proxy
proxy_user	Your proxy server username, if authenticating to a proxy
proxy_password	Your proxy server password, if authenticating to a proxy

Installation

# Logstash 2.3 and higher
bin/logstash-plugin install logstash-filter-handsetdetection

# Prior to Logstash 2.3
bin/plugin install logstash-filter-handsetdetection

Building from source

gem build  logstash-filter-handsetdetection.gemspec

logstash-filter-handsetdetection's Issues

Does not work with LS 6.3

Just upgraded our stack to LS 6.3 and handsetdetection no longer installs

Issue on LS 2.X

Installed the plugin to my test LS on my local windows machine and I'm getting this error when starting Logstash:

Pipeline aborted due to error {:exception=>#<IO::EAGAINWaitReadable: Unknown error - errno backtraces disabled; run with -Xerrno.backtrace=true to enable>, :backtrace=>[], :level=>:error}

I'm going to turn on backtrace and will post that asap

Front page example should have detection_type value enclosed with quotes

The front page configuration example shows

filter {
grok {
match => {
"message" => "%{COMBINEDAPACHELOG}"
}
}
handsetdetection {
username => "xxxxxxxxxx"
password => "xxxxxxxxxxxxxxxx"
site_id => 000000
detection_type => ultimate
match => {
"agent" => "user-agent"
}
filter => [
"general_vendor", "general_model", "general_type"
]
}
}

Following this will throw an error concerning detection_type unknown. This is because the value, ultimate, should be enclosed in quotes:
detection_type => "ultimate"

Update documentation for correct install command

Hey guys,
quick one, the install command should be:

bin/logstash-plugin install logstash-filter-handsetdetection

Please update to support Logstash 5.0

Just installed and have the following error on install:

$sudo bin/logstash-plugin install logstash-filter-handsetdetection
Validating logstash-filter-handsetdetection
Installing logstash-filter-handsetdetection
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core":
  In snapshot (Gemfile.lock):
    logstash-core (= 5.0.0)

  In Gemfile:
    logstash-core-plugin-api (>= 0) java depends on
      logstash-core (= 5.0.0) java

    logstash-filter-handsetdetection (>= 0) java depends on
      logstash-core (< 3.0.0, >= 2.0.0) java

    logstash-core (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

No handset_detection.specs.general_aliases values for iPhone, iPad etc.

When using the field handset_detection.specs.general_aliases for a visualisation it gives skewed data compared to handset_detection.specs.general_model as shown here: http://imgur.com/a/fW1YX

As there are no Alias for Windows PC, iPhone, OS X PC or iPad then it seems like we don't get any traffic from those devices. Looking at the general_model though, it's hard to make out which Samsung devices are being used as the model number is not commonly used (hence the reason for the alias).

Perhaps if there is no general_alias then it can default to general_model?