Invalid bounds from data and all fields beside elastic generated (_type) are null

[ngageoint]

Invalid bounds from data and all fields beside elastic generated (_type) are null

[sjudeng]

Try adding the below lines to your logging file (see here)

log4j.category.mil.nga.giat.data.elasticsearch=DEBUG
log4j.category.mil.nga.giat.process.elasticsearch=DEBUG

This should show the queries that are being sent to Elasticsearch and you can then execute them manually to see if there's any obvious issue with the response. What version of the plugin are you using? Are all the fields nested?

[bytemedwb]

Sjudeng:     Thanks.   I have enabled the debug logging.   The querys to retreive the data bounds or a just a GML preview are simple match_all queries.  Here is the query log:

2018-05-17 12:49:26,384 DEBUG [mil.nga.giat.data.elasticsearch] - getBoundsInternal 2018-05-17 12:49:26,384 DEBUG [mil.nga.giat.data.elasticsearch] - getReaderInternal 2018-05-17 12:49:26,401 DEBUG [mil.nga.giat.data.elasticsearch] - Preparing recordset (oerepodev.incadencecorp.com:recordset) query 2018-05-17 12:49:26,401 DEBUG [mil.nga.giat.data.elasticsearch] - Unlimited maxFeatures not supported. Using default: 10000 2018-05-17 12:49:26,795 DEBUG [mil.nga.giat.data.elasticsearch] - Elasticsearch request: {   "size" : 10000,   "query" : {     "match_all" : { }   },   "from" : 0 } 2018-05-17 12:49:31,801 DEBUG [mil.nga.giat.data.elasticsearch] - Search response: ElasticResponse[total=16190676, hits=10000, aggregations=null, scrollId=null, maxScore=1.0]

Running that query via CURL produces the following results:

***@***.*** ~]# curl -k --cert "/etc/pki/java/temp.pem" -X GET "https://node08:9202/coalesce-oeevent/_search?pretty" -H 'Content-Type: application/json' -d' {     "query": {         "match_all": {}     }, "from" : 0 } ' {   "took" : 159,   "timed_out" : false,   "_shards" : {     "total" : 5,     "successful" : 5,     "failed" : 0   },   "hits" : {     "total" : 16190676,     "max_score" : 1.0,     "hits" : [       {         "_index" : "coalesce-oeevent",         "_type" : "recordset",         "_id" : "2c2c4af1-5756-48c1-ae4b-1feac57696e6",         "_score" : 1.0,         "_source" : {           "eventrecordset.pmesiiptinfrastructure" : 0.25,           "coalesceentity.name" : "OEEvent",           "coalesceentity.datecreated" : "2018-03-11T15:11:33.985Z",           "eventrecordset.actiongeofullname" : "Massachusetts, United States",           "eventrecordset.pmesiipteconomic" : 0.25,           "coalesceentity.source" : "G2Core",           "eventrecordset.pmesiiptphysicalenvironment" : 0.5,           "eventrecordset.ontologyreference" : "Event",           "coalesceentity.version" : "0.2",           "eventrecordset.actiongeoadm2code" : "",           "eventrecordset.day" : 20180311,           "eventrecordset.quadclass" : 1,           "eventrecordset.tags" : [ ],           "eventrecordset.fractiondate" : 2018.1945,           "eventrecordset.goldsteinscale" : 1.9,           "eventrecordset.numsources" : 1,           "eventrecordset.sourceurl" : "http://nhpr.org/post/winter-storm-watch-tuesday-nhs-town-meeting-day",           "eventrecordset.globaleventid" : 737769639,           "eventrecordset.actiongeofeatureid" : "MA",           "eventrecordset.pmesiiptmilitary" : 0.25,           "coalesceentity.lastmodified" : "2018-03-11T15:11:33.996Z",           "eventrecordset.eventcode" : "042",           "eventrecordset.pmesiiptsocial" : 0.5,           "eventrecordset.pmesiiptinformation" : 0.5,           "eventrecordset.actiongeoadm1code" : "USMA",           "eventrecordset.avgtone" : -4.989154,           "eventrecordset.actiongeocountrycode" : "US",           "eventrecordset.isrootevent" : 0,           "eventrecordset.issimulation" : false,           "eventrecordset.nummentions" : 10,           "eventrecordset.pmesiipttime" : 0.25,           "coalesceentity.objectkey" : "2c639634-7fd3-47bb-893d-120feb492ac9",           "eventrecordset.year" : 2018,           "eventrecordset.actiongeolocation" : "42.237300872802734, -71.53140258789062",           "eventrecordset.pmesiiptpolitical" : 0.5,           "eventrecordset.eventrootcode" : "04",           "eventrecordset.monthyear" : 201803,           "eventrecordset.actiongeotype" : 2,           "eventrecordset.datetime" : "2018-03-11T14:15:00.000Z",           "eventrecordset.numarticles" : 10,           "eventrecordset.datasource" : "GDELT",           "eventrecordset.eventbasecode" : "042",           "eventrecordset.dateadded" : 0         }       },       {         "_index" : "coalesce-oeevent",         "_type" : "recordset",         "_id" : "c831ec3b-2ab2-40b8-bbac-47ec627d3333",

As you can see all data is there.   In regards to the nesting this is what shows up in the elasticsearch field configuration:

Use allShort names Use Name Type Default Geometry Stored Analyzed SRID Date Format _id String _index String _type String _score Float _relative_score Float _aggregation byte[] coalesceentity.datecreated Date date_optional_time coalesceentity.lastmodified Date date_optional_time coalesceentity.name String coalesceentity.objectkey String coalesceentity.source String coalesceentity.version String eventrecordset.actiongeoadm1code String x eventrecordset.actiongeoadm1code.fields.keyword String eventrecordset.actiongeoadm2code String x eventrecordset.actiongeoadm2code.fields.keyword String eventrecordset.actiongeocountrycode String x eventrecordset.actiongeocountrycode.fields.keyword String eventrecordset.actiongeofeatureid String x eventrecordset.actiongeofeatureid.fields.keyword String eventrecordset.actiongeofullname String x eventrecordset.actiongeofullname.fields.keyword String eventrecordset.actiongeolocation* 4326 eventrecordset.actiongeotype Integer eventrecordset.avgtone Float eventrecordset.datasource String x eventrecordset.datasource.fields.keyword String eventrecordset.dateadded Integer eventrecordset.datetime Date date_optional_time eventrecordset.day Integer eventrecordset.eventbasecode String x eventrecordset.eventbasecode.fields.keyword String eventrecordset.eventcode String x eventrecordset.eventcode.fields.keyword String eventrecordset.eventrootcode String x eventrecordset.eventrootcode.fields.keyword String eventrecordset.fractiondate Float eventrecordset.globaleventid Integer eventrecordset.goldsteinscale Float eventrecordset.isrootevent Integer eventrecordset.issimulation Boolean eventrecordset.monthyear Integer eventrecordset.numarticles Integer eventrecordset.nummentions Integer eventrecordset.numsources Integer eventrecordset.ontologyreference String x eventrecordset.ontologyreference.fields.keyword String eventrecordset.pmesiipteconomic Float eventrecordset.pmesiiptinformation Float eventrecordset.pmesiiptinfrastructure Float eventrecordset.pmesiiptmilitary Float eventrecordset.pmesiiptphysicalenvironment Float eventrecordset.pmesiiptpolitical Float eventrecordset.pmesiiptsocial Float eventrecordset.pmesiipttime Float eventrecordset.quadclass Integer eventrecordset.sourceurl String x eventrecordset.sourceurl.fields.keyword String eventrecordset.tags String x eventrecordset.year Integer Apply <http://oeservices:8080/geoserver/web/wicket/page?24#>

So yes everyfield is a subfield.   When I do the GML or GEOJSON preview request only the non-subfields (_id,_type, _index, etc.) have values. Does the plug-in not handle the nested subfields?

…

On 5/15/18 12:23 PM, sjudeng wrote: Try adding the below lines to your logging file (see here <https://github.com/ngageoint/elasticgeo/blob/master/gs-web-elasticsearch/doc/index.rst#configuring-logging>) |log4j.category.mil.nga.giat.data.elasticsearch=DEBUG log4j.category.mil.nga.giat.process.elasticsearch=DEBUG | This should show the queries that are being sent to Elasticsearch and you can then execute them manually to see if there's any obvious issue with the response. What version of the plugin are you using? Are all the fields nested <https://www.elastic.co/guide/en/elasticsearch/reference/5.4/nested.html>? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#69 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADslzGHMB9IkAE3EL0DWnZlwzROSju4yks5tywEPgaJpZM4T-x3C>.

-- ========= mailto:[email protected] ============ David W. Boyd VP, Data Solutions 10432 Balls Ford, Suite 240 Manassas, VA 20109 office: [phone] cell: [phone] ============== http://www.incadencecorp.com/ ============ ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture Chair ANSI/INCITS TC Big Data Co-chair NIST Big Data Public Working Group Reference Architecture First Robotic Mentor - FRC, FTC - www.iliterobotics.org Board Member- USSTEM Foundation - www.usstem.org The information contained in this message may be privileged and/or confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and deleting the material from any computer.

[sjudeng]

It should. Filtering isn't supported on nested geometries (it's okay for the object type). But this looks like an issue with feature reading not filtering. Can you post your Elasticsearch mapping (or a snippet including the geo and a couple of the other fields)?

[bytemedwb]

Sjudeng:     Below is the mapping.  Attached is a dump of 5 records.

…

***@***.*** ~]# curl -k --cert "/etc/pki/java/temp.pem" -XGET "https://node08:9202/coalesce-oeevent?pretty" {   "coalesce-oeevent" : {     "aliases" : { },     "mappings" : {       "recordset" : {         "properties" : {           "coalesceentity" : {             "properties" : {               "datecreated" : {                 "type" : "date"               },               "lastmodified" : {                 "type" : "date"               },               "name" : {                 "type" : "keyword"               },               "objectkey" : {                 "type" : "keyword"               },               "source" : {                 "type" : "keyword"               },               "version" : {                 "type" : "keyword"               }             }           },           "eventrecordset" : {             "properties" : {               "actiongeoadm1code" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "actiongeoadm2code" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "actiongeocountrycode" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "actiongeofeatureid" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "actiongeofullname" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "actiongeolocation" : {                 "type" : "geo_point"               },               "actiongeotype" : {                 "type" : "integer"               },               "avgtone" : {                 "type" : "float"               },               "datasource" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "dateadded" : {                 "type" : "integer"               },               "datetime" : {                 "type" : "date"               },               "day" : {                 "type" : "integer"               },               "eventbasecode" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "eventcode" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "eventrootcode" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "fractiondate" : {                 "type" : "float"               },               "globaleventid" : {                 "type" : "integer"               },               "goldsteinscale" : {                 "type" : "float"               },               "isrootevent" : {                 "type" : "integer"               },               "issimulation" : {                 "type" : "boolean"               },               "monthyear" : {                 "type" : "integer"               },               "numarticles" : {                 "type" : "integer"               },               "nummentions" : {                 "type" : "integer"               },               "numsources" : {                 "type" : "integer"               },               "ontologyreference" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "pmesiipteconomic" : {                 "type" : "float"               },               "pmesiiptinformation" : {                 "type" : "float"               },               "pmesiiptinfrastructure" : {                 "type" : "float"               },               "pmesiiptmilitary" : {                 "type" : "float"               },               "pmesiiptphysicalenvironment" : {                 "type" : "float"               },               "pmesiiptpolitical" : {                 "type" : "float"               },               "pmesiiptsocial" : {                 "type" : "float"               },               "pmesiipttime" : {                 "type" : "float"               },               "quadclass" : {                 "type" : "integer"               },               "sourceurl" : {                 "type" : "text",                 "fields" : {                   "keyword" : {                     "type" : "keyword",                     "ignore_above" : 256                   }                 }               },               "tags" : {                 "type" : "text"               },               "year" : {                 "type" : "integer"               }             }           }         }       }     },     "settings" : {       "index" : {         "creation_date" : "1525982572184",         "number_of_shards" : "5",         "number_of_replicas" : "1",         "uuid" : "t2Ww1t3eQbuyRca5fve8ZQ",         "version" : {           "created" : "5040099"         },         "provided_name" : "coalesce-oeevent"       }     }   } } On 5/18/18 8:35 AM, sjudeng wrote: It should. Filtering isn't supported on nested geometries (it's okay for the |object| type). But this looks like an issue with feature reading not filtering. Can you post your Elasticsearch mapping (or a snippet including the geo and a couple of the other fields)? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#69 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADslzJumQpeGQvc07dQMJC1dvZYWxy56ks5tzsAUgaJpZM4T-x3C>.

-- ========= mailto:[email protected] ============ David W. Boyd VP, Data Solutions 10432 Balls Ford, Suite 240 Manassas, VA 20109 office: [phone] cell: [phone] ============== http://www.incadencecorp.com/ ============ ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture Chair ANSI/INCITS TC Big Data Co-chair NIST Big Data Public Working Group Reference Architecture First Robotic Mentor - FRC, FTC - www.iliterobotics.org Board Member- USSTEM Foundation - www.usstem.org The information contained in this message may be privileged and/or confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and deleting the material from any computer.

[sjudeng]

I have not been able to reproduce this yet. I took the mapping and sample document you provided and added them to ES 5.4.3.

curl -XPUT http://localhost:9200/coalesce-oeevent
curl -XPUT -H 'content-type: application/json' http://localhost:9200/coalesce-oeevent/_mapping/recordset -d @mapping.json
curl -XPOST -H 'content-type: application/json' http://localhost:9200/coalesce-oeevent/recordset -d @doc.json

I was then able to add the store and publish the layer in GeoServer. Fields in WFS (GML/GeoJSON) results were populated with expected non-null values.

curl "http://localhost:8080/geoserver/test/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=test:recordset&maxFeatures=50&outputFormat=json"

{
  "crs": {
    "type": "name", 
    "properties": {
      "name": "urn:ogc:def:crs:EPSG::4326"
    }
  }, 
  "totalFeatures": 1, 
  "type": "FeatureCollection", 
  "features": [
    {
      "geometry": {
        "type": "Point", 
        "coordinates": [
          -71.53140259, 
          42.23730087
        ]
      }, 
      "properties": {
        "coalesceentity.version": "0.2", 
        "eventrecordset.actiongeofeatureid": "MA", 
        "_score": 1, 
        "eventrecordset.ontologyreference.fields.keyword": "Event", 
        "eventrecordset.eventbasecode": "042", 
        "eventrecordset.dateadded": 0, 
        "eventrecordset.actiongeoadm2code.fields.keyword": "", 
        "eventrecordset.actiongeoadm1code.fields.keyword": "USMA", 
        "_aggregation": null, 
        "coalesceentity.source": "G2Core", 
        "eventrecordset.pmesiipttime": 0.25, 
        "eventrecordset.eventrootcode": "04", 
        "_relative_score": 1, 
        "coalesceentity.name": "OEEvent", 
        "eventrecordset.actiongeocountrycode.fields.keyword": "US", 
        "eventrecordset.pmesiiptmilitary": 0.25, 
        "eventrecordset.datasource.fields.keyword": "GDELT", 
        "eventrecordset.day": 20180311, 
        "eventrecordset.eventcode": "042", 
        "eventrecordset.pmesiiptinfrastructure": 0.25, 
        "eventrecordset.avgtone": -4.989154, 
        "eventrecordset.pmesiiptpolitical": 0.5, 
        "eventrecordset.actiongeofeatureid.fields.keyword": "MA", 
        "eventrecordset.numarticles": 10, 
        "eventrecordset.numsources": 1, 
        "eventrecordset.actiongeofullname.fields.keyword": "Massachusetts, United States", 
        "_type": "recordset", 
        "eventrecordset.issimulation": false, 
        "eventrecordset.nummentions": 10, 
        "eventrecordset.tags": null, 
        "eventrecordset.globaleventid": 737769639, 
        "eventrecordset.ontologyreference": "Event", 
        "eventrecordset.datasource": "GDELT", 
        "eventrecordset.actiongeofullname": "Massachusetts, United States", 
        "eventrecordset.monthyear": 201803, 
        "coalesceentity.objectkey": "2c639634-7fd3-47bb-893d-120feb492ac9", 
        "eventrecordset.eventrootcode.fields.keyword": "04", 
        "eventrecordset.sourceurl.fields.keyword": "http://nhpr.org/post/winter-storm-watch-tuesday-nhs-town-meeting-day", 
        "coalesceentity.lastmodified": "2018-03-11T15:11:33.996Z", 
        "eventrecordset.sourceurl": "http://nhpr.org/post/winter-storm-watch-tuesday-nhs-town-meeting-day", 
        "eventrecordset.actiongeotype": 2, 
        "eventrecordset.actiongeocountrycode": "US", 
        "eventrecordset.year": 2018, 
        "eventrecordset.quadclass": 1, 
        "eventrecordset.datetime": "2018-03-11T14:15:00Z", 
        "eventrecordset.pmesiiptinformation": 0.5, 
        "eventrecordset.actiongeoadm2code": "", 
        "eventrecordset.actiongeoadm1code": "USMA", 
        "eventrecordset.isrootevent": 0, 
        "eventrecordset.pmesiipteconomic": 0.25, 
        "_index": "coalesce-oeevent", 
        "eventrecordset.pmesiiptphysicalenvironment": 0.5, 
        "coalesceentity.datecreated": "2018-03-11T15:11:33.985Z", 
        "eventrecordset.goldsteinscale": 1.9, 
        "eventrecordset.eventcode.fields.keyword": "042", 
        "eventrecordset.eventbasecode.fields.keyword": "042", 
        "eventrecordset.pmesiiptsocial": 0.5, 
        "_id": "AWOfnRqCqTF0HjaY37Wm", 
        "eventrecordset.fractiondate": 2018.1945
      }, 
      "type": "Feature", 
      "id": "recordset.AWOfnRqCqTF0HjaY37Wm", 
      "geometry_name": "eventrecordset.actiongeolocation"
    }
  ]
}

Compute from bounds also worked as expected when adding the feature type. Note in practice with larger number of documents it's recommended to manually enter the layer bounds to avoid expensive computation, though it doesn't explain the issue with null values in your features.

[bytemedwb]

Sjudeng:     Thanks.  This is puzzling.   Will look into what is happening.

…

On 5/26/18 11:43 PM, sjudeng wrote: I have not been able to reproduce this yet. I took the mapping <https://github.com/ngageoint/elasticgeo/files/2042248/mapping.txt> and sample document <https://github.com/ngageoint/elasticgeo/files/2042249/doc.txt> you provided and added them to ES 5.4.3. curl -XPUT http://localhost:9200/coalesce-oeevent curl -XPUT -H'content-type: application/json' http://localhost:9200/coalesce-oeevent/_mapping/recordset -d @mapping.json curl -XPOST -H'content-type: application/json' http://localhost:9200/coalesce-oeevent/recordset -d @doc.json I was then able to add the store and publish the layer in GeoServer. Fields in WFS (GML/GeoJSON) results were populated with expected non-null values. curl"http://localhost:8080/geoserver/test/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=test:recordset&maxFeatures=50&outputFormat=json" { "crs": { "type":"name", "properties": { "name":"urn:ogc:def:crs:EPSG::4326" } }, "totalFeatures":1, "type":"FeatureCollection", "features": [ { "geometry": { "type":"Point", "coordinates": [ -71.53140259, 42.23730087 ] }, "properties": { "coalesceentity.version":"0.2", "eventrecordset.actiongeofeatureid":"MA", "_score":1, "eventrecordset.ontologyreference.fields.keyword":"Event", "eventrecordset.eventbasecode":"042", "eventrecordset.dateadded":0, "eventrecordset.actiongeoadm2code.fields.keyword":"", "eventrecordset.actiongeoadm1code.fields.keyword":"USMA", "_aggregation":null, "coalesceentity.source":"G2Core", "eventrecordset.pmesiipttime":0.25, "eventrecordset.eventrootcode":"04", "_relative_score":1, "coalesceentity.name":"OEEvent", "eventrecordset.actiongeocountrycode.fields.keyword":"US", "eventrecordset.pmesiiptmilitary":0.25, "eventrecordset.datasource.fields.keyword":"GDELT", "eventrecordset.day":20180311, "eventrecordset.eventcode":"042", "eventrecordset.pmesiiptinfrastructure":0.25, "eventrecordset.avgtone":-4.989154, "eventrecordset.pmesiiptpolitical":0.5, "eventrecordset.actiongeofeatureid.fields.keyword":"MA", "eventrecordset.numarticles":10, "eventrecordset.numsources":1, "eventrecordset.actiongeofullname.fields.keyword":"Massachusetts, United States", "_type":"recordset", "eventrecordset.issimulation":false, "eventrecordset.nummentions":10, "eventrecordset.tags":null, "eventrecordset.globaleventid":737769639, "eventrecordset.ontologyreference":"Event", "eventrecordset.datasource":"GDELT", "eventrecordset.actiongeofullname":"Massachusetts, United States", "eventrecordset.monthyear":201803, "coalesceentity.objectkey":"2c639634-7fd3-47bb-893d-120feb492ac9", "eventrecordset.eventrootcode.fields.keyword":"04", "eventrecordset.sourceurl.fields.keyword":"http://nhpr.org/post/winter-storm-watch-tuesday-nhs-town-meeting-day", "coalesceentity.lastmodified":"2018-03-11T15:11:33.996Z", "eventrecordset.sourceurl":"http://nhpr.org/post/winter-storm-watch-tuesday-nhs-town-meeting-day", "eventrecordset.actiongeotype":2, "eventrecordset.actiongeocountrycode":"US", "eventrecordset.year":2018, "eventrecordset.quadclass":1, "eventrecordset.datetime":"2018-03-11T14:15:00Z", "eventrecordset.pmesiiptinformation":0.5, "eventrecordset.actiongeoadm2code":"", "eventrecordset.actiongeoadm1code":"USMA", "eventrecordset.isrootevent":0, "eventrecordset.pmesiipteconomic":0.25, "_index":"coalesce-oeevent", "eventrecordset.pmesiiptphysicalenvironment":0.5, "coalesceentity.datecreated":"2018-03-11T15:11:33.985Z", "eventrecordset.goldsteinscale":1.9, "eventrecordset.eventcode.fields.keyword":"042", "eventrecordset.eventbasecode.fields.keyword":"042", "eventrecordset.pmesiiptsocial":0.5, "_id":"AWOfnRqCqTF0HjaY37Wm", "eventrecordset.fractiondate":2018.1945 }, "type":"Feature", "id":"recordset.AWOfnRqCqTF0HjaY37Wm", "geometry_name":"eventrecordset.actiongeolocation" } ] } Compute from bounds also worked as expected when adding the feature type. Note in practice with larger number of documents it's recommended to manually enter the layer bounds to avoid expensive computation, though it doesn't explain the issue with null values in your features. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#69 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADslzKHOr1MfNETxKmntWWqrcD9y5gysks5t2iDYgaJpZM4T-x3C>.

-- ========= mailto:[email protected] ============ David W. Boyd VP, Data Solutions 10432 Balls Ford, Suite 240 Manassas, VA 20109 office: [phone] cell: [phone] ============== http://www.incadencecorp.com/ ============ ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture Chair ANSI/INCITS TC Big Data Co-chair NIST Big Data Public Working Group Reference Architecture First Robotic Mentor - FRC, FTC - www.iliterobotics.org Board Member- USSTEM Foundation - www.usstem.org The information contained in this message may be privileged and/or confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and deleting the material from any computer.

[dclemenzi]

Here are some additional details. Placing a break point on line 313 in ElasticParserUtil here is the content of the source mapping:

source = {LinkedHashMap@7557}  size = 10
 0 = {LinkedHashMap$Entry@7956} "test1.int" -> "10"
 1 = {LinkedHashMap$Entry@7957} "coalesceentity.name" -> "UNIT_TEST"
 2 = {LinkedHashMap$Entry@7958} "test1.geo" -> "-3.18, 51.4347"
 3 = {LinkedHashMap$Entry@7959} "coalesceentity.datecreated" -> "2018-06-07T14:47:57.343Z"
 4 = {LinkedHashMap$Entry@7960} "coalesceentity.source" -> "DSS"
 5 = {LinkedHashMap$Entry@7961} "test1.string" -> "EUROPE"
 6 = {LinkedHashMap$Entry@7962} "test1.date" -> "2018-06-07T14:47:57.374Z"
 7 = {LinkedHashMap$Entry@7963} "coalesceentity.lastmodified" -> "2018-06-07T14:47:57.375Z"
 8 = {LinkedHashMap$Entry@7964} "coalesceentity.version" -> "1.0"
 9 = {LinkedHashMap$Entry@7965} "coalesceentity.objectkey" -> "b4cbb5fa-7106-4b75-a2bc-791f88cc8373"

Line 310 splits the name being passed in so on line 313 its looking for the key coalesceentity instead of coalesceentity.name which results in a null value. If line 313 could pass in the name unmodified the code would work. I don't understand ElasticSearch well enough to know why its splitting the name in the first place. Any guidance would be appreciate.

I tried pulling the source and modifing it but that resulted in a non-shaded jar. How was the 2.13.0 version compiled and deployed?

[dclemenzi]

Don't know if this is a bad approach or not but this code change resolves the issue:

    public List<Object> readField(Map<String, Object> source, String name) {
        final List<String> keys = Arrays.asList(name.split("\\."));
        List<Object> values = new ArrayList<>();
        if (!keys.isEmpty()) {
            readField(source.get(keys.get(0)), keys.subList(1, keys.size()), values);
        }
        final List<Object> result;
        if (!values.isEmpty()) {
            result = values;
        } else {
            result = null;
        }
        return result;
    }

To:

public List<Object> readField(Map<String, Object> source, String name) {
        final List<String> keys = Arrays.asList(name.split("\\."));
        List<Object> values = new ArrayList<>();
        if (!keys.isEmpty()) {

            Object entry = source.get(keys.get(0));

            if (entry == null)
            {
                entry = source.get(name);
            }

            readField(entry, keys.subList(1, keys.size()), values);
        }
        final List<Object> result;
        if (!values.isEmpty()) {
            result = values;
        } else {
            result = null;
        }
        return result;
    }

[sjudeng]

The change looks reasonable, can you add a test and open a PR with the update? Looks like you figured it out but when building locally you'll find the shaded jar in gs-web-elasticsearch/target/elasticgeo-2.13-SNAPSHOT.jar.

[dclemenzi]

Getting a 403 when attempting to push to a branch. Not sure what I am missing.

[sjudeng]

Pushing to your fork or this repo? You'd only be able to push to your fork and then you can open a PR from there. If you haven't already click Fork above and then in your local folder do a git remote add dclemenzi https://github.com/dclemenzi/elasticgeo.git. Then you should be able to do a git push dclemenzi your-branch.

[sjudeng]

Fixed by #71 (thanks @dclemenzi)