imotov/elasticsearch-native-script-example

Example of Now Deprecated Native Script Plugin for Elasticsearch

elasticsearch-native-script-example's Issues

Failed to apply plugin [id 'carrotsearch.randomized-testing']

I'm getting an error when trying to build with gradle. I've never used gradle before so I don't know if I'm doing something wrong.

$ java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

$ gradle -version

------------------------------------------------------------
Gradle 3.5
------------------------------------------------------------

Build time:   2017-04-10 13:37:25 UTC
Revision:     b762622a185d59ce0cfc9cbc6ab5dd22469e18a6

Groovy:       2.4.10
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.8.0_102 (Oracle Corporation 25.102-b14)
OS:           Mac OS X 10.12.4 x86_64

$ gradle build

FAILURE: Build failed with an exception.

* Where:
Build file '/path/to/elasticsearch-native-script-example/build.gradle' line: 18

* What went wrong:
A problem occurred evaluating root project 'elasticsearch-native-script-example'.
> Failed to apply plugin [id 'carrotsearch.randomized-testing']
   > Could not create task of type 'RandomizedTestingTask'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 4.1 secs

Example will not work with latest ElasticSearch 1.4.0

Hi Igor,
Please refer to the issue I submitted here:
elastic/elasticsearch#8377

Thanks,
Andy

building the plugin

The git documentation suggests to build the package using mvn package, but the project has moved to gradle, can you please specify how to build the master project with gradle.

Could not find plugin descriptor 'plugin-descriptor.properties'

I used your source of the 2.0 branch.
For build the project i changed "2.0.0-beta2-SNAPSHOT" to "2.0.0" in the POM and this is the result:

org.elasticsearch.plugin
plugins
2.0.0

<name>elasticsearch-native-script-example</name>
<groupId>org.elasticsearch.plugin</groupId>
<artifactId>elasticsearch-native-script-example</artifactId>
<version>2.0.0</version>
<description>ElasticSearch Plugin with Native Script Examples</description>
<inceptionYear>2013</inceptionYear>

...

The build was successful but when i tried to install the plugin that's happened:
/usr/share/elasticsearch/bin/plugin install file:./target/elasticsearch-native-script-example-2.0.0.jar
-> Installing from file:./target/elasticsearch-native-script-example-2.0.0.jar...
Trying file:./target/elasticsearch-native-script-example-2.0.0.jar ...
Downloading .DONE
Verifying file:./target/elasticsearch-native-script-example-2.0.0.jar checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
ERROR: Could not find plugin descriptor 'plugin-descriptor.properties' in plugin zip

"script_score query does not support [params]"-es 5.3.1

I wanna test cosing similarity of termscoring.sh, but returns error as title. I have google some answers, it seems that this is a es error (but author said it is fixed) I don't know whether this example can run or not in es 5.3.1 version

Confused about versioning

The readme files make this a bit confusing because in master it says use 2.x. But then when you goto 2.x it says use master for "2.1.0 and above".

Is master for the latest ES 2.x (2.3.4) release or is it for ES 5.0 (might have been named ES3.0 when you last updated the project)?

how to debug this project using idea

how to debug this project using idea

question about script window_size

Hey @imotov

Thanks for your help before! Have one more question, in your script
https://github.com/imotov/elasticsearch-native-script-example/blob/master/example/termscoring.sh#L64
is there a way to add a window_size for reranking within each shard? In regular "rescore" part, we can always add a "window_size": 100 to restrict the data points being ranked within each shard. Is there a way to add such a parameter in "script_score" ? Thanks!

plugin.xml link is broken

plugin.xml link is broken in the README

There is no example of using Settings in native scripts after moving to 5.3

It's absolutely unclear how to create ScriptPlugin able to read settings.
There was an example of using AbstractComponent, but it's not working in 5.3

Search scope parameters

Hi Igor,

Thanks for this input. I'm wandering in Elasticsearch capabilities, and think I can use Native scripts to cover my specific requirements. I have a couple of questions regarding this :

• Is there a way I can access to the request itself ? For example, access to the searched terms for a Term Query ?
• Is there a way I can load an entry from a type indexed in Elasticsearch that I would use as a configuration ?

Thanks for your support.

Regards,
Yann

porting to script engine

HI @imotov
Is there any chance that these examples will be ported to script engine?

Native plugin script under nested

Hi imotov,

I went through your examples, but wasn't able to find one where you use a script for a nested document under 'nested'. The challenge I am facing there is that the source()/doc() fetch gives me 0 results even though I do expect matches. I believe that may have to do with nested documents being stored as a separate document, but I am not sure.

Would appreciate any help! Thanks for writing up all these examples! :)

group by + Consolidation + Filter + sort

hi, I would like to complete this operation by elasticsearch-native-script-Java

data：

    {
        "city": "Los Angeles",
        "name": "1",
        "time": "1464240096"
    }   

    {
        "city": "Los Angeles",
        "name": "2",
        "time": "1464240096"
    }  

    {
        "city": "Los Angeles",
        "name": "3",
        "time": "1464240096"
    }  

    {
        "city": "Los Angeles",
        "name": "4",
        "time": "1464240096"
    } 

    {
        "city": "Los Angeles",
        "name": "5",
        "time": "1464240096"
    } 

    {
        "city": "Los Angeles",
        "name": "6",
        "time": "1464240096"
    } 

    {
        "city": "Los Angeles",
        "name": "7",
        "time": "1464240096"
    } 

    {
        "city": "Los Angeles",
        "name": "8",
        "time": "1464240096"
    }  

    {
        "city": "Los Angeles",
        "name": "9",
        "time": "1464240096"
    } 

    {
        "city": "Los Angeles",
        "name": "10",
        "time": "1464240096"
    }  

    {
        "city": "Los Angeles",
        "name": "11",
        "time": "1464240096"
    } 

    {
        "city": "Los Angeles",
        "name": "12",
        "time": "1464240096"
    } 

    {
        "city": "new York",
        "name": "21",
        "time": "1464240091"
    }  

    {
        "city": "new York",
        "name": "22",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "23",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "24",
        "time": "1464240091"
    }  

    {
        "city": "new York",
        "name": "25",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "26",
        "time": "1464240091"
    }  

    {
        "city": "new York",
        "name": "27",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "28",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "29",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "30",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "31",
        "time": "1464240091"
    } 

    {
        "city": "new York",
        "name": "32",
        "time": "1464240091"
    }

then return：

[
    {
        "city": "new York",
        "name": "21",
        "time": "1464240091",
        "names": [
            "1",
            "2",
            "3",
            "4",
            "5",
            "6",
            "7",
            "8",
            "9",
            "10",
            "11",
            "12"
        ]
    },
    {
        "city": "Los Angeles",
        "name": "1",
        "time": "1464240096",
        "names": [
            "21",
            "22",
            "23",
            "24",
            "25",
            "26",
            "27",
            "28",
            "29",
            "30",
            "31",
            "32"
        ]
    }
]

the ultimate result I need meets :
firstly: sorted by time ，
secondly ：merge into 2 records finally

I found the method is not perfect，
1，sort by time is not supported

{
    "query": {
        "bool": {
            "must": [
                {
                    "match_all": {}
                }
            ],
            "must_not": [],
            "should": []
        }
    },
    "aggs": {
        "city_groupBy": {
            "terms": {
                "field": "city"
            },
            "aggs": {
                "profit": {
                    "scripted_metric": {
                        "init_script": "_agg['names'] = [];",
                        "map_script": "_agg.names.add(doc['time'].value);",
                        "reduce_script": " return _aggs"
                    }
                },
                "max_time": {
                    "max": {
                        "field": "time"
                    }
                }
            }
        }
    },
    "from": 0,
    "size": 0,
    "sort": [],
    "facets": {}
}

LookupScript ，Can not be filtered

please help me @imotov

question compiling

Hello

When we download it and try to compile, we got the following error. Is there a way to fix it? Thanks!

[ERROR] Failed to execute goal on project elasticsearch-native-script-example: Could not resolve dependencies for project org.elasticsearch.plugin:elasticsearch-native-script-example:jar:3.0.0-SNAPSHOT: Failed to collect dependencies at org.elasticsearch:elasticsearch:jar:tests:3.0.0-SNAPSHOT -> org.apache.lucene:lucene-grouping:jar:5.5.0-snapshot-1719088: Failed to read artifact descriptor for org.apache.lucene:lucene-grouping:jar:5.5.0-snapshot-1719088: Could not transfer artifact org.apache.lucene:lucene-grouping:pom:5.5.0-snapshot-1719088 from/to elasticsearch-releases (http://maven.elasticsearch.org/releases): Not authorized , ReasonPhrase:Unauthorized. -> [Help 1]

how to install native script plugin

hi,

im created native script using java but im not able to install as plugin
how to install as plugin in elasticsearch

Elasticsearch 2.0 scripting: Invoke custom script from plugin

Can someone help me with this question.
If we defined native script in plugin: com.test.nativescript.TestScript, can we invoke that package inside another groovy script and create new instance ?

Ex:
inside config/scripts/test.groovy:

import com.test.nativescript.TestScript;

def testScript = new TestScript();
Result: "startup failed:\r\nc78a82bc96ca7751e83e256b4ee0c322cda13bea: 1: unable to resolve com.test.nativescript.TestScript"

Missing pom.xml in Master branch

From README, there is a link to pom.xml file that point to https://github.com/imotov/elasticsearch-native-script-example/blob/master/pom.xml but this file is not exist in master branch.

Sample native scripted metric aggrgation

It would be useful if you could accommodate sample java code which can do scripted metric aggregation

example of custom native script with query string

This is one of the best example I found online for writing custom script using the native script support in elasticsearch. Thanks for the good work :)

Can you add some examples on how to use the custom native script with a search query ? I wrote a custom native script based on your example, it works only with match_all query, when i include query_string it doesnt work.

How to lookup a date field?

Any chance someone could add an example on working with date fields?

There isn't a lot of documentation for building native plugins outside of this project. ScriptDocValues doesn't contain a .Date class. Some other stale examples suggested it can be casted to a Longs, someone said it didn't work, and then the trail went dead.

Any pointers would be most appreciated!

Differenciate plugin version and ES parent plugin version

Hi @imotov,

Is there an easy way to differentiate ES parent plugin version and plugin version ?

Thanks.
Yann