Comments (10)
This seems to be the error:
[2019-04-01 12:10:03][INFO] Job caused exception mapper [downloads] of different type, current_type [long], merged_type [text] in /var/www/mysite/releases/f62b7d0443bd65b2e0e71cb04651b00d73f80b41/vendor/ruflin/elastica/lib/Elastica/Transport/Guzzle.php at line 101
[2019-04-01 12:10:03][INFO] Defining the mappings
ERROR [Emergency]: Uncaught Elastica\Exception\ResponseException: mapper [downloads] of different type, current_type [long], merged_type [text]
IN GET dev/tasks/ProcessJobQueueTask
Line 101 in /var/www/mysite/releases/f62b7d0443bd65b2e0e71cb04651b00d73f80b41/vendor/ruflin/elastica/lib/Elastica/Transport/Guzzle.php
Source
======
92:
93: $response->setTransferInfo(
94: [
95: 'request_header' => $request->getMethod(),
96: 'http_code' => $res->getStatusCode(),
97: ]
98: );
99:
100: if ($response->hasError()) {
* 101: throw new ResponseException($request, $response);
102: }
103:
104: if ($response->hasFailedShards()) {
105: throw new PartialShardFailureException($request, $response);
106: }
107:
Trace
=====
Elastica\Transport\Guzzle->exec({"path":"ss_mysite\/Addon\/_mapping","method":"PUT","data":{"Addon":{"properties":{"name":{"type":"string"},"description":{"type":"string"},"type":{"type":"string"},"compatibility":{"type":"string"},"vendor":{"type":"string"},"tags":{"type":"string"},"released":{"type":"date"},"downloads":{"type":"string"},"readme":{"type":"string"}}}},"query":[],"contentType":"application\/json","connection":{"config":{"url":null,"bigintConversion":false},"host":"search-cloudy-ssaddon1-prod2xes-toi7xpgnfuyoa3x7zkipva7j5q.ap-southeast-2.es.amazonaws.com","port":80,"path":null,"proxy":null,"transport":"AwsAuthV4","persistent":true,"timeout":5,"connections":[],"roundRobin":false,"log":false,"retryOnConflict":0,"username":null,"password":null,"aws_region":"ap-southeast-2","enabled":true}}, Array)
Request.php:193
Elastica\Request->send()
Client.php:674
Elastica\Client->request(ss_mysite/Addon/_mapping, PUT, Array, Array)
Client.php:706
Elastica\Client->requestEndpoint(Elasticsearch\Endpoints\Indices\Mapping\Put)
Index.php:562
Elastica\Index->requestEndpoint(Elasticsearch\Endpoints\Indices\Mapping\Put)
Type.php:523
Elastica\Type->requestEndpoint(Elasticsearch\Endpoints\Indices\Mapping\Put)
Mapping.php:250
Elastica\Type\Mapping->send()
ElasticaService.php:218
Heyday\Elastica\ElasticaService->define()
ReindexTask.php:41
Heyday\Elastica\ReindexTask->run(SilverStripe\Control\HTTPRequest)
RunBuildTaskJob.php:80
Symbiote\QueuedJobs\Jobs\RunBuildTaskJob->process()
QueuedJobService.php:723
Symbiote\QueuedJobs\Services\QueuedJobService->runJob(568)
QueuedJobService.php:1117
Symbiote\QueuedJobs\Services\QueuedJobService->processJobQueue(2)
QueueRunner.php:23
Symbiote\QueuedJobs\Tasks\Engines\QueueRunner->runQueue(2)
QueuedJobService.php:1094
Symbiote\QueuedJobs\Services\QueuedJobService->runQueue(2)
ProcessJobQueueTask.php:59
Symbiote\QueuedJobs\Tasks\ProcessJobQueueTask->run(SilverStripe\Control\HTTPRequest)
TaskRunner.php:104
SilverStripe\Dev\TaskRunner->runTask(SilverStripe\Control\HTTPRequest)
RequestHandler.php:323
SilverStripe\Control\RequestHandler->handleAction(SilverStripe\Control\HTTPRequest, runTask)
Controller.php:284
SilverStripe\Control\Controller->handleAction(SilverStripe\Control\HTTPRequest, runTask)
RequestHandler.php:202
SilverStripe\Control\RequestHandler->handleRequest(SilverStripe\Control\HTTPRequest)
Controller.php:212
SilverStripe\Control\Controller->handleRequest(SilverStripe\Control\HTTPRequest)
RequestHandler.php:226
SilverStripe\Control\RequestHandler->handleRequest(SilverStripe\Control\HTTPRequest)
Controller.php:212
SilverStripe\Control\Controller->handleRequest(SilverStripe\Control\HTTPRequest)
Director.php:361
SilverStripe\Control\Director->SilverStripe\Control\{closure}(SilverStripe\Control\HTTPRequest)
VersionedHTTPMiddleware.php:41
SilverStripe\Versioned\VersionedHTTPMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
BasicAuthMiddleware.php:68
SilverStripe\Security\BasicAuthMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
AuthenticationMiddleware.php:61
SilverStripe\Security\AuthenticationMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
CanonicalURLMiddleware.php:188
SilverStripe\Control\Middleware\CanonicalURLMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
HTTPCacheControlMiddleware.php:42
SilverStripe\Control\Middleware\HTTPCacheControlMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
ChangeDetectionMiddleware.php:27
SilverStripe\Control\Middleware\ChangeDetectionMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
FlushMiddleware.php:29
SilverStripe\Control\Middleware\FlushMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
RequestProcessor.php:66
SilverStripe\Control\RequestProcessor->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
SessionMiddleware.php:20
SilverStripe\Control\Middleware\SessionMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
AllowedHostsMiddleware.php:60
SilverStripe\Control\Middleware\AllowedHostsMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
TrustedProxyMiddleware.php:176
SilverStripe\Control\Middleware\TrustedProxyMiddleware->process(SilverStripe\Control\HTTPRequest, Closure)
HTTPMiddlewareAware.php:62
SilverStripe\Control\Director->SilverStripe\Control\Middleware\{closure}(SilverStripe\Control\HTTPRequest)
HTTPMiddlewareAware.php:65
SilverStripe\Control\Director->callMiddleware(SilverStripe\Control\HTTPRequest, Closure)
Director.php:370
SilverStripe\Control\Director->handleRequest(SilverStripe\Control\HTTPRequest)
HTTPApplication.php:48
SilverStripe\Control\HTTPApplication->SilverStripe\Control\{closure}(SilverStripe\Control\HTTPRequest)
call_user_func(Closure, SilverStripe\Control\HTTPRequest)
HTTPApplication.php:66
SilverStripe\Control\HTTPApplication->SilverStripe\Control\{closure}(SilverStripe\Control\HTTPRequest)
HTTPMiddlewareAware.php:65
SilverStripe\Control\HTTPApplication->callMiddleware(SilverStripe\Control\HTTPRequest, Closure)
HTTPApplication.php:67
SilverStripe\Control\HTTPApplication->execute(SilverStripe\Control\HTTPRequest, Closure, )
HTTPApplication.php:47
SilverStripe\Control\HTTPApplication->handle(SilverStripe\Control\HTTPRequest)
cli-script.php:22
[2019-04-01 12:10:03][INFO]
[2019-04-01 12:10:03][INFO] Job paused at 2019-04-01 12:10:03
from addons.silverstripe.org.
In theory this could be fixed by changing this to "long" or something, but it's been a string since the SS3 version too, which is a big confusing.
Perhaps nuke the index and reindex?
from addons.silverstripe.org.
I'm running 5.6.16 locally and not having this issue. Will try and find out exactly which version SSP is running to see if there's a discrepancy.
from addons.silverstripe.org.
I've tried recreating a new index. I can't see a way to clear the current index. That didn't help anyway, so I'll look at adjusting the type instead.
from addons.silverstripe.org.
"long" feels like a more correct type for an Int than "string", in any case.
from addons.silverstripe.org.
I've had the index reset, it's still evaluating to "long", and ES is trying to save it as a string. My local environment's index looks the same as the UAT environment, but the production environment looks different.
I don't know enough about this, for now I'm going to try setting the downloads field to be indexed as a string via config.
Addon::getElasticaMapping()
has explicit casting for each field, but it looks like this isn't being used on production. The elastica module is probably being allowed to introspect the DB types and make its own decisions, which I assume the presence of the getElasticaMapping()
method should prevent. I don't know why this happens on production and not on UAT (and local) which would explain the difference in the indexes.
from addons.silverstripe.org.
Here's a summary of where I'm at:
- Local and UAT works fine
- Production has
downloads
defined aslong
- The code mapping is for
downloads
to be astring
- The error happens when trying to recreate the index with the index mapping we define, because a previous index spec already exists which it conflicts with
Failing call to set new index mapping:
Elastica\Transport\Guzzle->exec({"path":"ss_mysite\/Addon\/_mapping","method":"PUT","data":{"Addon":{"properties":{"name":{"type":"string"},"description":{"type":"string"},"type":{"type":"string"},"compatibility":{"type":"string"},"vendor":{"type":"string"},"tags":{"type":"string"},"released":{"type":"date"},"downloads":{"type":"string"},"readme":{"type":"string"}}}},"query":[],"contentType":"application\/json","connection":{"config":{"url":null,"bigintConversion":false},"host":"MYELASTICHOSTHERE","port":80,"path":null,"proxy":null,"transport":"AwsAuthV4","persistent":true,"timeout":5,"connections":[],"roundRobin":false,"log":false,"retryOnConflict":0,"username":null,"password":null,"aws_region":"ap-southeast-2","enabled":true}}, Array)
Request.php:193
Note "downloads":{"type":"string"}
above.
Production ES index spec:
{
"ss_mysite": {
"aliases": {},
"mappings": {
"Addon": {
"properties": {
"SS_Published": {
"type": "boolean"
},
"compatibility": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"description": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"downloads": {
"type": "long"
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"readme": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"released": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"tags": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"vendor": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"creation_date": "1554338601480",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "qMeCpAhgR_SyptSOimjWZA",
"version": {
"created": "5060899"
},
"provided_name": "ss_mysite"
}
}
}
}
UAT/local index spec:
{
"ss_mysite": {
"aliases": {},
"mappings": {
"Addon": {
"properties": {
"SS_Published": {
"type": "boolean"
},
"compatibility": {
"type": "text"
},
"description": {
"type": "text"
},
"downloads": {
"type": "text"
},
"name": {
"type": "text"
},
"readme": {
"type": "text"
},
"released": {
"type": "date"
},
"tags": {
"type": "text"
},
"type": {
"type": "text"
},
"vendor": {
"type": "text"
}
}
}
},
"settings": {
"index": {
"creation_date": "1551313022147",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "wXdfr8TySb-R6KiVxVLUJw",
"version": {
"created": "5060899"
},
"provided_name": "ss_mysite"
}
}
}
}
@stogj manually cleared the index for me and said that when he checked it again it still had an index definition. It should have responded with something like this:
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index",
"resource.type": "index_or_alias",
"resource.id": "addons",
"index_uuid": "_na_",
"index": "addons"
}
],
"type": "index_not_found_exception",
"reason": "no such index",
"resource.type": "index_or_alias",
"resource.id": "addons",
"index_uuid": "_na_",
"index": "addons"
},
"status": 404
}
Nevertheless, he did something and I tried again. The reindex task then worked successfully, and when I refreshed to run it again (via browser) it reverted back to failing again.
It feels like something is interjecting and changing the index spec outside of the application layer (to me).
I'm going to try switching the mapping to a long
explicitly instead.
from addons.silverstripe.org.
Cool. If this fixes it, it might be worth filing some upstream issues relating to changing index types.
from addons.silverstripe.org.
Alright well now I'm getting the inverse error (string vs long) on UAT from this change. At least I can reproduce the issue locally now (addons
is my index name):
curl -XDELETE http://localhost:9200/addons
- Run
dev/tasks/Heyday-Elastica-ReindexTask
to set initial index spec $ curl -s http://localhost:9200/addons/Addon/_mapping | jq .addons.mappings.Addon.properties.downloads
(type: string)- Edit
Addon::getElasticaMapping()
and changedownloads
from string to long - Run
dev/tasks/Heyday-Elastica-ReindexTask
to update index spec (see error)
For whatever reason the ES infrastructure in production isn't clearing correctly, but when reindexing ES from the app layer it should delete the entire index including mappings. I'll see if I can adjust the module to do this.
from addons.silverstripe.org.
Ok I've raised the issue at WPP-Public/akqa-nz-silverstripe-elastica#19 and made a patch PR at #243, and have fixed this in our project with #243 so it deletes the existing index every time it goes to reindex it.
Tested on UAT and deployed to production, working. Happy to close this now.
from addons.silverstripe.org.
Related Issues (20)
- BuildQueued not to be trusted HOT 1
- Page showing what SilverStripe integrates with
- Error: Class 'HTTPCacheControl' not found HOT 1
- Error: E_USER_WARNING: None of the following templates could be found: Array ( [0] => ErrorPage_404 [1] => ErrorPage [2] => Page ) in themes "Array ( [0] => silverstripe/admin:cms-forms [1] => $default ) "
- Error: E_WARNING: count(): Parameter must be an array or an object that implements Countable
- Replace SiteErrorPageFilter with a middleware
- Configure temporary module storage location to be within the SSP shared directories HOT 1
- Better defining the interoperability of supported modules
- Core version compatibility filter doesn't work with recipe constraints (rather than framework) HOT 4
- Addons doesn't list recipes
- Investigate RDS utilisation HOT 2
- Duplicate screenshots created on each update, filling up server.
- Update to composer v2 HOT 4
- Could not parse version constraint ^: Invalid version string "^" HOT 5
- BuildAddonJob pulls the wrong package?! HOT 1
- Search appears to be broken HOT 1
- Travis build badges still point to travisci.org (is broken) HOT 3
- Addons data not updating HOT 4
- Repositories with main branch HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from addons.silverstripe.org.