Search is broken: always comes up empty

[dscho]

Search is broken: always comes up empty

[pedrorijo91]

i think there is a cron job who adds stuff to the database, which may have failed recently

could it be due to the update in heroku @ttaylorr ?

[ttaylorr]

Thanks for reporting. After some looking, it seems that we do have data in the Bonsai cluster, but that something (likely to do with the Heroku stack upgrade from a few days ago) broke things:

Of course, we eat StandardError, so it's not immediately obvious what's going on when you run Doc.search("config") in the rails console. But open-coding Searchable#search gives us a little more information:

irb(main):046:0> client.search index: ELASTIC_SEARCH_INDEX, body: query_options
/app/vendor/bundle/ruby/3.1.0/gems/elasticsearch-transport-2.0.2/lib/elasticsearch/transport/transport/base.rb:202:in `__raise_transport_error': [406] {"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406} (Elasticsearch::Transport::Transport::Errors::NotAcceptable)

@pedrorijo91 Do you know what to do here?

[ttaylorr]

...annnd it works again:

Some of the details are in PRs #1869, #1870, and #1871. But the gist is that we had to upgrade the elasticsearch gem not quite to the latest version, but to the highest version that still supports Bonsai, which is not an officially supported ES backend. Then we had to tweak some of our query parameters, which was done in #1871.

Finally, it looks like either Bonsai dropped our data, or my earlier rake search_clear run dropped the existing data, but the subsequent rake search_index did not re-populate it as it was running prior to #1869.

So I ran one more round of:

for task in search_clear search_index search_index_book
do
  heroku run -a git-scm "rake $task" || return 1
done

which landed that data back into Bonsai, and validated that search works again. Thanks for reporting, @dscho!