nathanl/searchlight

Searchlight helps you build searches from options via Ruby methods that you write.

Searchlight

Status

I consider searchlight "done". It has no production dependencies, so there's no reason it shouldn't work indefinitely. I've also moved on to other things.

If you find a bug, feel free to open an issue so others can find it and discuss, but I'm unlikely to respond personally. If Searchlight doesn't meet your needs anymore, fork away! :)

Description

Searchlight is a low-magic way to build database searches using an ORM.

Searchlight can work with any ORM or object that can build a query using chained method calls (eg, ActiveRecord's .where(...).where(...).limit(...), or similar chains with Sequel, Mongoid, etc).

Getting Started

A demo app and the code for that app are available to help you get started.

Overview

Searchlight's main use is to support search forms in web applications.

Searchlight doesn't write queries for you. What it does do is:

• Give you an object with which you can build a search form (eg, using form_for in Rails)
• Give you a sensible place to put your query logic
• Decide which parts of the search to run based on what the user submitted (eg, if they didn't fill in a "first name", don't do the WHERE first_name = part)

For example, if you have a Searchlight search class called YetiSearch, and you instantiate it like this:

  search = YetiSearch.new(
    # or params[:yeti_search]
    "active" => true, "name" => "Jimmy", "location_in" => %w[NY LA]
  )

... calling search.results will build a search by calling the methods search_active, search_name, and search_location_in on your YetiSearch, assuming that you've defined them. (If you do it again but omit "name", it won't call search_name.)

The results method will then return the return value of the last search method. If you're using ActiveRecord, this would be an ActiveRecord::Relation, and you can then call each to loop through the results, to_sql to get the generated query, etc.

Usage

Search class

A search class has two main parts: a base_query and some search_ methods. For example:

class PersonSearch < Searchlight::Search

  # This is the starting point for any chaining we do, and it's what
  # will be returned if no search options are passed.
  # In this case, it's an ActiveRecord model.
  def base_query
    Person.all # or `.scoped` for ActiveRecord 3
  end

  # A search method.
  def search_first_name
    # If `"first_name"` was the first key in the options_hash,
    # `query` here will be the base query, namely, `Person.all`.
    query.where(first_name: options[:first_name])
  end

  # Another search method.
  def search_last_name
    # If `"last_name"` was the second key in the options_hash,
    # `query` here will be whatever `search_first_name` returned.
    query.where(last_name: last_name)
  end
end

Calling PersonSearch.new("first_name" => "Gregor", "last_name" => "Mendel").results would run Person.all.where(first_name: "Gregor").where(last_name: "Mendel") and return the resulting ActiveRecord::Relation. If you omitted the last_name option, or provided "last_name" => "", the second where would not be added.

Here's a fuller example search class. Note that because Searchlight doesn't write queries for you, you're free to do anything your ORM supports. (See spec/support/book_search.rb for even more fanciness.)

# app/searches/city_search.rb
class CitySearch < Searchlight::Search

  # `City` here is an ActiveRecord model
  def base_query
    City.includes(:country)
  end

  # Reach into other tables
  def search_continent
    query.where('`countries`.`continent` = ?', continent)
  end

  # Other kinds of queries
  def search_country_name_like
    query.where("`countries`.`name` LIKE ?", "%#{country_name_like}%")
  end

  # .checked? considers "false", 0 and "0" to be false
  def search_is_megacity
    query.where("`cities`.`population` #{checked?(is_megacity) ? '>=' : '<'} ?", 10_000_000)
  end

end

Here are some example searches.

CitySearch.new.results.to_sql
  # => "SELECT `cities`.* FROM `cities` "
CitySearch.new("name" => "Nairobi").results.to_sql
  # => "SELECT `cities`.* FROM `cities`  WHERE `cities`.`name` = 'Nairobi'"

CitySearch.new("country_name_like" =>  "aust", "continent" => "Europe").results.count # => 6

non_megas = CitySearch.new("is_megacity" => "false")
non_megas.results.to_sql 
  # => "SELECT `cities`.* FROM `cities`  WHERE (`cities`.`population` < 10000000"
non_megas.results.each do |city|
  # ...
end

Option Readers

For each search method you define, Searchlight will define a corresponding option reader method. Eg, if you add def search_first_name, your search class will get a .first_name method that returns options["first_name"] or, if that key doesn't exist, options[:first_name]. This is useful mainly when building forms.

Since it considers the keys "first_name" and :first_name to be interchangeable, Searchlight will raise an error if you supply both.

Examining Options

Searchlight provides some methods for examining the options provided to your search.

• raw_options contains exactly what it was instantiated with
• options contains all raw_options that weren't empty?. Eg, if raw_options is categories: nil, tags: ["a", ""], options will be tags: ["a"].
• empty?(value) returns true for nil, whitespace-only strings, or anything else that returns true from value.empty? (eg, empty arrays)
• checked?(value) returns a boolean, which mostly works like !!value but considers 0, "0", and "false" to be false

Finally, explain will tell you how Searchlight interpreted your options. Eg, book_search.explain might output:

Initialized with `raw_options`: ["title_like", "author_name_like", "category_in",
"tags", "book_thickness", "parts_about_lolcats"]

Of those, the non-blank ones are available as `options`: ["title_like",
"author_name_like", "tags", "book_thickness", "in_print"]

Of those, the following have corresponding `search_` methods: ["title_like",
"author_name_like", "in_print"]. These would be used to build the query.

Blank options are: ["category_in", "parts_about_lolcats"]

Non-blank options with no corresponding `search_` method are: ["tags",
"book_thickness"]

Defining Defaults

Sometimes it's useful to have default search options - eg, "orders that haven't been fulfilled" or "houses listed in the last month".

This can be done by overriding options. Eg:

class BookSearch < SearchlightSearch

  # def base_query...

  def options
    super.tap { |opts|
      opts["in_print"] ||= "either"
    }
  end

  def search_in_print
    return query if options["in_print"].to_s == "either"
    query.where(in_print: checked?(options["in_print"]))
  end

end

Subclassing

You can subclass an existing search class and support all the same options with a different base query. This may be useful for single table inheritance, for example.

class VillageSearch < CitySearch
  def base_query
    Village.all
  end
end

Or you can use super to get the superclass's base_query value and modify it:

class SmallTownSearch < CitySearch
  def base_query
    super.where("`cities`.`population` < ?", 1_000)
  end
end

Custom Options

You can provide a Searchlight search any options you like; only those with a matching search_ method will determine what methods are run. Eg, if you want to do AccountSearch.new("super_user" => true) to find restricted results, just ensure that you check options["super_user"] when building your query.

Usage in Rails

ActionView adapter

Searchlight plays nicely with Rails forms - just include the ActionView adapter as follows:

require "searchlight/adapters/action_view"

class MySearch < Searchlight::Search
  include Searchlight::Adapters::ActionView

  # ...etc
end

This will enable using a Searchlight::Search with form_for:

# app/views/cities/index.html.haml
...
= form_for(@search, url: search_cities_path) do |f|
  %fieldset
    = f.label      :name, "Name"
    = f.text_field :name

  %fieldset
    = f.label      :country_name_like, "Country Name Like"
    = f.text_field :country_name_like

  %fieldset
    = f.label  :is_megacity, "Megacity?"
    = f.select :is_megacity, [['Yes', true], ['No', false], ['Either', '']]

  %fieldset
    = f.label  :continent, "Continent"
    = f.select :continent, ['Africa', 'Asia', 'Europe'], include_blank: true

  = f.submit "Search"
  
- @results.each do |city|
  = render partial: 'city', locals: {city: city}

Controllers

As long as your form submits options your search understands, you can easily hook it up in your controller:

# app/controllers/orders_controller.rb
class OrdersController < ApplicationController

  def index
    @search  = OrderSearch.new(search_params) # For use in a form
    @results = @search.results                # For display along with form
  end
  
  protected
  
  def search_params
    # Ensure the user can only browse or search their own orders
    (params[:order_search] || {}).merge(user_id: current_user.id)
  end
end

Compatibility

For any given version, check .travis.yml to see what Ruby versions we're testing for compatibility.

Installation

Add this line to your application's Gemfile:

gem 'searchlight'

And then execute:

$ bundle

Or install it yourself as:

$ gem install searchlight

Contributing

rake runs the tests; rake mutant runs mutation tests using mutant.

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

Shout Outs

• The excellent Mr. Adam Hunter, co-creator of Searchlight.
• TMA for supporting the initial development of Searchlight.

searchlight's Issues

aliasing options to to_h

What do you think about aliasing options as to_h method like this:

  alias_method :to_h, :options

?

I'm very often using to_h method but with searchlight have to use options.
Usually I'm aliasing it in my classes to to_h

Use cancancan scope on searchlight (tricks)

Hi.

I use cancan and searchlight on my application and i need to use something like that :
http://www.bencurtis.com/2013/12/searchlight-and-cancan/

But i wanted to write something like that :

    @search = MyModelSearch.new(@my_models, search_params)
    @my_models = @search.results

So i write the following monkey_patch :

module Searchlight
  module DSL
    def search_on(target)
      @search_target = @base_target = target
    end

    def base_target
      return @base_target if defined?(@base_target)
      return search_target()
    end
  end
end

module Searchlight
  class Search
    def initialize(*args)
      options = args.extract_options!

      if (ar_relation = args.first) && ar_relation.is_a?(ActiveRecord::Relation)
        self.class.search_target = ar_relation
      else
        self.class.search_target = self.class.base_target
      end
      filter_and_mass_assign(options)
    end
  end
end

Merge option doesn't seem to work.

Hi. I wanted to limit the search results to only ones user allowed to view and I followed the example in the docs, but it seems that the merge option is completely being ignored. I checked the terminal and the option was never added to the query. I even hardcoded it (ex: company_id: 14), but still didn't work. Can you verify if this is a bug or am I doing it wrong? Thanks.

class AssetsController < ApplicationController

 def search
    @search = AssetSearch.new(search_params)
    @results = @search.results
  end

private
 def search_params
    (params[:asset_search] || {}).merge(company_id: current_company.id)
  end

 def current_company
    current_user.company
  end

end

Add mutation testing

I'd like to start doing mutation testing on Searchlight with the Mutant gem. Ideally, we'd have a task like rake mutate which could take a class/module argument and run those tests.

I would love to get a PR for this. :)

Add convenient way to get a db-formatted date

I keep doing this kind of thing in my ActiveRecord-based search classes:

  %w[completed_on_or_after completed_on_or_before].each do |name|
    define_method(name) do
      value = super()
      value.to_date.to_s(:db) if value.present?
    end
  end

It would be nice to have the ActiveRecord adapter build in a way to get db-formatted date strings more easily. Maybe just db_string(completed_on_or_after).

ActiveRecord adapter shouldn't define methods for non-existent columns

If a search declares searches :last_name_in, where last_name is the actual column name, the ActiveRecord adapter shouldn't define

  def search_last_name_in
     search.where(last_name_in: last_name_in)
  end

because that's just going to generate an "unknown column" SQL error; the table doesn't have a last_name_in column.

Instead, it should define that method only if the model class's columns_hash contains that exact attribute. Then if the user forgets to define search_last_name_in, they'll get a missing method error instead of a SQL error.

undefined method `validators_on' for EmissionComponentSearch:Class

Hi Nathan.

First off, great library. I find it much clearer and less magical than ransack. Provides organization and structure without obscuring too much code.

Onto the issue:

I get undefined method 'validators_on' for EmissionComponentSearch:Class when trying to use fields_for as follows:

reports_controller.rb

@search_params = EmissionComponentSearch.new( @report.search_params )

reports/_form.html.erb

<%= form_for( @report ) do |f| %>

  <%= f.fields_for :search_params, @search_params do |search_params_f| %>

    <%= search_params_f.collection_select( :facility_id, @facilities, :id, :name ) %> 

Just as a preface, I'm using the Report model to allow Users to save their search. The Report model contains a search_params column that is a serialized hash.

Any ideas what's going on and how to get around it?

Many Thanks!

Josh

Update deps

Dependabot notifies me that ActionView and Bundler need to be updated. I haven't worked in Ruby in a few years but I'd accept a PR for this if anybody cares to make one.

Searchlight is caching results

When a record is updated it's not reflected in the results when called after the update e.g.

search
update record in the search
search (displays old record till app restart)

Doesn't re-evaluates procs inside named scopes

Example:

class Product < ActiveRecord::Base
  scope :not_deleted, ->(time=Time.now) { where(arel_table[:deleted_at].gt(time).or(arel_table[:deleted_at].eq(nil))) }
end

class ProductSearch < Searchlight::Search
  search_on Product.not_deleted
end

search = ProductSearch.new
search.results # time = 2014-08-01 17:46:36.623009

new_search = ProductSearch.new
new_search.results # time = 2014-08-01 17:46:36.623009 (same time)

Time.now inside the not_deleted scope does not get re-evaluated despite that a new ProductSearch object was initialised.

automatically infer search_on

It might be nice if searchlight could guess my search on if I don't declare one.

For example:

Brain::AmygdalaSearch < Searchlight could infer that my search target would be Brain::Amygdala if I don't specify one..

Some issue with html form elements

There is some weird behaviour about forms element. I get the whole thing working in the console and from the url but the form itself is causing some errors when I try to put a select-box, if if the method dosen't exist though its there:

undefined method `instruments_like' for #User::FriendlyIdActiveRecordRelation:0x007fa6a1d6e238

If change to text_field thats working again though.

Here is the whole code.
https://gist.github.com/borisrorsvort/5688847

Don't work with collection_check_boxes

= form_for(@search, url: search_path) do |f|
  = f.collection_check_boxes(:post, :author_ids, Author.all, :id, :name)

after submit none of the selected option is not checked
because id is not converted to an integer

Search param is `nil` in odd cases of search method.

This is an odd one.

Here is my Search class:

class ToDoSearch < Searchlight::Search

  search_on ToDo.scoped

  searches :state

  def search_state
    debugger                  # state == 'due_soon'
    # state = state.to_sym
    search.send( state )
  end

end

Now, when I uncomment state = state.to_sym and hit the debugger the value of state becomes nil. Nothing else is changed.

def search_state
  debugger                    # state == nil
  state = state.to_sym
  search.send( state )
end

Why do you think that is? Does search somehow need to be the first thing in the method?

Thanks!

Redefining `query` in a search method sets it to `nil`.

After upgrading to v4 and replacing all of our search with query, this is something we noticed.

If you use query inside of a search method, query gets set to nil instead of carrying over from the base_query or any previous search methods. An example may be the best way to explain.

Working

def search_last_name
  # query.class => Person::ActiveRecord_Relation
  query.where(last_name: last_name)
end

def search_last_name
  # query.class => Person::ActiveRecord_Relation
  new_query = query.where(last_name: last_name)
  new_query
end

Not Working

def search_last_name
  # query.class => NilClass
  query = query.where(last_name: last_name)
  query
end

I'm not sure what's going on that query is being set to nil before it hits the method if you locally define query in the method.

It's pretty easy to work around by using new_query or something similar but I'm curious if you were aware of this and if it's something that we can help fix.

I know you're more or less hands off with this gem (thanks for all your efforts by the way) but if you can help with this I may be able to help with some PRs to keep it maintained.

Many thanks!

cool searchlight tricks

i had the fun task of trying to create a search that would normally start at an association of the current user (which was a has many through) but I needed to use that relation in the searchlight class. thankfully merge helped me out. the base of the search was the target table of the has many through (otherwise I'd have to join onto the search I was merging)

def search_user
  user.associated_has_many_through.merge(search)
end

thought i'd share ❤️

ActiveRecord - Posibility to merge multiple search methods with OR instead of AND?

Is it possible to merge different search methods by OR instead of AND ? Meaning this:

def search_some_method
    Post.where(some_column: some_method)
end

def search_some_method_2
    Post.where(another_column: some_method_2)
end

Instead of resulting in:
Post.where(some_column: some_method).where(another_column: some_method_2)

results in:
Post.where(some_column: some_method).or(Post.where(another_column: some_method_2))

Maybe by sending some parameter in the search params to specify joining of "where's" by OR.

Should have an accessor for non-blank options provided

It can be useful to get a list of non-blank options provided to a search. For instance, I only want the "advanced search options" section of a form to show on page load if the search that was just run had any advanced options given to it.

It would also be useful to have a more explicit definition of defaults for the scenario above; maybe some advanced options were given but they had the default values so we don't care (eg, radio buttons, which always submit something but maybe something that doesn't affect the search).

How to implement pagination?

gem 'kaminari'

try

def search_page
  search.page(page).per(30)
end

not work

Problem with the Sequel ORM

Hi, I have a project where I'd like to user searchlight with the Sequel ORM. However, after constructing a new search object instance, I get an error when retrieving the results.

I setup a simple search object to try to narrow down the problem:

class UserSearch < Searchlight::Search
  search_on User.dataset
  searches :username

  def search_username
     search.where(username: username)
  end
end

Here's the calling code:

 UserSearch.new({username: 'altanodouro'}).results

And the error:

 ArgumentError: wrong number of arguments (0 for 1+)

Th error seems to originate here, when target.call is executed:

def search
  @search ||= begin
                target = self.class.search_target
                if target.respond_to?(:call)
                  target.call
                else
                  target
                end
              end

As an experiment, I've commented the if..else block and just made the method return target.
It seems to fix it, in this case.

Add nice AR logging like Dossier has

Maybe not possible since we're not executing queries manually, but it would be nice if we could find a way to label SQL output in the logger with the search it came from, at least when we know we're working with ActiveRecord.

For reference, see:

adamhunter/dossier@76a95e1