joel/refine-ruby

Ruby client library for controlling Google Refine

refine-ruby

is a Ruby Gem client library for OpenRefine. It was written for Google Refine 2.x, but should work with OpenRefine as well.

If you want to port this to another language, check out the Refine API documentation.

NOTE: The Refine client/server protocol is an internal API which is subject to change, so use at your own risk (although it has stayed relatively stable for the last few versions).

Install

gem install refine-ruby

Use Cases

Apply Operations / Export Operations

Given that you have the following raw data:

  
    Date
    7 December 2001
    July 1 2002
    10/20/10
  

Refine Ruby lets you clean up the data and export your operation history as a JSON instruction set. Here is an example that extracts the year from the above dates:

  
    [
      {
        "op": "core/text-transform",
        "description": "Text transform on cells in column Date using expression grel:value.toDate()",
        "engineConfig": {
          "facets": [],
          "mode": "row-based"
        },
        "columnName": "Date",
        "expression": "grel:value.toDate()",
        "onError": "set-to-blank",
        "repeat": false,
        "repeatCount": 10
      },
      {
        "op": "core/text-transform",
        "description": "Text transform on cells in column Date using expression grel:value.datePart(\"year\")+1",
        "engineConfig": {
          "facets": [],
          "mode": "row-based"
        },
        "columnName": "Date",
        "expression": "grel:value.datePart(\"year\")",
        "onError": "set-to-blank",
        "repeat": false,
        "repeatCount": 10
      }
    ]
  

You can use this gem to apply the operation set to the raw data from ruby. You will need to have OpenRefine running on your local computer, or specify an external address (see source):

  
  require 'refine'
    prj = Refine.new('project_name' => 'date cleanup', 'file_name' => 'dates.txt')
    prj.apply_operations('operations.json')
    puts prj.export_rows('csv')
  

Which outputs:

  
    Date
    2001
    2002
    2010
  

Link to Facets

Continuing on the project above – assuming ‘project_id’ = 1594197247031.
Link to facets generates a link to the facets set up. The method can take an array of hashes, with the keys as the column name and the optional values: ‘sort’, ‘invert’, and a ‘’.

  
  require 'refine'
    prj = Refine.new('project_id' => 1594197247031)
    prj.link_to_facets('Date' => ['invert', 'isNonBlank(value)'])
  

Which outputs:

 "http://127.0.0.1:3333/project?project=1594197247031&ui=%7B%22facets%22%3A%5B%7B%22c%22%3A%7B%22columnName%22%3A%22Date%22%2C%22expression%22%3A%22value%22%2C%22name%22%3A%22Date%22%2C%22invert%22%3Atrue%7D%2C%22o%22%3A%7B%22sort%22%3A%22name%22%7D%7D%5D%7D"

Screenshot:

Compute Facet

Compute facet returns a JSON object with meta info on the output of the column based on the GREL expression provided. The method can take an array of hashes with the keys as the column name and the optional values: ‘sort’, ‘invert’, and a ‘’.

  
      prj = Refine.new('project_id' => 1594197247031)
      prj.compute_facet('Date' => ['value'])
  

Which outputs:

[
  {
    "columnName"=>"Date",
    "name"=>"Date",
    "expression"=>"value",
    "choices"=>[
        {"value"=>2002, "label"=>"2002", "count"=>1, "selected"=>false},
        {"value"=>2001, "label"=>"2001", "count"=>1, "selected"=>false},
        {"value"=>2010, "label"=>"2010", "count"=>1, "selected"=>false}
    ]
  }
]

See test_refine.rb to see all the optional arguments for link_to_facets, and compute_facet methods.

Copyright

Copyright © 2018 David Huynh, Max Ogden & Distill Analytics Inc.