projectblacklight/blacklight-marc

MARC enhancements for Blacklight

blacklight-marc's Issues

Worldcat integration

Lookup by OCLC number.

Use BL7 compatible solr configuration in test apps and generators

"for SolrMarc"

Github repo description says "rake task to run SolrMarc" -- but of course, this has a bunch more in it than that, it also has 'librarian_view', as well as code to translate from MARC to other formats like Refworks, as well as probably other things.

The gemspec also says "SolrMarc support for Blacklight".

This gem most definitely isn't just SolrMarc -- I don't even use SolrMarc and it still has tons of stuff I need in it.

Should the repo description and gemspec descriptions be changed to say "MARC support for Blacklight", or "various MARC features for Blacklight", or something?

Deprecation warning for @document

I'm seeing a number of deprecation warnings in the Rails log surrounding @document which was deprecated here

DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_endnote_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:29)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_endnote_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:29)

I am not sure how this can be fixed. I tried the following, but just got new warnings:

diff --git a/lib/blacklight/marc/catalog.rb b/lib/blacklight/marc/catalog.rb
index 29b3830..8508cbc 100644
--- a/lib/blacklight/marc/catalog.rb
+++ b/lib/blacklight/marc/catalog.rb
@@ -22,15 +22,15 @@ module Blacklight::Marc
     private

     def render_refworks_action? config, options = {}
-      options[:document] && options[:document].respond_to?(:export_formats) && options[:document].export_formats.keys.include?(:refworks_marc_txt )
+      @response.docs.first && @response.docs.first.respond_to?(:export_formats) && @response.docs.first.export_formats.keys.include?(:refworks_marc_txt )
     end

     def render_endnote_action? config, options = {}
-      options[:document] && options[:document].respond_to?(:export_formats) && options[:document].export_formats.keys.include?(:endnote )
+      @response.docs.first && @response.docs.first.respond_to?(:export_formats) && @response.docs.first.export_formats.keys.include?(:endnote )
     end

     def render_librarian_view_control? config, options = {}
-      respond_to? :librarian_view_solr_document_path and options[:document] and options[:document].respond_to?(:to_marc)
+      respond_to? :librarian_view_solr_document_path and @response.docs.first and @response.docs.first.respond_to?(:to_marc)
     end

   end

And the warning here was:

DEPRECATION WARNING: The @response instance variable is deprecated; use @document.response instead. (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)

This is the whole list of deprecation warning I am seeing when I am using Blacklight 7.1.0 with blacklight-marc:

DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_endnote_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:29)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_endnote_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:29)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from single_endnote_catalog_path at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/app/helpers/blacklight_marc_helper.rb:16)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_refworks_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:25)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_endnote_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:29)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from render_endnote_action? at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/lib/blacklight/marc/catalog.rb:29)
DEPRECATION WARNING: The @document instance variable is deprecated and will be removed in Blacklight-marc 8.0 (called from single_endnote_catalog_path at /Users/cdm32/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/blacklight-marc-c0ff1d9cb5d3/app/helpers/blacklight_marc_helper.rb:16)

Second return value from search_service.fetch should be deprecated

As it is in blacklight itself:
https://github.com/projectblacklight/blacklight/blob/master/app/controllers/concerns/blacklight/bookmarks.rb#L43-L44

Generate blacklight_advanced_search configured for a MARC-based index

QR Code generation

Because everyone needs more QR codes (I believe this is a feature also present in some vufind installs too.)

LoadError: Cannot load such file -- ./app/models/marc_indexer

rake solr:marc:index MARC_FILE= /var/www/project1/my_new_blacklightapp/1.mrc
rake aborted!
LoadError: cannot load such file -- ./app/models/marc_indexer

Tasks: TOP => solr:marc:index => solr:marc:index:work
(See full trace by running task with --trace)

Update RefWorks export to use form based approach

Current link callback to RefWorks seems to break on text that includes diacritics. We have resolved this by moving towards the form based approach (same as Vufind) in SearchWorks: sul-dlss/SearchWorks#2066

We would also want to make export_as_refworks_marc_txt a public method to do this.

Internal test app is pointing toward nonexistant branch

projectblacklight/blacklight now uses a branch called main, rather than master. However, spec/test_app_templates/Gemfile.extra in this repository hasn't been updated accordingly.

It should probably be changed to this if it is compatible with blacklight 8.0:

gem 'blacklight', github: 'projectblacklight/blacklight', branch: 'main'

Or this if it is not yet compatible:

gem 'blacklight', github: 'projectblacklight/blacklight', branch: 'release-7.x'

Bump version for 5.x

In response to @jrochkind 's versioning proposal for plugins, we should bump this plugin to blacklight_marc 5.0.0 for the blacklight 5.0.0 release.

Also, what do think about renaming this to blacklight-marc, so we can have paths like lib/blacklight/marc/engine? I think that might be more in line with other rails conventions, but I don't particularly either way.

NoMethodError: undefined method `downcase' for nil:NilClass

This may occur if you call export_as_openurl_ctx_kev with an empty array.

https://github.com/projectblacklight/blacklight-marc/blob/main/app/models/concerns/blacklight/marc/document_export.rb#L66

    issn = to_marc.find{|field| field.tag == '022'}
    unless format.nil?
      format.is_a?(Array) ? format = format[0].downcase.strip : format = format.downcase.strip
    end
      export_text = ""

Parsing and displaying subject hierarchy

Move to traject 3

CI is failing for blacklight 8

I created a fork and made a test pr to force CI to run and there is a test failing. This test is also failing when run locally.

Failing PR: rladdusaw#1

Failures:

1. Librarian view should show marc fields
   Failure/Error: expect(page).to have_content "Librarian View"
   expected to find text "Librarian View" in "Skip to search Skip to main content\nBlacklight\nBookmarks 0 History Login\nSearch in\nAll Fields Title Author Subject search for\nSearch\n×\nLEADER 01213nam a22003614a 4500\n001\n2009373513\n003\nDLC\n005\n20090121153231.0\n008\n090114s2008 ch 000 0 chi d\n010\n \n \na| 2009373513\n020\n \n \na| 9789573908678\n020\n \n \na| 9573908670\n035\n \n \na| (OCoLC)ocn268619391\n040\n \n \na| HUA c| HUA d| HKP d| DLC\n042\n \n \na| lccopycat\n043\n \n \na| a-cc-hk\n050\n0\n0\na| HC59.15 b| .L533 2008\n066\n \n \nc| $1\n100\n1\n \n6| 880-01 a| Lin, Xingzhi.\n245\n1\n0\n6| 880-02 a| Ci an zhou bian / c| Lin Xingzhi zhu.\n250\n \n \n6| 880-03 a| Chu ban.\n260\n \n \n6| 880-04 a| Taibei Xian Banqiao Shi : b| Yuan jing chu ban shi ye you xian gong si, c| 2008.\n300\n \n \na| 5, 300 p. ; c| 21 cm.\n490\n0\n \n6| 880-05 a| Lin Xingzhi zuo pin ji ; v| 51\n651\n \n0\na| Economic history y| 1990-\n651\n \n0\na| World politics y| 2005-2015.\n651\n \n0\na| Hong Kong (China) x| Economic conditions y| 1997-\n651\n \n0\na| Hong Kong (China) x| Politics and government y| 1997-\n880\n1\n \n6| 100-01/$1 a| 林行止.\n880\n1\n0\n6| 245-02/$1 a| 次按驟變 / c| 林行止著.\n880\n \n \n6| 250-03/$1 a| 初版.\n880\n \n \n6| 260-04/$1 a| 臺北縣板橋市 : b| 遠景出版事業有限公司, c| 2008.\n880\n0\n \n6| 490-05/$1 a| 林行止作品集 ; v| 51"
   # ./spec/features/librarian_view_spec.rb:7:in `block (2 levels) in <top (required)>'

Fix "Librarian View" display

• use a fixed-width font
• legibly style tags

stackmap integration

Availability lookup

translation_map parameter to to_field should permit array of values

when the translation_map parameter is an array of strings, each value should be resolved a s a translation map and merged. The source maps should be merged to give priority to the first value in the array.

to_field "mapped", map_value(literal('k'),translation_map:["test_format_overrides","test_formats"])

Exception in clean_end_punctuation

Method clean_end_punctuation in marc_export.rb throws an exception if the value passed is nil

blacklight-marc/app/models/concerns/blacklight/solr/document/marc_export.rb

Lines 493 to 498 in db85fe3

	def clean_end_punctuation(text)
	if [".",",",":",";","/"].include? text[-1,1]
	return text[0,text.length-1]
	end
	text
	end

This is a problem when chicago_citation(marc) is called with a nil value for marc["502"]["d"]. This is the block that makes the call to clean_end_punctuation with nil

blacklight-marc/app/models/concerns/blacklight/solr/document/marc_export.rb

Lines 303 to 305 in db85fe3

	elsif marc["502"] and (marc["502"]["b"] or marc["502"]["c"] or marc["502"]["d"]) #sometimes the dissertation note is encoded in pieces in the $b $c and $d sub fields instead of lumped into the $a
	pub_info << "#{marc["502"]["b"]}, #{marc["502"]["c"]}, #{clean_end_punctuation(marc["502"]["d"])}"
	end

Add book cover images to search results

Use the Google Books API:
https://developers.google.com/books/docs/dynamic-links

Add a preview button to the show page:
https://developers.google.com/books/images/gbs_preview_button1.gif

Can blacklight-marc tests be run independently of blacklight?

I am having difficulty running the rspec tests for the gem. Using Ruby 2.5.3, bundle install works fine. I would like to get tests working because at Yale we made some modifications to how MARC fields are stored and then used with Endnote/Refworks in order to provide a more complete citation system, and would like to contribute it back to the community (if desired), but first we need to make sure we didn't break any tests.

This is what I'm getting:

bundle exec rspec
2019-10-30 09:54:49 WARN Selenium [DEPRECATION] Selenium::WebDriver::Chrome#driver_path= is deprecated. Use Selenium::WebDriver::Chrome::Service#driver_path= instead.

An error occurred while loading ./spec/controllers/catalog_controller_spec.rb.
Failure/Error: add_show_tools_partial(:librarian_view, if: :render_librarian_view_control?, define_method: false)

NoMethodError:
  undefined method `add_show_tools_partial' for CatalogController:Class
# ./lib/blacklight/marc/catalog.rb:6:in `block in <module:Catalog>'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/concern.rb:122:in `class_eval'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/concern.rb:122:in `append_features'
# ./.internal_test_app/app/controllers/catalog_controller.rb:5:in `include'
# ./.internal_test_app/app/controllers/catalog_controller.rb:5:in `<class:CatalogController>'
# ./.internal_test_app/app/controllers/catalog_controller.rb:2:in `<top (required)>'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:291:in `block in require'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:257:in `load_dependency'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:291:in `require'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:378:in `block in require_or_load'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:37:in `block in load_interlock'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:14:in `block in loading'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:151:in `exclusive'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:13:in `loading'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:37:in `load_interlock'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:356:in `require_or_load'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:48:in `block in require_or_load'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:16:in `allow_bootsnap_retry'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:47:in `require_or_load'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:510:in `load_missing_constant'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:60:in `block in load_missing_constant'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:16:in `allow_bootsnap_retry'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/bootsnap-1.4.4/lib/bootsnap/load_path_cache/core_ext/active_support.rb:59:in `load_missing_constant'
# /Users/joshualevinson/.rvm/gems/ruby-2.5.3/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:195:in `const_missing'
# ./spec/controllers/catalog_controller_spec.rb:4:in `<top (required)>'
/Users/joshualevinson/code/blacklight-marc/app/models/concerns/blacklight/solr/document/marc_export.rb:187: warning: key "SN" is duplicated and overwritten on line 188


Finished in 0.00028 seconds (files took 1.71 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

Obsolete partials?

It seems to me _endnote.html.erb is obsolete because of how document action links are now handled in Blacklight. _show_tools.html.erb basically just takes the content of endnote and refworks and does a link_to on them inside of a uniform li. Wondering if _endnote.html.erb, _marc_tools.html.erb and _refworks.html.erb should be deleted (i.e., blacklight-marc/app/views/bookmarks)? Or do they still serve a purpose I'm not seeing?

This has come up for me because I'm trying to override in a way that is copacetic to blacklight and blacklight_marc.

Records without an 008 field crashing on index from .mrc file.

When running rake solr:marc:index MARC_FILE="some_marc_records.mrc", records without an 008 field crash when extract_marc is called to set the value of formatCode, at

blacklight-marc/lib/blacklight/marc/indexer/formats.rb

Line 142 in dc32e02

formatCode = extract_marc("008[21]", first: true) do |r,a|

.

See full stacktrace:

** Invoke solr:marc:index (first_time)
** Invoke solr:marc:index:work (first_time)
** Invoke solr:environment (first_time)
** Execute solr:environment
** Execute solr:marc:index:work
2016-03-11T09:17:48-05:00  INFO    Traject::SolrJsonWriter writing to 'http://127.0.0.1:8983/solr/blacklight-core/update/json' in batches of 100 with 1 bg threads
2016-03-11T09:17:48-05:00  INFO    Indexer with 1 processing threads, reader: Traject::MarcReader and writer: Traject::SolrJsonWriter
2016-03-11T09:17:48-05:00 ERROR Unexpected error on record id `76/76` at file position 1
    while executing (to_field format at /$LOCAL_DIRECTORY/blacklight/app/models/marc_indexer.rb:23)
    Exception: NoMethodError: undefined method `extract_marc' for Blacklight::Marc::Indexer::Formats::FormatMap:Module
    /$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/blacklight-marc-6.0.0/lib/blacklight/marc/indexer/formats.rb:143:in `map_leader'

2016-03-11T09:17:48-05:00 ERROR Unexpected error on record id `222/222` at file position 3
    while executing (to_field format at /$LOCAL_DIRECTORY/blacklight/app/models/marc_indexer.rb:23)
    Exception: NoMethodError: undefined method `extract_marc' for Blacklight::Marc::Indexer::Formats::FormatMap:Module
    /$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/blacklight-marc-6.0.0/lib/blacklight/marc/indexer/formats.rb:143:in `map_leader'

2016-03-11T09:17:48-05:00 ERROR Unexpected error on record id `293/293` at file position 4
    while executing (to_field format at /$LOCAL_DIRECTORY/blacklight/app/models/marc_indexer.rb:23)
    Exception: NoMethodError: undefined method `extract_marc' for Blacklight::Marc::Indexer::Formats::FormatMap:Module
    /$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/blacklight-marc-6.0.0/lib/blacklight/marc/indexer/formats.rb:143:in `map_leader'

rake aborted!
NoMethodError: undefined method `extract_marc' for Blacklight::Marc::Indexer::Formats::FormatMap:Module
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/blacklight-marc-6.0.0/lib/blacklight/marc/indexer/formats.rb:143:in `map_leader'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/blacklight-marc-6.0.0/lib/blacklight/marc/indexer/formats.rb:179:in `block in get_format'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer/step.rb:144:in `call'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer/step.rb:144:in `execute'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer.rb:347:in `block (2 levels) in map_to_context!'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer.rb:401:in `log_mapping_errors'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer.rb:346:in `block in map_to_context!'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer.rb:340:in `each'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer.rb:340:in `map_to_context!'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/indexer.rb:475:in `block (2 levels) in process'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/traject-2.3.0/lib/traject/thread_pool.rb:108:in `block in maybe_in_thread_pool'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `call'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/$RVM_LOCATION/.rvm/gems/ruby-2.2.1@blacklight/gems/concurrent-ruby-1.0.1/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
Tasks: TOP => solr:marc:index => solr:marc:index:work

Use routing concerns

Use idiomatic traject 3 macros and remove unnecessary subclass behaviors

• remove first_value
• use the first_only, translation_map, and trim_punctuation macros where possible
• rewrite trim and map_value as idiomatic lambda generators

trim and map_value still provide functionality from the historic Java indexer absent in traject.

Target the correct scss file for appending

https://gist.github.com/jcoyne/e6e3ec290c205427e3a5bc2e8aabc198

Interfiling support

Add more interesting solr indexing

Holdings data panel

Why use marc_ss and not marc_ts for Marc source?

When loading records to a Blacklight 7 instance with blacklight-marc tools, I've found several errors like this one:

2021-07-28T08:11:55+02:00 ERROR Could not add record <record #3333 ([...]/ddd.marcxml #3333), source_id:245096 output_id:245096>: Solr error response: 400: {
"responseHeader":{
"status":400,
"QTime":7},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException"],
"msg":"Exception writing document id 245096 to the index; possible analysis error: Document contains at least one immense term in field="marc_ss" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[60, 114, 101, 99, 111, 114, 100, 62, 60, 108, 101, 97, 100, 101, 114, 62, 32, 32, 32, 32, 32, 32, 90, 32, 32, 32, 50, 50, 32, 32]...', original message: bytes can be at most 32766 in length; got 46362. Perhaps the document has an indexed string field (solr.StrField) which is too large",
"code":400}}

According to https://github.com/projectblacklight/blacklight-marc/blob/master/solr/conf/schema.xml, fields using the _ss prefix mean that effectively they are stored as a string (and not indexed, which is fine). However, for very large records, it hits a Solr (more properly Lucene, as I have learned) hard limit.

Changing it to marc_ts in marc_indexer.rb and solr_document.rb allows those sources to be stored. Then, output formats like Librarian view or citation do not show anything, but before investigating further, I'd like to know the reason of using marc_ss, and if changing it to marc_ts would have other consequences beyond those temporarily missing output formats.

Documentation on metadata MAP for less technical audiences.

I am wondering if someone has created documentation, such as the simple table FOLIO has created, that we could point a less technical audience at to explain how Blaklight-marc uses different MARC fields.

If not, is this something we can allocate some of our own resources to create and then have as part of the community documentation?

If there is existing documentation that I missed, let me know. I see extensive documentation for Blacklight, but not necessarily this MARC specific plug-in.

Deprecate features removed in 7.0 in 6.x

Needs to be pushed to 6.3.0:

• Deprecate catalog/endnote route in favor of endnote format
• Deprecate map_value: 'use traject translation directly and ascending priority of maps

OpenURL resolver integration (by pulling from solr fields?)

Travis does not build on latest rails

Any PR that does not peg the rails version to 5.1.6 or lower fails. I think the reason is related to engine_cart not working for rails 5.2.0. I'll create an issue with engine cart, but I'm adding this issue here, just in case that's not it.

better solr configuration for library search use cases

Pull publisher citation information from the 264 field in records when 260 is unavailable

Records that have publisher information recorded in the 264 field instead of the 260 field are not displaying in the citation modal in blacklight. The 264 field should be used as a fallback source when the 260 is not populated.

Atom tests fail against recent versions of blacklight 7

Steps to recreate:

1. Use ruby 2.7.5
2. Get a clean version of this repo (either clone it fresh or remove your Gemfile.lock and .internal_test_app)
3. Edit line one of spec/test_app_templates/Gemfile.extra to read:

gem 'blacklight', '7.23.0.1'

5. export RAILS_VERSION=6.0.3.4
6. bundle
7. bundle exec rake
8. Note that you get 2 failing tests

The tests pass when I use blacklight 7.19, so I bet this was introduced recently. PR coming soon to get this fixed up.

Add nearby_on_shelf to the generator, views.

Bento-box search example

Librarian view does not work after quickstart in Rails 6

Steps to recreate:

1. Create an application the hard way:

• rails new my_new_blacklightapp
• cd my_new_blacklightapp
• Add blacklight to the Gemfile
• bundle install
• gem install devise devise-guests
• rails generate blacklight:install --devise --marc --solr_version=latest
• rake db:migrate
• bundle exec solr_wrapper
• rake solr:marc:index_test_data
• rails s

2. Check app/models/solr_document.rb. Note that the marc source field is called :marc_ss and it's in the marcxml format.
3. Open up solr, and look at the documents. Notice that they all have marc_ss fields with marcxml data
4. Open up the show view of a record. Note that the librarian view link does not appear.
5. Go to http://localhost:3000/catalog/2007020969/librarian_view. Note that it says "No MARC data found."

I tried this with ruby 2.5.7 and jruby 9.2.9.0. I'm on rails 6.0.2.1, so it could be a rails 6 issue?