mutopiaproject/mutopiaweb

MutopiaProject site evolution to HTML5

The Mutopia Project

The Mutopia Project offers sheet music based on editions in the public domain. All sheet music is written in LilyPond by volunteers and the entire archive is openly available for download. If you are new to the project please visit MutopiaProject.org and browse the sheet music.

This GITHUB repository is made available to promote this volunteer effort. There is a wiki available if you are interested in contributing using GITHUB. Please see the main website contribution page for information on how to submit by email if you are not comfortable with GITHUB.

mutopiaweb's Issues

Font and background consistency

@dominicus regarding color use on logo,

The "deep cerulean" blue used for MUTOPIA in the logo is very appealing. Implement this as a palette font color for all blue fonts, especially links.
Same goes for the near-black background at the top menu. This near black could be used for all titles.

This task can drive discussion as well as implementation. The logo font color is was picked from the original; top menu colors can be determined from bootstrap's styling and used in ours. Implementation of this task should strive to make colors consistent through all pages and as flexible as possible for future style changes.

Popularity metric

Originally mentioned by @dominicus in issue #32 : Use a popularity metric to order browse lists.

• First entry in Instrument list should be the instrument whose count is specified in the most pieces.
• First entry in Composer list should be the composer with the most pieces.

Advanced, or perhaps another issue: allow users to switch between alphabetical or popularity sorting, use cookies to remember the user's choice.

n.b., we only do dynamic pages on search results so the advanced task will take some work.

*Downoad counts: * The only good way to do this is with a database.

Content URLs for the Portugal mirror still serving from Canada server

As reported by Bob Bib (10/7/2018) via email discussion list
http://lists.bcn.mythic-beasts.com/mailman/private/mutopia-discuss/2018-October/001002.html

Well, according to the start page, there're 2 mirrors:

1. Canada (main): http://www.mutopiaproject.org/
2. Portugal: http://eremita.di.uminho.pt/mutopia/

It turns out that currently Portuguese mirror is serving web pages only,
and all the music content is loaded from the main mirror,
and I wonder whether it should be that way.

E.g. You can view info about the Blue Danube Waltz on both mirrors:
http://www.mutopiaproject.org/cgibin/piece-info.cgi?id=519
http://eremita.di.uminho.pt/mutopia/cgibin/piece-info.cgi?id=519

But when you try to download MIDI, you have the same link on both pages:
http://www.mutopiaproject.org/ftp/StraussJJ/O314/blue_danube/blue_danube.mid

Out of curiosity, I've tried to guess if we can download the music files
from the Portuguese mirror,
and it looks like the answer is positive -- here is the guessed mirror
MIDI link:
http://eremita.di.uminho.pt/mutopia/ftp/StraussJJ/O314/blue_danube/blue_danube.mid

--
Best wishes,
Bob

Folding sections on small devices?

Can we do this with some of our pages? Should we try? Can it be done with a non-dynamic website?

The above is a shot from my phone but it can be simulated in most browser developer tools. In Firefox:

• browse to a wiki page (the above is a search for Reginald Hill)
• in the web site edit box, change en.wikipedia.org to en.m.wikipedia.org
• got to Developer / Responsive Design Viewer

Download server throws 403 error

Hi,

when I try to get music sheets from mutopiaproject.org, I get always 403 (Forbidden) errors. E.g. this link.

Jonathan

Jumbotron and logo

The "logo" sits inside a bootstrap "Jumbotron" as a grid that is divided up to replicate the original site. The left third of the grid is the logo, the middle the "mission statement" and the right contains links to mirrors for the site.

• Does it have to be this big?
• The logo was meant,
  • as a placeholder for the eventual, "real" logo
  • to see how well vector graphics (svg) work in the browser

I like the look of this but it was done without oversight or feedback. It is time for that.

About the logo: It was created in inkscape and uses the inkscape svg file directly. The font is an open source font called "Utopia"; the name fits our purpose and looks nice. The characters were converted to a path and manipulated slightly. The color was matched from the previous logo.

Pagination needed

There are several instances where a user can get a substantial amount of output:

• If they click on 'List all Pieces'
• If they click on the submit button of the advanced search without filling out any search criteria

The problem is twofold:

• It is a lot of data, especially on a small device which may be on a cellular network
• The piece info uses relatively large tables that require lots of scrolling on a small device

The difficulty to a solution will be that we employ a filtered linear search.

• To know we have 100 results we have to do a complete search with the filters applied.
• To display the first 25 is simple enough with a counter.
• To display the last 25 requires that we search through the first 75.

The functionality we need is similar to a database query using offset and limit. Can this be done relatively efficiently with our current search mechanism?

Make updating bootstrap smoother

Bootstrap will be updated from time to time, but currently the bootstrap version number is hardcoded into several places. Change the folder name to just "bootstrap" and note the version number in a README file. Remove hardcoding from the files.

No more tables in search results

This has been discussed in #59 and #60 , but there was feedback on the list about this as well. The goal would be to provide search results that scale better than tables. This is prime opportunity to improve how we show the results (icons, etc.) to make the the display as tight and concise as possible.

Front page tweaks

From user Mark Van den Borre,

• On the front page, I would drop the words "browse (by)" aboven the
  columns ("browse latest additons"->"latest additions", "browse by
  instrument"->"instrument", "browse collections"-> "collections"). This
  saves some prime screen real estate, while still being perfectly
  clear.
• Still on the front page, I would make the column title ("Latest
  additions", "Instrument", ...) a clickable link and drop the "Show
  all" link at the bottom. It's no use there anyway, as it drops off
  most screens...
• The links currently exposed by "show all" at the bottom of the front
  page: I would not point them all at one page crowded with multiple
  tables, but at a separate page per subject (a "by instrument" page for
  example). When one wants to browse all instrument links, no use
  pointing the user to a page with tables for composer, instrument and
  style. Just a link to a page with "by instrument" should be much
  clearer and get rid of some more tables.

The first bullet was pushed in without discussion but I agree with Mark that it reads better.

The second bullet proposes column headers be clickable links. This was easy enough to add but the CSS wasn't modified to display them as links. Opinions are always welcome here.

The third bullet is covered in issue #74 .

Java tests fail to build

Java tests don't appear to build at present. I get the following error:

$ gradle build
[...]
:mudb:compileTestJava
/mutopia/MutopiaWeb/mpcore/mudb/src/test/java/mutopia/mudb/RdfTest.java:34: error: exception MuException is never thrown in body of corresponding try statement
            catch (MuException ex) {
            ^
1 error
:mudb:compileTestJava FAILED

FAILURE: Build failed with an exception.

Fortunately doing an install skips the tests so this isn't a showstopper, but would be nice to have it fixed. It appears only one test is broken - commenting it out allows the rest to pass.

My Gradle/Java/OS versions as follows:

------------------------------------------------------------
Gradle 2.14.1
------------------------------------------------------------

Build time:   2016-07-18 06:38:37 UTC
Revision:     d9e2113d9fb05a5caabba61798bdb8dfdca83719

Groovy:       2.4.4
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.8.0_102 (Oracle Corporation 25.102-b14)
OS:           Linux 3.10.0-327.28.3.el7.x86_64 amd64

Home page mock-up

Here's a possible redesign of the home page that I mocked up:
http://clairnote.org/html5mutopia/index-full-wide-6.html

This moves the contents of the Browse page to the home page, except for the huge list of composers, which would, in this scenario, become its own “Composers" page. It also allows all 25 recent additions to go on the home page, removing a need for a separate page for them. It includes the Acknowledgements and RSS Feed sections from the current Contact page, putting the RSS feed next to the new additions. It goes in a somewhat different direction in terms of the header. I incorporated a 5 line staff into the logo. Everything still works at different screen sizes. Bootstrap with its CSS grid system is working great for that.

lastesadditions.html-in needs new formatting

Left align headers (main page titles)

From @dominicus on title,

Left-align titles for browse, search, licensing, contribute, and in-progress sub-pages.

This is a hold-over from the original prototype that attempted to implement the original site in HTML5.

The goal of the task is as stated above, left-aligned header elements on all pages.

Detailed HTTP pings

Quoting myself from MutopiaProject's #1018:

You might want to use some free ping service, for example uptimerobot.com. Setup one check for every type of download, pointing each to some small file of that type on the website. For text files, you can even check for presence of some short text in the file.

This would prevent embarrassing outages like #97 that last for weeks before they are fixed.

src/README has possible typo

In describing how the software is built, the README states:

$ cd src/Mutopia_Archive
$ perl ./Build.PL --install_base=$HOME/MutopiaWeb
$ ./Build
$ ./Build test
$ ./Build install

But --install_base is not a valid option. It seems to "fail" quietly, defaulting to the home folder. But --install-base (no underscore) is a valid option, and installs the programs in lib, probably where they should be. But the .pm programs sometimes assume they are in MUTOPIA_WEB. This may have to change.

Search results page: FTP Area link not working in Windows

This may be a Windows-only problem, but when I click on the FTP Area link in a search page, I get an error (the URL is pointing to somewhere that doesn't exist).

Search by composer broken?

1. Mutopia overflow menu
2. Browse
3. Click any composer name
4. "Sorry, no matches were found for your search criteria."

Alternatively...

1. Mutopia front page
2. "Composers" section
3. Click on any composer name
4. "Sorry, no matches were found for your search criteria."

I've tried Firefox v77 on Debian Testing, and Fennec 68.11.0esr and Tor Browser (Fennec 68.10.1esr) on Android 9.

Advanced search page rework

I think we have some opportunities with HTML5, bootstrap, and form groups to improve the usability of the advanced search page. This task may drive a new advanced search look in prototype 3 but is mostly about starting a discussion about presentation consistency in web search forms.

On "no discussion" I will close this issue for prototype 3 with no changes to this page.

Multi-page results pages missing requested preview images

From discussion in issue #57:
In a search with multi-page results, if I choose to include preview images, these will display only on the first batch of 10 results. Hitting "next 10" will not include preview images.

Jumbotron background

From @dominicus in a group email:

I don't care for the faded sheet music background at top. I find it distracting, plus impacts the readability of text elements. Layering these readable elements is not good practice.

The prototype 2 jumbotron background has the faded sheet music graphic. This issue's purpose is to:

• drive a discussion on background ideas
• change it to the previous light gray background in prototype #3 if no clear idea emerges from the previous bullet

Integrate authority control data

It would be great to have authority control data integrated into Mutopia.
Preferably use IDs from https://VIAF.org or WikiData (and add them to composers.dat).

Thus a BEACON file could be generated by script:
http://www.andreas-praefcke.de/temp/BEACON-PND-Mutopia.txt

MenuBar, fixed or not?

When I first moved the site to bootstrap the fixed MenuBar at the top seemed interesting.

• Will this work on all browsers?
• Should the menu (can it?) be fixed on small devices but not on desktops?
• The left side "logo" and "Home" item do the same thing, should one be eliminated?
• Should "Browse" and "Search" be combined somehow?

Working on this issue should include prototype images (small and large format devices) and discussion.

Javadoc & tests for java utilities

Need javadoc for MutopiaPIece, MutopiaRDF.
Need tests for everything.

HiDPI support in preview images

I would like to investigate HiDPI support in preview images so people with hi-resolution displays (or zoomed in) can see the full effect of LilyPond's typesetting before downloading.

This would involve creating two preview images, one at double the resolution, and using the srcset attribute to choose between them. XKCD does something like this:

http://xkcd.com/1769/
http://imgs.xkcd.com/comics/never_seen_star_wars.png
http://imgs.xkcd.com/comics/never_seen_star_wars_2x.png

Instrument search failure

[ reported in mutopia-discuss by Max Bengtsson ]

If searching for 'Viol', pieces for other instruments are displayed (Violin, Viola, etc.). This is probably the case using our keyword search box, but the advanced search returns a similar search if the instrument pulldown is used and 'Viol' is selected. This should be a more specific search and only pieces for the one instrument should be returned.

Advanced search must be fixed to honor exact matches when requesting specific terms such as instrument, style, and composer.

Icons anybody?

Some love them, some hate them. Can we make our links more concise with icons or ... are they just confusing?

Note that this is marked as a question; I am not against prototyping some ideas to answer the question, does this improve the user experience?

Bootstrap icons are available here: http://getbootstrap.com/components/

Search box design

Implement this search box design by @PaulMorris

From @dominicus regarding the jumbotron search box,

The search box is subdued and relegated. Dark bold outline or add a dark matting, a button with a bold "magnifier" icon. Center horizontally or elongate.

This is the place to consider a search icon. See http://getbootstrap.com/components/ for available icons.

Diacritical insensitive search

This is moved here from issue #77 by @dominicus.

From the home page, do a keyword search for "Pathétique". I get unrelated results.
Go to "Advanced Search", search for keyword="Pathétique" with no other filters. I get no results. Yet we should get at least one hit http://www.mutopiaproject.org/cgibin/piece-info.cgi?id=299

Request: Multi-Language support

Imagine the foreign experience with English-only webpages.
Does the new web template facilitate this in any way?

IE clips lower part of logo in banner

The "banner" is a bootstrap "jumbotron", defined in their CSS.
There are some tweaks in mutopia.css so it wouldn't be so high; the problem is probably there.

Request: left-aligned preview images

Since titles are now left-aligned, it may look better to also align preview images on piece-info pages to the left, so they land below the piece title on piece-info pages.
See sample http://www.mutopiaproject.org/cgibin/piece-info.cgi?id=640

Show top categories by piece count on home page shortened lists

Category lists in the home page are truncated to the first 16 from an alphabetical sort of all existing entries in each category (Instruments, Composers, Styles).
This results in categories with high representation in our repository to not appear, while prime real-estate is devoted to low-count entries:

• Instruments like Piano, Voice, Violin, not shown
• Composers like Bach, Mozart, Beethoven, Giuliani, not shown.

I believe we increase user value by selecting these short lists from the top 16 entries by piece count.

For the "Show All" link, I think it's safe to assume the user at that point is searching for a known entry, in which case it's appropriate to retain the current alphabetical order on the expanded list.

Clicking "hamburger" button does not show a menu

When I view the demo site[0] on a small window/screen, using my phone or the Firefox responsive design view[1], the "hamburger" button (3 horizontal lines) appears on the right side of the menu bar, but clicking it does not show a menu.

[0] http://www.mutopiaproject.org/new/index.html
[1] https://developer.mozilla.org/en-US/docs/Tools/Responsive_Design_View

Working prototype #2

Since there have been several revisions to the code base and a new version of bootstrap, the task is to create a new iteration of the prototype. This also gives us the opportunity to incorporate some of the design work and discussion in issue #32 and, specifically, Paul's prototype 18.

Several folders can be seen from http://www.mutopiaproject.org/

Only the ftp folder should be visible directly from the URL, that is, http://www.mutopiaproject.org/ftp.

The following folders need an .htaccess file to prevent them from being viewed from the website:

• cgibin
• images
• datafiles
• css
• bootstrap-3.3.5-dist

Browse page and link references

The browse page contains a list of collections followed beneath with a table of instruments, composers, and styles that the user can use to refine a search. The problem is with the tabular data since links arriving here from the home page always appear to arrive at the same place (the top of the table.) A temporary solution in the first release leaves the browse page untouched but the links from the home page go to individual pages.

For discussion:
The "temporary solution" is redundant. Do we,

1. Deprecate the browse page in favor of individual pages.
2. Convert the browse page to a list using bootstrap's flexible grids. Links arriving from the home page will traverse into the browse anchors more favorably than a table (i.e., they won't appear to go to the same place.)

"Inappropriate" FTP areas (files for 2236 and 2238 swapped)

@glenl @chrissawer

Recent submissions for two different editions of MozartWA/KV265 were placed in each other's "appropriate FTP areas" on the web, despite being placed correctly in their respective directories under MutopiaProject.

These are the two editions:
2236: guitar-duo-complete (all 13 variations)
2238: guitar-duo-varV (just 1 variation)

Here is the correct directory structure under MutopiaProject:
$ cd /f/MutopiaProject/ftp/MozartWA
$ find KV265
KV265
KV265/guitar-duo-complete
KV265/guitar-duo-complete/guitar-duo-complete-lys
KV265/guitar-duo-complete/guitar-duo-complete-lys/guitar-duo-complete.ly
KV265/guitar-duo-complete/guitar-duo-complete-lys/preview.svg
KV265/guitar-duo-complete/guitar-duo-complete-lys/text-macros.ily
KV265/guitar-duo-varV
KV265/guitar-duo-varV/guitar-duo-varV.ly

Because the moreInfo field for 2238 links to 2236, you can see both ids by searching for 2236:
http://www.mutopiaproject.org/cgibin/make-table.cgi?searchingfor=2236

Go to that page and mouseover the appropriate ftp areas and letter pdf file link.

The ftp area and letter pdf for the "Var. V" edition (see title) are inappropriately under guitar-duo-complete:
http://www.mutopiaproject.org/ftp/MozartWA/KV265/guitar-duo-complete/
http://www.mutopiaproject.org/ftp/MozartWA/KV265/guitar-duo-complete/guitar-duo-varV-let.pdf

Let me highlight that: guitar-duo-complete/guitar-duo-varV-let.pdf

Likewise the ftp area and content for the complete edition are inappropriately under guitar-duo-varV
http://www.mutopiaproject.org/ftp/MozartWA/KV265/guitar-duo-varV/
http://www.mutopiaproject.org/ftp/MozartWA/KV265/guitar-duo-varV/guitar-duo-complete-let.pdf

It gets more complicated.

Since the complete edition has multiple lys, the guitar-duo-complete.ly file and its companion text-macros.ily file are (correctly) under a guitar-duo-complete-lys directory:
http://www.mutopiaproject.org/ftp/MozartWA/KV265/guitar-duo-complete/guitar-duo-complete-lys/
even though that directory is mistakenly under the ftp area for the Var. V edition.

But if you go to the id listing for 2236 to look for those files ...
http://www.mutopiaproject.org/cgibin/piece-info.cgi?id=2236

... you'll see that the lys zip file (for complete 2236) is listed under varV:
http://www.mutopiaproject.org/ftp/MozartWA/KV265/guitar-duo-varV/guitar-duo-complete-lys.zip

Search fixes

There seems to be something needing fix with search function.
Case1:
From home page, select "Beethoven Piano Sonatas" link from the collections section. The expected first page in the collection is displayed. Click on "Next 10", an unrelated set of pieces is shown. Seems the "next 10" link for collections is not being formed properly.

Case2:
From the home page, do a keyword search for "Pathétique". I get unrelated results.
Go to "Advanced Search", search for keyword="Pathétique" with no other filters. I get no results. Yet we should get at least one hit http://www.mutopiaproject.org/cgibin/piece-info.cgi?id=299

Case3:
From home page, search for "Beethoven". I get unrelated results.
Repeat keyword search "Beethoven" from "Advanced Search" page, I get expected results.

Case4:
From "advanced search", no keywords, request pieces added/updated in last 3 weeks. The results include pieces that go beyond the time range.

Impossible to search for terms with non-ASCII characters like é or ü

Personal motivation: I wanted to search for Burgmüller but there is no way to make it work. In that case there is a workaround by browsing through the composer list. But the issue also applies to searching for étude or "für elise".

** I have a small code change that corrects this, but in order to make a pull request I need for this project to have a clearly indicated license, which I don't see. That's not for me, but for my employer, who allows me to submit patches like this easily with many licenses.

If you'd rather just make the change yourself:

• Input from the URL needs to be decoded from UTF-8, as with https://perldoc.perl.org/Encode.html
• Uppercasing of the search terms should use uc() or some other approach that handles more than ASCII characters.

Search box design

From @dominicus regarding the jumbotron search box,

The search box is subdued and relegated. Dark bold outline or add a dark matting, a button with a bold "magnifier" icon. Center horizontally or elongate.

This task should drive a discussion on design as well as a working implementation.

Music listing enhancements

From issue #57

• The titles for search results "Music Listing - with preview images" or "Music Listing - without preview images" could be re-worded. I would drop the "with/without image warning". I'm guessing the intent of adding the "- without preview images" warning is to make the user aware that showing previews is possible...in case they didn't read through all the advanced options. Maybe more productive than a warning could be adding a button at top where the user can toggle "show preview images"/"hide preview images" while reviewing results. Preview images are a welcome feature for those who can quickly sight-read music.

Other/wishful:

• For multi-page results, a subtitle or right-aligned indication of which page is currently displayed (e.g. "results 21-30")?
• An option in advanced search to request pieces added in the last X weeks (choice to include or exclude updates). Maybe this is a feature best tackled when there's a running database?
• Would be nice to have a link at the top of search results to "refine search". This would show the Advanced Search screen with the previously applied search terms. As it is right now, the only way to refine an existing search while on page X is to hit back-button however many pages you may be into the results.

Develop a piece download counter or other means to sort by piece download popularity

Moved from MutopiaProject/MutopiaProject#10 (12/12/2012)
"Suggestion by Paul Morris" @PaulMorris

@dominicus:
Found this setup to count file download requests on the Apache server side.
http://www.webdesignoutsource.net/download-counter-with-apache-and-php/