szabgab / metacpan-sco Goto Github PK

6.0 7.0 4.0 885 KB

Open source search.cpan.org clone with MetaCPAN API back-end

Perl 100.00%

metacpan-sco's Introduction

search.cpan.org clone based on MetaCPAN API

This is an attempt to recreate the look-and-feel of http://search.cpan.org/ using the
MetaCPAN API as a back-end. https://github.com/CPAN-API/cpan-api/

For objectives, plans, and a description of what the site, and the code behind it should look like see
http://perlmaven.com/search-cpan-org

For a live version of this application visit http://sco.perlmaven.com/


Development environment:
------------------------

* Install the prerequisites via Makefile.PL
* Run bin/fetch.pl to fetch some meta-meta-data from MetaCPAN::API
* Run plackup -r
* Borwse to http://localhost:5000/


Enhancements
=============

Various places where the clone differs from the real sco that can be considered enhancements.

The permalink in the top-right corner of
--------------
   http://sco.perlmaven.com/~szabgab/CPAN-Test-Dummy-SCO-Special-0.04/lib/CPAN/Test/Dummy/SCO/Special.pm
  links directly to the permanent url and not to /perldoc?


#18

Canonical URLs
---------------
All the version-specific files set the non-version specific file as the canonical URL.
For example
       http://search.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Special-0.04/lib/CPAN/Test/Dummy/SCO/Separate.pod
  sets http://search.cpan.org/dist/CPAN-Test-Dummy-SCO-Special/lib/CPAN/Test/Dummy/SCO/Separate.pod

       http://search.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Special-0.04/
  sets http://search.cpan.org/dist/CPAN-Test-Dummy-SCO-Special/

See https://support.google.com/webmasters/answer/139066?hl=en
#17


Redirect URLs
--------------

Historically many people have been linking to version specific files on search.cpan.org.
When that version of the module is removed from CPAN, these links become broken.

Instead of showing a 'Not found' error sco. could redirect those URLs to the stable URL.

For example

    http://saerch.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Pirated-1.02/lib/CPAN/Test/Dummy/SCO/Pirated.pm
 to http://sco.perlmaven.com/dist/CPAN-Test-Dummy-SCO-Pirated/lib/CPAN/Test/Dummy/SCO/Pirated.pm

and

    http://saerch.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Pirated-1.02/
 to http://sco.perlmaven.com/dist/CPAN-Test-Dummy-SCO-Pirated/

See #19


Author lists
-------------

All these pages used to server the same content:

   http://search.cpan.org/author/?a
   http://search.cpan.org/author/?A
   http://search.cpan.org/author/?Abc

And then pages with different query_string had different content.

Instead of that now all of them redirect to

   http://search.cpan.org/author/A

metacpan-sco's People

Contributors

Stargazers

Watchers

Forkers

shlomif mishin grian stro

metacpan-sco's Issues

Redirect old URLs

Historically many people have been linking to version specific files on search.cpan.org.
When that version of the module is removed from CPAN, these links become broken.

Instead of showing an 'Not found' error sco. could redirect those URLs to the stable URL.

This is only relevant if this code is used to run sco.

For example

http://sco.perlmaven.com/~szabgab/CPAN-Test-Dummy-SCO-Pirated-1.03/lib/CPAN/Test/Dummy/SCO/Pirated.pm is going to be deleted soon. It should redirect to
http://sco.perlmaven.com/dist/CPAN-Test-Dummy-SCO-Pirated/lib/CPAN/Test/Dummy/SCO/Pirated.pm

In order to avoid the need to keep track of all the old releases we might want to redirect even urls such as
http://sco.perlmaven.com/~szabgab/CPAN-Test-Dummy-SCO-Pirated-1.02/lib/CPAN/Test/Dummy/SCO/Pirated.pm
that were never on CPAN.

"Related Modules" are missing

http://search.cpan.org/dist/HTML-Template/lib/HTML/Template.pm has a box called "Related Modules" on the right hand side. This is some information apparently fetched from PerlMonks.

Latest release link missing

http://search.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Special-0.02/ has it

http://sco.perlmaven.com/~szabgab/CPAN-Test-Dummy-SCO-Special-0.02/ does not have it.

Version specific files should be excluded from indexing

In order to get search engines bring people to the stable url, we should disallow the indexing of the version specific files. Either in robots.txt or in the header of each file. (This is only relevant if the code becomes the real sco)

disallow http://search.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Special-0.04/lib/CPAN/Test/Dummy/SCO/Separate.pod

allow:
http://search.cpan.org/dist/CPAN-Test-Dummy-SCO-Special/lib/CPAN/Test/Dummy/SCO/Separate.pod

The site should be much faster

A couple of directions:

*) Caching results
*) Executing API requests asynchronously
*) Asking for less data (use fields where possible)

*) Hosting it on a faster box closer to the location of the API server

Request for "Spreadsheet" returns "Internal Server Error"

CGI-Simple does not show bug count

http://search.cpan.org/~szabgab/CGI-Simple-1.115/lib/CGI/Simple.pm shows numbers on the right hand side
On http://sco.perlmaven.com/~szabgab/CGI-Simple-1.115/lib/CGI/Simple.pm they are empty.

It is an issue of case sensitivity.

See metacpan/metacpan-api#356
(request for renaming sent to RT admins)

Use canonical URLs

All the version-specific files should should set the non-version specific file as the canonical URL
for example http://search.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Special-0.04/lib/CPAN/Test/Dummy/SCO/Separate.pod should set http://search.cpan.org/dist/CPAN-Test-Dummy-SCO-Special/lib/CPAN/Test/Dummy/SCO/Separate.pod

See https://support.google.com/webmasters/answer/139066?hl=en

This is interesting only if this code will be used for the real sco.
See also #18

HTML-Template-2.95 marked as unauthorized

This http://sco.perlmaven.com/~wonko/HTML-Template-2.95/ is marked as unauthorized while http://search.cpan.org/~wonko/HTML-Template-2.95/ is not.
In this case the clone is actually more correct.

See metacpan/metacpan-api#357

CPAN::Test::Dummy::SCO::Separate needs to be listed under modules

Because there is both .pm (without any pod) and a .pod version of the file.

I am not sure about this. but that's how sco does it in http://search.cpan.org/~szabgab/CPAN-Test-Dummy-SCO-Special-0.04/

/dist/CPAN-Test-Dummy/ lists suggestions

This gives a nice list of suggestions http://search.cpan.org/dist/CPAN-Test-Dummy/
http://search.cpan.org/dist/CPAN-Test-Dummy-x/ too.

OTOH http://sco.perlmaven.com/dist/CPAN-Test-Dummy/
and http://sco.perlmaven.com/dist/CPAN-Test-Dummy-x/ show some broken page.

Pages work as expected but...

The author page takes several seconds to load, long enough that I thought no authors were listed and had already started to move on to the next page.

No matches found

When I type in a partial module name, e.g. 'MarpaX', or 'MarpaX::Demo', or a full module name, e.g. 'MarpaX::Demo::StringParser', I always get the no matches found msg. Is it meant to be working?

/dist/Dist-Name pages should be different from ~pauseid/Dist-Name-1.12

They should have slightly different content.

Other tools /tools need to be implemented

Look and Feel

When I deliberately type in a partial module name, e.g. 'MarpaX', the search box, which starts off in the middle of the screen (which I like), is displayed next, with the no matches msg, high up on the left. That seems odd, even though it's a small problem.