skaji/cpm

fast CPAN client

cpm

a fast CPAN module installer

Install

There are 2 ways.

1) self-contained version

$ curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm
$ chmod +x cpm
$ ./cpm --version

2) From CPAN

$ curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm | perl - install -g App::cpm
$ cpm --version

Description

cpm is a fast CPAN module installer, which uses Menlo (cpanm 2.0) in parallel.

Moreover cpm keeps the each builds of distributions in your home directory. Then, cpm install will use these prebuilt distributions. That is, if prebuilts are available, cpm never build distributions again, just copy the prebuilts into an appropriate directory. This is (of course!) inspired by Carmel.

Roadmap

See #181

License

Copyright 2015 Shoichi Kaji [email protected]

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See Also

• You may want to check install-with-cpm when you use cpm with GitHub Actions.
• Perl Advent Calendar 2015
• App::cpanminus
• Menlo
• Carton

cpm's Issues

wishlist: Support --mirror /some/path in addition to --mirror file:/some/path

like cpanm.

Failure to run the app

C:\> cpm install --verbose Code::TidyAll
Use of uninitialized value $ENV{"HOME"} in concatenation (.) or string at I:/Dwimperl/perl/site/lib/App/cpm.pm line 25.
local: No such file or directory at I:/Dwimperl/perl/site/lib/App/cpm.pm line 64

I use the latest Dwimperl distribution, on Windows 10 x64. Installed as written via cpanm with no issues.

stable API server

Currently this module uses an API https://cpanmetadb-provides.herokuapp.com/ which running on heroku.

If I release this module on cpan, I have to:

• fix spec of API server
• setup it in more reliable way

More diagnostics in case of failures

For one of my github projects with a large dependency list I replaced cpanm with cpm for travis-ci testing. This gives me some speed improvements (formerly ~300s for dependency installation, now ~200s). But unfortunately there are occasional fails:

• https://travis-ci.org/eserte/bbbike/jobs/175221900 (1st attempt)
  perl 5.10.1 -> FAIL configure (0.115sec) File-Share-0.25
• https://travis-ci.org/eserte/bbbike/jobs/175221900 (2nd attempt)
  perl 5.10.1 -> FAIL configure (0.138sec) Array-Heap-3.21
• https://travis-ci.org/eserte/bbbike/jobs/175534860
  perl 5.8.8 -> FAIL configure (0.183sec) JSON-2.90
• https://travis-ci.org/eserte/bbbike/jobs/177167612
  perl 5.8.8 -> FAIL configure (0.165sec) JSON-2.90

So this happens only for older perls (the tested perl versions here range from 5.8 to 5.20). I see no pattern in the affected CPAN modules. And successful installations with these perls may also happen (for example this build: https://travis-ci.org/eserte/bbbike/builds/175858436 )

Unfortunately I don't know a way in finding out the reasons for the failures. It would be nice if there was more information in case of errors, e.g. output the log for the failing module at the end of the installation. Or maybe there's already a log file of the build process, but it's not documented.

Manpage not installed?

I don't know yet if it's a Menlo issue or cpm. But when I install using cpanm (1.7040):

% cpanm -n App::CSVUtils

then:

% cpanm --uninstall App::CSVUtils

the result is:

App::CSVUtils contains the following files:

  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-add-field
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-delete-field
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-list-field-names
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-munge-field
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-replace-newline
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csvutil
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/replace-newline-in-csv
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0/App/CSVUtils.pm
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/csv-add-field.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/csv-delete-field.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/csv-list-field-names.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/csv-munge-field.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/csv-replace-newline.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/csvutil.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man1/replace-newline-in-csv.1
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/man/man3/App::CSVUtils.3

Are you sure you want to uninstall App::CSVUtils? [y]

But if I use cpm (0.211):

% cpm install -g App::CSVUtils

then:

% cpanm --uninstall App::CSVUtils

the result is:

App::CSVUtils contains the following files:

  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-add-field
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-delete-field
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-list-field-names
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-munge-field
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csv-replace-newline
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/bin/csvutil
  /home/u1/perl5/perlbrew/perls/perl-5.24.0/lib/site_perl/5.24.0/App/CSVUtils.pm

Are you sure you want to uninstall App::CSVUtils? [y]

There are no manpages being installed by cpm.

cpm [options] install

Hi, the 'cpm [options] install Module' syntax is actually not working: you should say 'cpm install Module [options]' instead - can you please either make it work as documented or fix the documentation?
App::cpm 0.111

check whether packages are installed or not

Currently cpan-installer always install all distributions even if they are already installed.
Should be skipped.

Log file is messy

All workers currently emit to a single file, resulting in a mixed and messy log file. (added after YAPC talk =)

Respect local::lib / perlbrew settings

Hi.

It would be great if App::cpm would respect already available environment settings of local::lib and perlbrew - and would install modules to the appropriate locations.

Thanks!

Support adjusting the Menlo configure_timeout

I'm using cpm on a very slow machine. I want to use cpm because the machine is very slow and cpm will speed up the install a little bit. But sometimes a large dist like Moose will take more than 60 seconds for the configure stage, causing the whole build to bail out with "Terminated". I would like to have the option to wait longer :)

Fails to install/parse meta data from path

cpanm install ../My-Module and cpanm install ../My-Module.tar.gz just work.

But I can't figure out how to get this same behavior from cpm. I'd expect both of the following to work, but they don't:
cpm install ../My-Module and cpm install ../My-Module.tar.gz

better error message for circular deps

currently:

> cpm install -v CPAN::Test::Dummy::Perl5::Make::CircDepeOne
47799 DONE resolve   (0.482sec) CPAN::Test::Dummy::Perl5::Make::CircDepeOne -> CPAN-Test-Dummy-Perl5-Make-CircDepeOne-1.00 (from MetaDB)
47799 DONE fetch     (0.226sec) CPAN-Test-Dummy-Perl5-Make-CircDepeOne-1.00
47799 DONE configure (0.173sec) CPAN-Test-Dummy-Perl5-Make-CircDepeOne-1.00
47799 DONE resolve   (0.182sec) CPAN::Test::Dummy::Perl5::Make::CircDepeThree -> CPAN-Test-Dummy-Perl5-Make-CircDepeThree-1.00 (from MetaDB)
47799 DONE fetch     (0.152sec) CPAN-Test-Dummy-Perl5-Make-CircDepeThree-1.00
47799 DONE configure (0.173sec) CPAN-Test-Dummy-Perl5-Make-CircDepeThree-1.00
47799 DONE resolve   (0.201sec) CPAN::Test::Dummy::Perl5::Make::CircDepeTwo -> CPAN-Test-Dummy-Perl5-Make-CircDepeTwo-1.00 (from MetaDB)
47799 DONE fetch     (0.154sec) CPAN-Test-Dummy-Perl5-Make-CircDepeTwo-1.00
47799 DONE configure (0.171sec) CPAN-Test-Dummy-Perl5-Make-CircDepeTwo-1.00
0 distribution installed.

need to figure out why cpm fails

cc @eserte

As reported in #51, cpm sometimes fails to install some modules, especially with old perls.

I confirmed that cpm failed to install $module with perl 5.10:

Cpanel::JSON::XS (via cpm install Plack)

2016-12-23T18:27:00,25429,Cpanel-JSON-XS-3.0225| Unpacking Cpanel-JSON-XS-3.0225.tar.gz
2016-12-23T18:27:00,49090,Cpanel-JSON-XS-3.0225| Checking if your kit is complete...
2016-12-23T18:27:00,49090,Cpanel-JSON-XS-3.0225| Looks good
2016-12-23T18:27:00,49090,Cpanel-JSON-XS-3.0225| Generating a Unix-style Makefile
2016-12-23T18:27:00,49090,Cpanel-JSON-XS-3.0225| Writing Makefile for Cpanel::JSON::XS
2016-12-23T18:27:00,49090,Cpanel-JSON-XS-3.0225| Writing MYMETA.yml and MYMETA.json
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| cp XS/Boolean.pm blib/lib/Cpanel/JSON/XS/Boolean.pm
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| cp XS.pm blib/lib/Cpanel/JSON/XS.pm
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| Running Mkbootstrap for XS ()
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| chmod 644 "XS.bs"
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| "/Users/skaji/env/plenv/versions/5.10.1/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- XS.bs blib/arch/auto/Cpanel/JSON/XS/XS.bs 644
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| "/Users/skaji/env/plenv/versions/5.10.1/bin/perl" "/Users/skaji/env/plenv/versions/5.10.1/lib/perl5/5.10.1/ExtUtils/xsubpp"  -typemap '/Users/skaji/env/plenv/versions/5.10.1/lib/perl5/5.10.1/ExtUtils/typemap' -typemap '/Users/skaji/.perl-cpm/work/Cpanel-JSON-XS-3.0225/typemap'  XS.xs > XS.xsc
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| Undefined subroutine &ExtUtils::ParseXS::errors called at /Users/skaji/env/plenv/versions/5.10.1/lib/perl5/5.10.1/ExtUtils/xsubpp line 41.
2016-12-23T18:27:01,49884,Cpanel-JSON-XS-3.0225| make: *** [XS.c] Error 255

https://gist.github.com/skaji/0b4c5d5acefd17f95b9ab08fedd462ab

HTTP::Entity::Parser (via cpm install Plack)

2016-12-23T18:32:40,74375,HTTP::Entity::Parser| Resolved HTTP::Entity::Parser (0.17) -> http://www.cpan.org/authors/id/K/KA/KAZEBURO/HTTP-Entity-Parser-0.18.tar.gz from MetaDB
2016-12-23T18:32:40,74375,HTTP-Entity-Parser-0.18| Unpacking HTTP-Entity-Parser-0.18.tar.gz
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| Can't locate CPAN/Meta/Prereqs.pm in @INC (@INC contains: /Users/skaji/work/2016-12-18/local/lib/perl5/5.10.1/darwin-2level /Users/skaji/work/2016-12-18/local/lib/perl5/5.10.1 /Users/skaji/work/2016-12-18/local/lib/perl5/darwin-2level /Users/skaji/work/2016-12-18/local/lib/perl5 /Users/skaji/env/plenv/versions/5.10.1/lib/perl5/5.10.1/darwin-2level /Users/skaji/env/plenv/versions/5.10.1/lib/perl5/5.10.1 /Users/skaji/env/plenv/versions/5.10.1/lib/perl5/site_perl/5.10.1/darwin-2level /Users/skaji/env/plenv/versions/5.10.1/lib/perl5/site_perl/5.10.1 .) at /Users/skaji/work/2016-12-18/local/lib/perl5/CPAN/Meta/Feature.pm line 8.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| BEGIN failed--compilation aborted at /Users/skaji/work/2016-12-18/local/lib/perl5/CPAN/Meta/Feature.pm line 8.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| Compilation failed in require at /Users/skaji/work/2016-12-18/local/lib/perl5/CPAN/Meta.pm line 57.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| BEGIN failed--compilation aborted at /Users/skaji/work/2016-12-18/local/lib/perl5/CPAN/Meta.pm line 57.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| Compilation failed in require at /Users/skaji/work/2016-12-18/local/lib/perl5/Module/Build/Tiny.pm line 8.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| BEGIN failed--compilation aborted at /Users/skaji/work/2016-12-18/local/lib/perl5/Module/Build/Tiny.pm line 8.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| Compilation failed in require at Build.PL line 9.
2016-12-23T18:32:40,81529,HTTP-Entity-Parser-0.18| BEGIN failed--compilation aborted at Build.PL line 9.

I'm not sure, but...

• local::lib issue ?
• the order of @INC ?

idea: make MetaCPAN resolver primary

MetaCPAN index API (eg: https://fastapi.metacpan.org/v1/download_url/App::cpm) has some advantage over other resolvers:

• it indexes almost all versions of distributions (i.e it indexes App-cpm-0.200, App-cpm-0.201, .. at the same time)
• it can index distirbuitons faster than other resolvers (eg 02packages.details.txt.gz)
• it knows the status (latest, cpan, backpan) of distributions
• it indexes develop (TRIAL) releases
• it is developed by active MetaCPAN community

So why not make MetaCPAN resolver primary?

However, I have noticed that

• sometimes MetaCPAN index does not match PAUSE index
• MetaCPAN is slower than MetaDB (which is critical for cpm :-) )

# MetaDB: 0.061~0.325sec
> cpm install -w1 -v -r metadb Moose Mouse Moo Mo Plack Dancer Catalyst
38197 DONE resolve   (0.325sec) Catalyst -> Catalyst-Runtime-5.90112 (from MetaDB)
38197 DONE resolve   (0.133sec) Moose -> Moose-2.1806 (from MetaDB)
38197 DONE resolve   (0.065sec) Moo -> Moo-2.002005 (from MetaDB)
38197 DONE resolve   (0.068sec) Dancer -> Dancer-1.3202 (from MetaDB)
38197 DONE resolve   (0.061sec) Mo -> Mo-0.40 (from MetaDB)
38197 DONE resolve   (0.063sec) Mouse -> Mouse-v2.4.5 (from MetaDB)
38197 DONE resolve   (0.211sec) Plack -> Plack-1.0042 (from MetaDB)

# MetaCPAN: 0.277~1.356sec
> cpm install -w1 -v -r metacpan Moose Mouse Moo Mo Plack Dancer Catalyst
38248 DONE resolve   (1.356sec) Catalyst -> Catalyst-Runtime-5.90112 (from MetaCPAN)
38248 DONE resolve   (0.614sec) Plack -> Plack-1.0042 (from MetaCPAN)
38248 DONE resolve   (0.557sec) Moo -> Moo-2.002005 (from MetaCPAN)
38248 DONE resolve   (0.323sec) Mo -> Mo-0.40 (from MetaCPAN)
38248 DONE resolve   (0.447sec) Mouse -> Mouse-v2.4.5 (from MetaCPAN)
38248 DONE resolve   (0.614sec) Dancer -> Dancer-1.3202 (from MetaCPAN)
38248 DONE resolve   (0.277sec) Moose -> Moose-2.1806 (from MetaCPAN)

Resolve script name?

Hi Shoichi,

This is just a random idea that you might want to consider.

Often I want to install a script from CPAN, but I don't know the name of the module. So I need to open browser, go to metacpan.org, and search for it (or, use my local CPAN mirror tool lcpan and type lcpan script2mod SCRIPTNAME).

Since now cpm has resolver modules, perhaps a resolver module can be written so that something like:

% cpm install csv-add-field

can be resolved to:

% cpm install App::CSVUtils

I've also written a simple script to demonstrate this for cpanm:

https://metacpan.org/source/PERLANCAR/App-cpanm-script-0.001/bin/cpanm-script

Please tell me what you think. I'm open to writing a separately distributed App::cpm::Resolver::* by myself.

cpm don't support pinto

Hello, we use pinto as our darkpan and cpm don't work with him...

cpm install -v --mirror "http://pinto.srv.int.avast.com:3111" App::Phoenix::Zorro

25492 FAIL resolve   (0.162sec) App::Phoenix::Zorro
FAIL resolve App::Phoenix::Zorro
0 distribution installed.

For old perls: should merge EUMM / EUPXS reqs

❯ ~/env/plenv/versions/5.10.1/bin/perl cpm install ExtUtils::[email protected]
DONE install ExtUtils-MakeMaker-7.28
DONE install ExtUtils-ParseXS-3.30
DONE install ExtUtils-MakeMaker-7.26
3 distribution installed.

Race conditions possible (installation + use of modules may happen at the same time)

Today I got a mysterious failure when using cpm on a travis-ci system. The build log with the failure can be seen here: https://travis-ci.org/eserte/bbbike/jobs/198528696#L5531

The error message ("Getopt/Long.pm did not return a true value") and the fact that the installation of Getopt::Long is reported a few log lines later suggests that the failure was caused by a use Getopt::Long while the installation of Getopt/Long.pm was in progress, so the calling script saw an incomplete file.

This is a new failure class which cannot happen in traditional serial installers. I don't have a good idea how to solve this. First, this possible problem should probably be documented. A possible fix could be to make sure that the installation step happens when no other build/configure/test processes run in parallel, but this would mean to slow down the whole installation process. Another possibility is to let these errors happen, but retry automatically once (after all, I am using cpm since some months and it's the first such failure). Finally it would probably be better to have the installation process more atomic. Currently ExtUtils::Install does a copy() of modules, but it would be more robust to copy to a temporary file on the destination filesystem, and then do an atomic rename(). Still it is not a real transaction if many files belonging to a distribution are to be installed, but at least incomplete file reads could not happen anymore.

cpanfile recursive resolving 2

Hi, Shoichi.

I have already asked a similar question but I'd like to reformulate it more clearly.

cpm can install dependencies from cpanfile. That's great!

Unfortunately cpm reads cpanfile only in installed distributive itself. It means that if distributive X has a dependency from distributive Y and both distributives have cpanfile then the only cpanfile that will be used is one in distributive X. cpanfile from distributive Y won't be used.

Yes, I remember you wrote here that dependencies should be described in META. But cpm already can read dependencies not from META but from cpanfile. It seems to me there's a good reason to take next step to make cpm read cpanfile not only in installed distributive but in its' dependencies as well.

What is it for? The main reason is the ability to specify repositories git in cpanfile. If cpm reads dependencies recursively from cpanfile then there will appear an ability to install all the dependencies exactly from git without using darkpan.

The ability to install dependencies directly from git without using darkpan is extremely convenient.

You can make this ability optional.

For example:

--use-cpanfile - to read cpanfile in installed distributive. This behavior already exists, this option is not needed and I bring it here just for clarity.
--use-cpanfile-in-deps - to read cpanfile in dependencies, recursively.

Shoichi, please, add this ability:)

Fail in configuring modules with exotic MB and EUMM setups

Hi @skaji, firstly thanks for making App::cpm!

I'm trying to install for a local dist having this cpanfile:

requires 'Mojolicious', '7.0';
requires 'Data::Utilities', '0.04';

However, cpm fails to do so:

θ61° [zakame:~/hoozuki] 21s % cpm install
DONE install Module-Build-0.4220
DONE install Mojolicious-7.29
FAIL install Data-Utilities-0.04
2 distributions installed.  
See /home/zakame/.perl-cpm/build.log for details. 

Checking the log shows this:

2017-03-22T14:16:36,1829,Data::Utilities| Resolved Data::Utilities (0.04) -> http://www.cpan.org/authors/id/C/CO/CORNELIS/Data-Utilities-0.04.tar.gz from MetaDB
2017-03-22T14:16:36,1829,Data-Utilities-0.04| Using cache /home/zakame/.perl-cpm/cache/authors/id/C/CO/CORNELIS/Data-Utilities-0.04.tar.gz
2017-03-22T14:16:36,1829,Data-Utilities-0.04| Unpacking Data-Utilities-0.04.tar.gz
2017-03-22T14:16:36,1829,Data-Utilities-0.04| Configuring distribution
2017-03-22T14:16:36,1851,Data-Utilities-0.04| Checking if your kit is complete... 
2017-03-22T14:16:36,1851,Data-Utilities-0.04| Looks good
2017-03-22T14:16:36,1851,Data-Utilities-0.04| Generating a Unix-style Makefile
2017-03-22T14:16:36,1851,Data-Utilities-0.04| Writing Makefile for Data::Utilities
2017-03-22T14:16:36,1851,Data-Utilities-0.04| Writing MYMETA.yml and MYMETA.json
2017-03-22T14:16:36,1829,Data-Utilities-0.04| ! Retrying (you can turn off this behavior by --no-retry) 2017-03-22T14:16:36,1922,Data-Utilities-0.04| Generating a Unix-style Makefile
2017-03-22T14:16:36,1922,Data-Utilities-0.04| Writing Makefile for Data::Utilities
2017-03-22T14:16:36,1922,Data-Utilities-0.04| Writing MYMETA.yml and MYMETA.json
2017-03-22T14:16:36,1829,Data-Utilities-0.04| Failed to configure distribution

Taking a look with cpanm --look Data::Utilities show a quite unusual setup of having both a Build.PL and Makefile.PL, (since it is an older module that uses Module::Install,) and furthermore, its Build.PL produces a Makefile instead of the expected Build:

θ62° [zakame:~] 9s % cpanm --look Data::Utilities 
--> Working on Data::Utilities Fetching file:///home/ftp/pub/CPAN/authors/id/C/CO/CORNELIS/Data-Utilities-0.04.tar.gz ... OK
Entering /home/zakame/.cpanm/work/1490163513.5051/Data-Utilities-0.04 with /bin/zsh
θ68° [zakame:~/.cpanm … Data-Utilities-0.04] % ls 
Build.PL lib Makefile.PL META.yml t Changes LICENSE MANIFEST README Todo 
θ61° [zakame:~/.cpanm … Data-Utilities-0.04] % cat Build.PL 
# Dear Distribution Packager. This use of require is intentional.  
# Module::Install detects Build.PL usage and acts accordingly.  
require 'Makefile.PL'; 
θ63° [zakame:~/.cpanm … Data-Utilities-0.04] % perl Build.PL 
Checking if your kit is complete...  Looks good 
Generating a Unix-style Makefile 
Writing Makefile for Data::Utilities 
Writing MYMETA.yml and MYMETA.json 
θ63° [zakame:~/.cpanm … Data-Utilities-0.04] % ls 
Build.PL lib Makefile MANIFEST MYMETA.json README Todo 
Changes LICENSE Makefile.PL META.yml MYMETA.yml t

Digging around App::cpm::Worker::Installer->configure(), I found that it doesn't try different configurators until it can get a proper Build or Makefile. I'll make a PR that tries to fix this.

02packages cache not getting updated

We have been running into an issue where the 02packages cache in .perl-cpm/sources is not always getting updated correctly. Our darkpan lives in a Git repo with multiple branches. We pass it to cpm using --resolver 02packages,file://.... What I think is happening is that sometimes the cache has a newer mtime even though it is out of date. This can happen when the user uses an older checkout with cpm and then later tries to install using the newer version.

02packages resolver should fail loudly if URI doesn't match /(file|https?):\/\//

When using the 02packages resolver with a minicpan on the local filesystem, I accidentally forgot the file:// prefix. cpm install appeared to mostly work except it would fail on FETCH for no specific reason due to Menlo::CLI::Compat thinking that the URI should be requested using the HTTP mirroring rather than the file mirroring. It took embarrassingly long time for me to figure out what was wrong.

check version requirements

Currently cpan-installer never check version requirements. Supprise!
Must fix this.

Silently fails to install some modules

I've decided to try App::cpm for a new (local) Perl installation using Perl 5.20 under Debian Jessie, so I set up local lib as usual by setting PERL5LIB, PERL_M{B,M}_OPT, then installed cpm and tried installing the modules I need:

% cpm install Mojo::DOM WWW::Mechanize
DONE install IO-Socket-IP-0.38
DONE install URI-1.71
DONE install LWP-MediaTypes-6.02
DONE install HTML-Tagset-3.20
DONE install HTTP-Date-6.02
DONE install Encode-Locale-1.05
DONE install IO-HTML-1.001
DONE install Net-HTTP-6.09
DONE install File-Listing-6.04
DONE install WWW-RobotRules-6.02
DONE install HTTP-Message-6.11
DONE install Mojolicious-7.03
DONE install HTTP-Negotiate-6.01
DONE install HTTP-Daemon-6.01
DONE install HTTP-Cookies-6.01
DONE install Try-Tiny-0.27
DONE install Test-Fatal-0.014
DONE install HTML-Parser-3.72
DONE install HTML-Form-6.03
DONE install HTML-Tree-5.03
DONE install libwww-perl-6.15
DONE install WWW-Mechanize-1.78
22 distributions installed.

Notice how Mojo::DOM isn't in the list! Moreover:

% cpm install Mojo::DOM
DONE install Mojo::DOM is up to date. (undef)
0 distribution installed.

While

% cpanm Mojo::DOM
--> Working on Mojo::DOM
Fetching http://www.cpan.org/authors/id/S/SR/SRI/Mojolicious-7.03.tar.gz ... OK
Configuring Mojolicious-7.03 ... OK
==> Found dependencies: IO::Socket::IP
--> Working on IO::Socket::IP
Fetching http://www.cpan.org/authors/id/P/PE/PEVANS/IO-Socket-IP-0.38.tar.gz ... OK
Configuring IO-Socket-IP-0.38 ... OK
Building and testing IO-Socket-IP-0.38 ... OK
Successfully installed IO-Socket-IP-0.38 (upgraded from 0.29)
Building and testing Mojolicious-7.03 ... OK
Successfully installed Mojolicious-7.03
2 distributions installed

Looks like a bug to me...

If cpm supports old perls (such as 5.8) seriously, need to upgrade toolchain

https://github.com/miyagawa/cpanminus/blob/devel/lib/App/cpanminus/script.pm#L981

some distributions are installed twice

https://travis-ci.org/skaji/cpm/jobs/241042064
perl 5.8

perl /tmp/cpm install -g --with-develop
DONE install CPAN-Meta-Requirements-2.140
DONE install CPAN-DistnameInfo-0.12
DONE install local-lib-2.000023
DONE install Class-Tiny-1.006
DONE install JSON-PP-2.94
DONE install HTTP-Tiny-0.070
DONE install Parallel-Pipes-0.003
DONE install HTTP-Tinyish-0.12
DONE install JSON-2.94
DONE install CPAN-Meta-Check-0.014
DONE install Module-CPANfile-1.1002
DONE install Win32-ShellQuote-0.003001
DONE install String-ShellQuote-1.04
DONE install ExtUtils-MakeMaker-CPANfile-0.07
DONE install Tie-Handle-Offset-0.003
DONE install Search-Dict-1.07
DONE install Parse-PMFile-0.41
DONE install Module-Load-Conditional-0.68
DONE install File-Fetch-0.52
DONE install Parse-LocalDistribution-0.19
DONE install CPAN-Mirror-Tiny-0.12
DONE install CPAN-Common-Index-0.009-TRIAL
DONE install CPAN-Common-Index-0.007
DONE install Menlo-1.9005

idea: with-develop

> cat cpanfile
on develop => sub {
  requires 'Path::Tiny';
};

> cpm install --with-develop
DONE install Path-Tiny-0.098

Milestone 0.200

• ? Support git, dist, ref syntax in cpanfile
• Support resolving distfile path from 02packages in remote/local
• ? --mirror-only (what does this mean exactly? i don't know)
• --sudo
• --with-develop
• ? --with-*
• ? --dev
• ? resolving "where modules are" from metacpan
• save "where modules are" in install.json, and hack Carton or Carmel.
• and...

Need a new name

Drop "Acme", and introduce a new name!

idea: download distribution tarball

It might be useful if cpm downloads distribution tarball for you.

# a new command or an option?
> cpm download App::cpm
Downloding http://www.cpan.org/authors/id/S/SK/SKAJI/App-cpm-0.214.tar.gz ... DONE

> ls
App-cpm-0.214.tar.gz

idea: opt in x_static_install

as in Menlo.

idea: save git revision of dists in install.json

Let's say you have the following cpanfile:

requires 'Plack';
requires 'MyApp', git => 'git://github.com/you/MyApp.git', ref => 'develop';

cpm already supports such cpanfile.

How about:

• cpm saves git revison of dists in install.json
• cpm (or some tool) creates cpanfile.snapshot with git revision
• and the cpanfile.snapshot will allow you to install the same version and revision of distributions at any time.

Is this useful? Is there any difficulties to achieve this?

cpm does not install DB_File

If DB_File is specified in a cpanfile, then it seems that cpm does not install this module.

A sample build log with this problem can be found here: https://api.travis-ci.org/jobs/171750983/log.txt?deansi=true

If DB_File is specified as a cpm command line argument, then installation works.

A quick look into the cpm code seems to suggest that core modules are marked as "satisfied". DB_File is a core module, but it is an optional one, so depending on perl's configuration it may be there or not (more specifically, if something like libdb*-dev is installed during perl's configuration, then DB_File will be built, otherwise not).

I guess that other optional core modules like GDBM_File have the same problem.

Doesn't handle local::lib

This is similar to #12

It would be nice if cpm would honour the same logic for installing modules to local::lib set locations as cpanm has

Pls, add possibility to read modules list from STDIN

This is required for using cpm with cpan-outdated or similar tools.

Typical usage:

cpan-outdated | cpanm

Sould suport vx.x.x version format in cpanfile

Hi.

Currently, if version is specified in version string, cpm throw error.

eg:

requires 'Geo::IP', v1.45.0;
requires 'GeoIP2',  v2.3.1;

idea: support distroprefs

I'm thinking of adding hook points to cpm somehow.

Some use-cases might be:

• inject dependencies
• patch distributions
• ...?

charsbar pointed out Distroprefs to me.
Actually I haven't used distroprefs. Are distroprefs useful?

Fails if ~/.perl-cpm does not exist (App-cpm-0.290)

cpm fails if ~/.perl-cpm does not exist. Sample failed run can be seen on https://travis-ci.org/eserte/sandbox/jobs/182802980

These are the interesting lines from the log:

$ if ! cpm install -g -v; then cat ~/.perl-cpm/build.log; false; fi

/home/travis/.perl-cpm/build.log.1481356160: No such file or directory at /home/travis/perl5/perlbrew/perls/5.10/lib/site_perl/5.10.1/App/cpm/Logger/File.pm line 9.

cat: /home/travis/.perl-cpm/build.log: No such file or directory

My workaround: do a mkdir ~/.perl-cpm myself, as done in https://travis-ci.org/eserte/sandbox/jobs/182803524

s/-/::/ before attempting a module lookup

cpanm does this and it's very handy, because I can just copy paste a dist name without having to manually do the translation.

eg:

MaxMind-MacBook:~ olaf$ cpm install -g Mac-PopClip-Quick
FAIL resolve Mac-PopClip-Quick
0 distribution installed.
See /Users/olaf/.perl-cpm/build.log for details.
MaxMind-MacBook:~ olaf$ cpm install -g Mac::PopClip::Quick
DONE install Archive-Zip-1.59
DONE install Mac-PopClip-Quick-1.000001
2 distribution installed.

It would have been nice to see that happen on the first attempt. I can't think of a reason not to do this as it shouldn't affect performance and there's no legitimate reason for a - to appear in a module name.

I can submit a pull request if you like.

wishlist: Support -v for --version too like in cpanm

Although --version is -V and --verbose is -v, since -v is popular to request version, cpanm also supports this:

% cpanm -v

to display version.

MSWin32 MSYS2: cpm logging causes configure process failure

When running cpm install on MSYS2's MINGW64 subsystem, I get the following error:

The process cannot access the file because it is being used by another process.

I have tracked it down to the line: App::cpm::Worker::Installer::Menlo/run_command

 $cmd .= " >> " . Menlo::Util::shell_quote($self->{log}) . " 2>&1";

Since the workers are trying to write to a file that is already open, the process gets killed by Windows.

Possibility to run tests

cpm doesn't run tests by default, and it seems like it's not possible to say it to do so. I think it can be useful to provide an option for it to run tests - and make it possible to specify this option in a configuration file / environment variable to make it a default.

cpanfile recursive resolving

Hi Shoichi,

I discovered cpm and very happy about it. Thank you a lot for this useful utility!

Unfortunatelly I found out that cpm can't resolve cpanfile recursively.

For instance, in cpanfile of module X there is dependence from module Y and in cpanfile of module Y there is dependence from module Z. In that case command cpm X will install module X, but won't install modules Y and Z.

If you download module X and run the command cpm in it then module Y will be installed, but module Z won't be installed anyway.

Are you planning to add the ability to recursively resolve cpanfile? That would be great!

backpan support

cpanm defaults to trying to fetch dists from backpan.perl.org when searching both cpanmetadb and metacpan. This is needed whenever cpanfiles pin old versions of modules that have since been cleaned from cpan.

Just out of curiosity, is there a reason cpm implemented its own cpanmetadb resolver, instead of using one of the Menlo::Index::* modules?

Does not support all of cpanm's options

@skaji, first of all, thanks very much for your efforts. :) I'm looking into implement this at $work, but I need to be able to pass the following flags:

--mirror-only
--mirror-index
--with-develop
--sudo

If you're open to adding support for these, I'd be happy to send a pull request.

have to add workaround for Module::Build

Not a few cpan modules miss Module::Build reqirement.
See https://gist.github.com/key-amb/3e37c32ee861eec00493

Must add some workaround for it.

Support arbitrary location for cpanfile

carton allows me to carton install --cpanfile path/to/cpanfile, which is handy if I want to pull in dependencies from multiple projects. Is this possible with cpm? Based on the documentation, it doesn't appear to be.

Name of package: App::cpm::Resolver::O2Packages

I'm just wondering why you didn't pick ::02Packages (subpackage starts with number), it's allowed.

Failed to resolve $MODULE

I have a failed cpm run which just says

2016-12-11T19:18:58,14236,IO::File| Failed to resolve IO::File

without any explanation why resolving failed. The full log (created by using #51 (comment)) may be seen here: https://travis-ci.org/eserte/bbbike/jobs/183061942#L5910

It would be good if more information would be printed here. Currently it's unknown to me if there is a problem in cpm itself or an external problem. For example if a web service was queried here, and failed, then either the service error message, or the http status message, or connection error message (whichever failed) should be printed.

idea: warn if dist specifies x_deprecated

see http://neilb.org/2015/01/17/deprecated-metadata.html