doherty/dist-zilla-pluginbundle-testingmania

test your dist with every testing plugin conceivable

dist-zilla-pluginbundle-testingmania's Issues

::Test::EOL

I just pushed a fork of EOLTests

https://metacpan.org/module/XENO/Dist-Zilla-Plugin-Test-EOL-0.03/lib/Dist/Zilla/Plugin/Test/EOL.pm

it's main feature is that it's an author test and not a release test Please consider replacing EOLTests with Test::EOL

PodSpelling tests

I think you can look at readding this module if you want, because the module that hadn't been updated in ten years has started getting updates again, so I believe this is no longer so volatile. please remember to allow the stopwords parameter though. I at least have to use them to allow my own name at this time

Test starts to break with RJBS/Dist-Zilla-6.026-TRIAL.tar.gz

@rjbs , please take note. Sample fail report: http://www.cpantesters.org/cpan/report/50240d72-ece5-11ec-83ba-0ec1c02a90c2

No default for Dist::Zilla::Plugin::ChangesTests

The addition of DZP::ChangesTests requires a changelog attribute in dist.ini even if my changelog is named the default "Changes".

Here's the error trace:

Attribute (changelog) does not pass the type constraint because: Validation failed for 'Str' with value undef at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/Role/Plugin.pm line 50
        Dist::Zilla::Role::Plugin::plugin_from_config('Dist::Zilla::Plugin::ChangesTests', '@TestingMania/ChangesTests', 'HASH(0x376b1f8)', 'Dist::Zilla::MVP::Section=HASH(0x3756590)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/Role/Plugin.pm line 61
        Dist::Zilla::Role::Plugin::register_component('Dist::Zilla::Plugin::ChangesTests', '@TestingMania/ChangesTests', 'HASH(0x376b1f8)', 'Dist::Zilla::MVP::Section=HASH(0x3756590)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/MVP/Section.pm line 46
        Class::MOP::Class:::after('Dist::Zilla::MVP::Section=HASH(0x3756590)') called at /usr/lib/perl5/site_perl/5.10/i686-cygwin/Class/MOP/Method/Wrapped.pm line 61
        Class::MOP::Method::Wrapped::__ANON__('Dist::Zilla::MVP::Section=HASH(0x3756590)') called at /usr/lib/perl5/site_perl/5.10/i686-cygwin/Class/MOP/Method/Wrapped.pm line 89
        Dist::Zilla::MVP::Section::finalize('Dist::Zilla::MVP::Section=HASH(0x3756590)') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Assembler/WithBundles.pm line 82
        Config::MVP::Assembler::WithBundles::_add_bundle_contents('Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)', 'bundle_config', 'HASH(0x3497d40)') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Assembler/WithBundles.pm line 36
        Config::MVP::Assembler::WithBundles::replace_bundle_with_contents('Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)', 'Dist::Zilla::MVP::Section=HASH(0x36422c0)', 'bundle_config') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Assembler/WithBundles.pm line 26
        Class::MOP::Class:::after('Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)') called at /usr/lib/perl5/site_perl/5.10/i686-cygwin/Class/MOP/Method/Wrapped.pm line 61
        Class::MOP::Method::Wrapped::__ANON__('Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)') called at /usr/lib/perl5/site_perl/5.10/i686-cygwin/Class/MOP/Method/Wrapped.pm line 89
        Dist::Zilla::MVP::Assembler::end_section('Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Assembler.pm line 87
        Config::MVP::Assembler::change_section('Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)', 'PerlTidy', 'PerlTidy') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Reader/INI.pm line 49
        Config::MVP::Reader::INI::INIReader::change_section('Config::MVP::Reader::INI::INIReader=HASH(0x29a1350)', 'PerlTidy') called at /usr/lib/perl5/site_perl/5.10/Config/INI/Reader.pm line 34
        Config::INI::Reader::read_handle('Config::MVP::Reader::INI::INIReader=HASH(0x29a1350)', 'IO::File=GLOB(0x29c2768)') called at /usr/lib/perl5/site_perl/5.10/Mixin/Linewise/Readers.pm line 85
        Mixin::Linewise::Readers::__ANON__() called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Reader/INI.pm line 21
        Config::MVP::Reader::INI::read_into_assembler('Config::MVP::Reader::INI=HASH(0x298f4f8)', './dist.ini', 'Dist::Zilla::MVP::Assembler::Zilla=HASH(0x293c7d0)') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Reader.pm line 19
        Config::MVP::Reader::read_config('Config::MVP::Reader::INI=HASH(0x298f4f8)', './dist.ini', 'HASH(0x29a1110)') called at /usr/lib/perl5/site_perl/5.10/Config/MVP/Reader/Finder.pm line 87
        Config::MVP::Reader::Finder::read_config('Dist::Zilla::MVP::Reader::Finder', 'Path::Class::File=HASH(0x29a10f8)', 'HASH(0x29a1110)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/Dist/Builder.pm line 187
        Dist::Zilla::Dist::Builder::__ANON__() called at /usr/lib/perl5/site_perl/5.10/Try/Tiny.pm line 76
        eval {...} called at /usr/lib/perl5/site_perl/5.10/Try/Tiny.pm line 67
        Try::Tiny::try('CODE(0x281c4c8)', 'Try::Tiny::Catch=REF(0x29a11e8)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/Dist/Builder.pm line 211
        Dist::Zilla::Dist::Builder::_load_config('Dist::Zilla::Dist::Builder', 'HASH(0x281baa8)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/Dist/Builder.pm line 27
        Dist::Zilla::Dist::Builder::from_config('Dist::Zilla::Dist::Builder', 'HASH(0x2932690)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/App.pm line 111
        Dist::Zilla::App::zilla('Dist::Zilla::App=HASH(0x21f3378)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/App/Command.pm line 13
        Dist::Zilla::App::Command::zilla('Dist::Zilla::App::Command::perltidy=HASH(0x222a218)') called at /usr/lib/perl5/site_perl/5.10/Dist/Zilla/App/Command/perltidy.pm line 23
        Dist::Zilla::App::Command::perltidy::execute('Dist::Zilla::App::Command::perltidy=HASH(0x222a218)', 'Getopt::Long::Descriptive::Opts::__OPT__::2=HASH(0x22a93c8)', 'ARRAY(0x1421f28)') called at /usr/lib/perl5/site_perl/5.10/App/Cmd.pm line 231
        App::Cmd::execute_command('Dist::Zilla::App=HASH(0x21f3378)', 'Dist::Zilla::App::Command::perltidy=HASH(0x222a218)', 'Getopt::Long::Descriptive::Opts::__OPT__::2=HASH(0x22a93c8)') called at /usr/lib/perl5/site_perl/5.10/App/Cmd.pm line 170
        App::Cmd::run('Dist::Zilla::App') called at /usr/local/bin/dzil line 11

critic config

Can you set Critic config with TestingMania? or should I disable it and include it on the side. (either way could you document it in the pod)

replace NoTabsTest with Test::NoTabs

Dist::Zilla logs:

!!! [NoTabsTests] is deprecated and may be removed in a future release; replace it with [Test::NoTabs]

Thanks!

VersionTests

So I finally release a new Version of Test::Version if you wanted to add that to your list of tests, it can replace HasVersion if you want.

Dist::Zilla::Plugin::VersionTests

Be careful about using LatestPrereqs

You may want to be more careful about using LatestPrereqs...
@Testingmania uses CompileTests to build itself which requires File::Find.
File::Find is a core module, so when LatestPrereqs checks for the latest version it finds 1.19
which is only in the latest dev release of perl (5.13.10).
Thankfully my cpan was smart enough to not make me install a dev release of perl.

Various test failures

cpanm (App::cpanminus) 1.7001 on perl 5.018002 built for x86_64-linux
Work directory is /home/ron/.cpanm/work/1414280382.27757
You have make /usr/bin/make
You have LWP 6.08
You have /bin/tar: tar (GNU tar) 1.26
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
You have /usr/bin/unzip
Searching Dist::Zilla::PluginBundle::TestingMania on cpanmetadb ...
--> Working on Dist::Zilla::PluginBundle::TestingMania
Fetching http://www.cpan.org/authors/id/D/DO/DOHERTY/Dist-Zilla-PluginBundle-TestingMania-0.23.tar.gz
-> OK
Unpacking Dist-Zilla-PluginBundle-TestingMania-0.23.tar.gz
Entering Dist-Zilla-PluginBundle-TestingMania-0.23
Checking configure dependencies from META.json
Checking if you have Module::Build 0.28 ... Yes (0.4210)
Checking if you have Module::Build 0.36 ... Yes (0.4210)
Configuring Dist-Zilla-PluginBundle-TestingMania-0.23
Running Build.PL
Created MYMETA.yml and MYMETA.json
Creating new 'Build' script for 'Dist-Zilla-PluginBundle-TestingMania' version '0.23'
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have Test::DZil 0 ... Yes (5.021)
Checking if you have Dist::Zilla::Plugin::Test::MinimumVersion 2.000001 ... Yes (2.000005)
Checking if you have Dist::Zilla::Plugin::Test::Pod::LinkCheck 1.001 ... Yes (1.001)
Checking if you have Dist::Zilla::Plugin::MetaTests 0 ... Yes (5.021)
Checking if you have Path::Tiny 0 ... Yes (0.056)
Checking if you have Moose::Autobox 0 ... Yes (0.15)
Checking if you have IPC::Open3 0 ... Yes (1.13)
Checking if you have namespace::autoclean 0 ... Yes (0.19)
Checking if you have Test::Version 0.07 ... Yes (1.002004)
Checking if you have Test::CPAN::Meta 0 ... Yes (0.23)
Checking if you have Dist::Zilla::Plugin::PodCoverageTests 0 ... Yes (5.021)
Checking if you have Test::Perl::Critic 0 ... Yes (1.02)
Checking if you have Dist::Zilla::Plugin::Test::Synopsis 2.000001 ... Yes (2.000005)
Checking if you have strict 0 ... Yes (1.07)
Checking if you have Dist::Zilla::Plugin::PodSyntaxTests 0 ... Yes (5.021)
Checking if you have Module::Build 0.28 ... Yes (0.4210)
Checking if you have IO::Handle 0 ... Yes (1.34)
Checking if you have Dist::Zilla::Plugin::Test::NoTabs 0.02 ... Yes (0.09)
Checking if you have warnings 0 ... Yes (1.18)
Checking if you have Test::More 0.88 ... Yes (1.001006)
Checking if you have Dist::Zilla::Plugin::Test::UnusedVars 2.000001 ... Yes (2.000006)
Checking if you have Dist::Zilla::Plugin::Test::Compile 2.033 ... Yes (2.051)
Checking if you have List::MoreUtils 0 ... Yes (0.33)
Checking if you have Dist::Zilla::Plugin::Test::Version 0.001002 ... Yes (0.002004)
Checking if you have Dist::Zilla::Plugin::Test::CPAN::Meta::JSON 0.003 ... Yes (0.003)
Checking if you have autodie 0 ... Yes (2.13)
Checking if you have Dist::Zilla::Plugin::Test::Perl::Critic 2.112410 ... Yes (2.112410)
Checking if you have Moose 0 ... Yes (2.1211)
Checking if you have Dist::Zilla::Plugin::MojibakeTests 0.2 ... Yes (0.7)
Checking if you have JSON::PP 2.27103 ... Yes (2.27203)
Checking if you have Dist::Zilla::Plugin::Test::Kwalitee 2.020000 ... Yes (2.11)
Checking if you have Dist::Zilla::Plugin::Test::CPAN::Changes 0.005 ... Yes (0.008)
Checking if you have Dist::Zilla::Plugin::Test::DistManifest 2.000001 ... Yes (2.000004)
Checking if you have Dist::Zilla::Plugin::Test::EOL 0 ... Yes (0.14)
Checking if you have Pod::Coverage::TrustPod 0 ... Yes (0.100003)
Checking if you have File::Spec 0 ... Yes (3.47)
Checking if you have Dist::Zilla::Plugin::Test::Portability 2.000001 ... Yes (2.000005)
Checking if you have Dist::Zilla::Role::PluginBundle::Easy 0 ... Yes (5.021)
Building and testing Dist-Zilla-PluginBundle-TestingMania-0.23
Building Dist-Zilla-PluginBundle-TestingMania
t/00-compile.t ........ ok

#   Failed test 'xtests are all there'
#   at t/01-test-manifest.t line 39.
#     Structures begin differing at:
#          $got->[4] = 'eol.t'
#     $expected->[4] = 'kwalitee.t'
# {
#   'have' => [
#     'mojibake.t',
#     'eol.t',
#     'unused-vars.t',
#     'distmeta.t',
#     'portability.t',
#     'no-tabs.t',
#     'minimum-version.t',
#     'kwalitee.t',
#     'pod-coverage.t',
#     'critic.t',
#     'pod-linkcheck.t',
#     'meta-json.t',
#     'dist-manifest.t',
#     'pod-syntax.t',
#     'synopsis.t',
#     'test-version.t',
#     'cpan-changes.t'
#   ],
#   'want' => [
#     'critic.t',
#     'test-eol.t',
#     'kwalitee.t',
#     'unused-vars.t',
#     'minimum-version.t',
#     'dist-manifest.t',
#     'portability.t',
#     'pod-coverage.t',
#     'test-version.t',
#     'cpan-changes.t',
#     'synopsis.t',
#     'no-tabs.t',
#     'pod-linkcheck.t',
#     'pod-syntax.t',
#     'distmeta.t',
#     'meta-json.t',
#     'mojibake.t'
#   ]
# }
# Looks like you failed 1 test of 2.

Failed test 'default'

at t/01-test-manifest.t line 41.

#   Failed test 'EOLTests enbled'
#   at t/01-test-manifest.t line 131.

#   Failed test at t/01-test-manifest.t line 133.
#     Structures begin differing at:
#          $got->[5] = 'eol.t'
#     $expected->[5] = 'kwalitee.t'
# Looks like you failed 2 tests of 2.

Failed test 'nonexistent test'

at t/01-test-manifest.t line 140.

Looks like you failed 2 tests of 5.

t/01-test-manifest.t ..
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/5 subtests
(less 1 skipped subtest: 2 okay)

Test Summary Report

t/01-test-manifest.t (Wstat: 512 Tests: 5 Failed: 2)
Failed tests: 1, 5
Non-zero exit status: 2
Files=2, Tests=6, 1 wallclock secs ( 0.02 usr 0.01 sys + 1.18 cusr 0.08 csys = 1.29 CPU)
Result: FAIL
Failed 1/2 test programs. 2/6 subtests failed.
-> FAIL Installing Dist::Zilla::PluginBundle::TestingMania failed. See /home/ron/.cpanm/work/1414280382.27757/build.log for details. Retry with --force to force install it.

using all the plugins makes it impossible to bootstrap with plugin

unable to bootstrap because obviously the VERSION hasn't been set yet

[Bootstrap::lib] lib added to @INC at /home/xenoterracide/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Module/Runtime.pm line 205
Dist::Zilla::Plugin::Test::Version does not define $Dist::Zilla::Plugin::Test::Version::VERSION--version check failed at /home/xenoterracide/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Dist/Zilla/PluginBundle/TestingMania.pm line 23, <GEN1> line 31.
BEGIN failed--compilation aborted at /home/xenoterracide/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Dist/Zilla/PluginBundle/TestingMania.pm line 23, <GEN1> line 31.
Compilation failed in require at /home/xenoterracide/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Module/Runtime.pm line 205, <GEN1> line 31.
 at constructor Dist::Zilla::MVP::Section::new (defined at /home/xenoterracide/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Dist/Zilla/MVP/Section.pm line 53) line 84

Dist::Zilla::Plugin::Test::Pod::LinkCheck not mentioned in pod

this is included, but not mentioned in the documentation

Add some tests

Issue #4 wouldn't have happened if there were tests.

Test::UnusedVars removal

please consider removing this, as Test::Vars only works properly between 5.10 and 5.14.x . I am attempting to debug a 5.8.8 bug in Test::PodSpelling and this is on the list of reasons that I can't get dzp working with my normal dist.ini

Bump Test::Compile prereq

<chorny> Request for all CPAN authors that use Dist::Zilla::Plugin::Test::Compile. Please update it to latest version - older versions have problems on Windows. Requiring latest version in bundles would also be useful.
<ether> ++
<dipsy> Thanks!
<ether> chorny: and thank you so much again for your help in debugging it
* ether sends the broken versions to backpan
<ether> (not that helps anyone who's already got it installed, of course..)
<leont> Which versions exactly are buggy? Is that known?
<ether> it is known
<ether> the last 2 versions are good; most version in the 2.02x range are suspect
<ether> there were no major issues on other platforms though, so only win32 users are affected (which is a big or a small deal depending on your dist)
<ether> the matrix results are a good benchmark - if there's a sea of red, uh oh! :)  e.g.  http://matrix.cpantesters.org/?dist=Dist-Zilla-Plugin-Test-Compile%202.025
<dipsy> [ CPAN Testers Matrix: Dist-Zilla-Plugin-Test-Compile 2.025 ] 
* ether checks to see if cpan-grep is back up -- I added a header to t/00-compile.t last month that identifies the plugin version, so that can be used to track down dists that need re-releasing
<ether> (and I'll go through that list tonight)

Load tests for META.{yml,json} only when they exist

We should check if MetaYAML/MetaJSON plugin is loaded or if the file is listed in the dist, and load the corresponding test only if appropriate.