Giter Site home page Giter Site logo

antoniogamiz / perl6-documentable Goto Github PK

View Code? Open in Web Editor NEW
3.0 4.0 3.0 1.13 MB

All you need to generate your module documentation in HTML.

License: Artistic License 2.0

Perl 6 48.19% HTML 1.32% JavaScript 31.58% CSS 17.75% CoffeeScript 1.17%
perl6 perl6-module documentation pod pod6 html

perl6-documentable's Introduction

Perl6 Doc Tools CI Status Build Status artistic

In this repository you can find all logic responsible of generate the official documentation of Perl6 or any other repository that follows the same specification. Could be used for modules that include a substantial amount of documentation, for instance.

Table of contents

Installation

$ zef install Perl6::Documentable

If you have already downloaded this repo,

$ zef install .

Usage

Before generating any documents you should execute:

documentable setup

in order to download the necessary files needed to generate the site (CSS, svg, ...). Alternatively, you can add your own. See default templates to get more information.

Usage:
  bin/documentable setup -- Downloads default assets to generate the site
  bin/documentable start [--topdir=<Str>] [--conf=<Str>] [-v|--verbose] [-c|--cache] [-p|--pods] [-k|--kind] [-s|--search-index] [-i|--indexes] [-t|--type-images] [-f|--force] [--highlight] [--typegraph-file=<Str>] [--highlight-path=<Str>] [--dirs=<Str>] [-a|--all] -- Start the documentation generation with the specified options
  bin/documentable update [--topdir=<Str>] [--manage] [--conf=<Str>] -- Check which pod files have changed and regenerate its HTML files.
  bin/documentable clean -- Delete files created by "documentable setup"

    --topdir=<Str>            Directory where the pod collection is stored
    --conf=<Str>              Configuration file
    -v|--verbose              Prints progress information
    -c|--cache                Enables the use of a precompiled cache
    -p|--pods                 Generates the HTML files corresponding to sources
    -k|--kind                 Generates per kind files
    -s|--search-index         Generates the search index
    -i|--indexes              Generates the indexes files
    -t|--type-images          Write typegraph visualizations
    -f|--force                Force the regeneration of the typegraph visualizations
    --highlight               Highlights the code blocks
    --typegraph-file=<Str>    TypeGraph file
    --highlight-path=<Str>    Path to the highlighter files
    --dirs=<Str>              Dirs where documentation will be found. Relative to :$topdir
    -a|--all                  Equivalent to -t -p -k -i -s
    --manage                  Sort Language page

See the CLI documentation to learn more.

Default templates

As you may have noticed, every page in the documentation follows the same template. You can get those templates in the releases page. I strongly recommend you to use the defaults, which can be set up executing

documentable setup

That command will download the latest assets tarfile in your directory. But what does this tar contain? Several things:

  • Necessary files to enable highlighting (highlights dir).
  • html dir containing several icons, graphics, etc. In this directory will be written all HTML documents generated by Perl6::Documentable.
  • assets dir containing the style sheets used and JS code to enable search functionality.
  • template dir containing the default templates used.
  • Two files to run the server (app-start and app.pl).
  • A Makefile to configure highlighting (make init-highlights) and easily start a dev server with make run.

Docker container

There is a specific docker container with all necessary dependencies (including the highlighter) at https://github.com/antoniogamiz/docker-documentable.

You can see an example about how to use it here.

Installing dependencies

Rakudo

You need Perl 6 installed. You can install the Rakudo Perl 6 compiler by downloading the latest Rakudo Star release from rakudo.org/downloads/star/.

zef

Zef is a Perl 6 module installer. If you installed Rakudo Star package, it should already be there. Feel free to use any other module installer for the modules needed (see below).

Mojolicious / Web Server (OPTIONAL)

Mojolicious is written in Perl 5, so assuming that you use cpanm, install this now:

    $ cpanm -vn Mojolicious

GraphViz

To generate the svg files for the typegraph representation you need to have installed graphviz. If you use Ubuntu/Debian:

    $ sudo apt-get install graphviz

Highlight (OPTIONAL)

This is necessary to appli highlighting to the code examples in the documentation. You can skip it, but have in mind that all code examples will appear with the same color (black).

You can set this up with the default Makefile (obtained using documentable setup):

    make init-highlights

SASS Compiler

To build the styles, you need to have a SASS compiler. You can either install the sass command

    $ sudo apt-get install ruby-sass

wget and tar

This programs are used by documentable setup to download the default assets and extract them. If you are on Ubuntu/Debian you will not have any problem (probably). If you are using Windows I recommend you to download the assets yourself from this link.

FAQ

Question: Do I need to regenerate all pages when I have only changed one?

Answer: No, you can execute documentable update to only regenerate those pages affected by your changes.

Is not your question here? Then, please raise an issue.

AUTHORS

Antonio Gámiz Delgado <@antoniogamiz>

Moritz Lenz <@moritz>

Jonathan Worthington <@jnthn>

Richard Hainsworth<@finanalyst>

Will Coleda <@coke>

Aleks-Daniel <@AlexDaniel>

Sam S <@smls>

Juan Julián Merelo Guervós <@JJ>

MorayJ <@MorayJ>

Zoffix <@zoffixnet>

Tison <@TisonKun>

Samantha Mcvey <@samcv>

Altai-Man <@Altai-man>

Tom Browder <@tbrowder>

Alexander Moquin <@Mouq>

Wenzel P. P. Peppmeyer <@gfldex>

COPYRIGHT AND LICENSE

Copyright 2019 Perl6 Team

This library is free software; you can redistribute it and/or modify it under the Artistic License 2.0.

This module uses some conventions: (this started in August, 2019, so previous work does not follow the conventions):

We use a ChangeLog generator to generate the CHANGELOG.md file.

perl6-documentable's People

Contributors

alexdaniel avatar antoniogamiz avatar finanalyst avatar jj avatar moritz avatar noisegul avatar tbrowder avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

jj uzluisf noisegul

perl6-documentable's Issues

zef install Perl6::Documentable doesn't work

What are the prerequisite steps right now to install and use this?
Assuming a fresh installation of Rakudo 2019.03 and zef v0.7.1, zef can only find it
after performing zef upgrade (which makes sense).

However it is then masked by perl6/doc:

$ zef install Perl6::Documentable
All candidates are currently installed
No reason to proceed. Use --force-install to continue anyway
$ zef search Perl6::Documentable
===> Found 3 results
-------------------------------------------------------------------------------------------------------------------
ID|From                            |Package                       |Description                                     
-------------------------------------------------------------------------------------------------------------------
0 |Zef::Repository::LocalCache     |p6doc:ver<1.003>              |Perl 6 documentation (tools and docs)           
1 |Zef::Repository::Ecosystems<p6c>|p6doc:ver<1.003>              |Perl 6 documentation (tools and docs)           
2 |Zef::Repository::Ecosystems<p6c>|Perl6::Documentable:ver<1.0.0>|Represents a piece of Perl 6 that is documented.
-------------------------------------------------------------------------------------------------------------------

Trying to install it then from the release directly, or via zef install "Perl6::Documentable:ver<1.0.0>" fails due to tests:

$ zef install ./Perl6-Documentable-1.0.0/
===> Searching for missing dependencies: Pod::To::Cached, Pod::Utilities
===> Testing: Pod::To::Cached:ver<0.3.0>
# test pod extraction
# testing freeze
# Heavy duty test optional
===> Testing [OK] for Pod::To::Cached:ver<0.3.0>
===> Testing: Pod::Utilities:ver<0.0.1>
===> Testing [OK] for Pod::Utilities:ver<0.0.1>
===> Testing: Perl6::Documentable:ver<1.0.0>
Too few positionals passed; expected 2 arguments but got 1
  in method new-from-file at /usr/local/Cellar/rakudo-star/2019.03/share/perl6/site/sources/BE2CAF04F8CDB952CB4722F9862EDD347DD13CC0 (Perl6::TypeGraph) line 29
  in submethod BUILD at /Users/tester/p/perl6/Perl6-Documentable-1.0.0/lib/Perl6/Documentable/Registry.pm6 (Perl6::Documentable::Registry) line 69
  in block <unit> at t/301-registry.t line 9

Too few positionals passed; expected 2 arguments but got 1
  in method new-from-file at /usr/local/Cellar/rakudo-star/2019.03/share/perl6/site/sources/BE2CAF04F8CDB952CB4722F9862EDD347DD13CC0 (Perl6::TypeGraph) line 29
  in submethod BUILD at /Users/tester/p/perl6/Perl6-Documentable-1.0.0/lib/Perl6/Documentable/Registry.pm6 (Perl6::Documentable::Registry) line 69
  in block <unit> at t/302-process-pod-source.t line 9

Too few positionals passed; expected 2 arguments but got 1
  in method new-from-file at /usr/local/Cellar/rakudo-star/2019.03/share/perl6/site/sources/BE2CAF04F8CDB952CB4722F9862EDD347DD13CC0 (Perl6::TypeGraph) line 29
  in submethod BUILD at /Users/tester/p/perl6/Perl6-Documentable-1.0.0/lib/Perl6/Documentable/Registry.pm6 (Perl6::Documentable::Registry) line 69
  in block <unit> at t/303-process-pod-dir.t line 9

Too few positionals passed; expected 2 arguments but got 1
  in method new-from-file at /usr/local/Cellar/rakudo-star/2019.03/share/perl6/site/sources/BE2CAF04F8CDB952CB4722F9862EDD347DD13CC0 (Perl6::TypeGraph) line 29
  in submethod BUILD at /Users/tester/p/perl6/Perl6-Documentable-1.0.0/lib/Perl6/Documentable/Registry.pm6 (Perl6::Documentable::Registry) line 69
  in block <unit> at t/304-compose-type.t line 8

Too few positionals passed; expected 2 arguments but got 1
  in method new-from-file at /usr/local/Cellar/rakudo-star/2019.03/share/perl6/site/sources/BE2CAF04F8CDB952CB4722F9862EDD347DD13CC0 (Perl6::TypeGraph) line 29
  in submethod BUILD at /Users/tester/p/perl6/Perl6-Documentable-1.0.0/lib/Perl6/Documentable/Registry.pm6 (Perl6::Documentable::Registry) line 69
  in block <unit> at t/305-search.t line 8

[...]

===> Testing [FAIL]: Perl6::Documentable:ver<1.0.0>
Aborting due to test failure: Perl6::Documentable:ver<1.0.0> (use --force-test to override)

Refactor processing logic

I think it would be better to have an independent part to process each pod file and populate the registry. In addition, indexing rules are not clear so I will change the index detection to use a grammar.

Time information

We do not know the time it takes to complete each phase so I all add some time trackers to bin/documentable.

Language page sorting

If you generate the site you will realize that any kind of sort is done (unlike the official doc site), so that needs to be fixed.

Compose pod source in Type dir

Currently we only take the pod source, process it and convert it to HTML. We need to add to every type its inheritance attributes from the other roles and classes.

Independent HTML generation

In order to only regenerate the pods which change, we need to be able to generate each of the documentation from "an independent function".
That means, we need to be able to regenerate whatever page we want, with a function from Perl6::Documentable::To::HTML.

Document the current build process

In order to make a new build system, we need to know what to index, how to index it, and what things need to be done.

To achieve that, in Definitions.md I am documenting how doc is built now.

All path relative to RESOURCES

If you try to install the module and execute it you will realise that it will not find the necessary files so all of them should be moved to resources directory.

Release a first version

Release a first version with all the current logic before start making changes to indexing, ... .

Windows support

This issue should mainly be viewed as a hub for discussion and information on this topic to determine the causes of Windows incompatibility, whether Windows support is desired and feasible (at this moment) and if so, how it could be achieved :)
If I can offer assistance I'd like to do so.

Describe the bug
Perl6::Documentable can't be installed using zef due to failing tests on Pod::To::Cached.
If tests are passed with --force-test, running examples will fail.

To Reproduce
Installation error:

  • Download and install Rakudo Star 2019.03
  • Clone Perl6-Documentable
  • zef upgrade
  • zef update
  • zef nuke StoreDir
  • zef install ./Perl6-Documentable

Usage error:

  • zef install --force-test ./Perl6-Documentable
  • Run the following code (a doc folder and the testile.p6 are inside the same folder in this scenario):
use Perl6::Documentable;
use Perl6::Documentable::Processing;

sub MAIN() {
	my $registry = process-pod-collection(
		cache => False,
		verbose => True,
		topdir => "doc",
		dirs => ["Language", "Programs", "Type", "Native"]
	);
}

Error output
Attempting zef install:

> zef install ./Perl6-Documentable/
===> Searching for missing dependencies: Pod::To::Cached, Pod::Utilities, Pod::Utilities::Build, Perl6::TypeGraph:ver<0.0.3>
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
# Failed test 'doc cache created with sub-directories'
# at t\030-sub-directories.t line 25
# $!path has corrupt doc-cache
Cannot look up attributes in a Pod::To::Cached type object
  in method update-cache at C:\Users\tester\.zef\store\pod-cached.git\cd97eb28cc582f845d2acffd8e88b8f7c7eb5228\lib\Pod\To\Cached.pm6 (Pod::To::Cached) line 240
  in block <unit> at t\030-sub-directories.t line 26

# test pod extraction
$!path has corrupt doc-cache
  in submethod TWEAK at C:\Users\tester\.zef\store\pod-cached.git\cd97eb28cc582f845d2acffd8e88b8f7c7eb5228\lib\Pod\To\Cached.pm6 (Pod::To::Cached) line 146
  in block <unit> at t\040-pod-extraction.t line 16

# Heavy duty test optional
===> Testing: Pod::To::Cached:ver<0.3.0>
===> Testing [FAIL]: Pod::To::Cached:ver<0.3.0>
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
Aborting due to test failure: Pod::To::Cached:ver<0.3.0> (use --force-test to override

When trying out example after forcing installation using zef --force-test:

> perl6 testfile.p6 
   1/72: doc/Language/\101-basics.pod6            => language/\101-basics
Too few positionals passed; expected 2 arguments but got 1
  in method new-from-file at C:\rakudo\share\perl6\site\sources\BE2CAF04F8CDB952CB4722F9862EDD347DD13CC0 (Perl6::TypeGraph) line 29
  in sub process-pod-source at C:\rakudo\share\perl6\site\sources\A6AB78CFFF7C59022104F9CB6E211500EDBAC45F (Perl6::Documentable::Processing) line 47
  in sub process-pod-dir at C:\rakudo\share\perl6\site\sources\A6AB78CFFF7C59022104F9CB6E211500EDBAC45F (Perl6::Documentable::Processing) line 38
  in sub process-pod-collection at C:\rakudo\share\perl6\site\sources\A6AB78CFFF7C59022104F9CB6E211500EDBAC45F (Perl6::Documentable::Processing) line 20
  in sub MAIN at testfile.p6 line 5
  in block <unit> at testfile.p6 line 2

Desktop:

  • OS: Windows 10 x64
  • Version 10.0.18362
  • Rakudo Star 2019.03

Additional context:
The installation error seems to happen due to Pod::To::Cached, while the usage error appears
to be mainly caused by the way paths are handled.

Proper subkind of a routine

Most of subkinds and categories values are set following this. In the case of a routine, depends on how it's defined.

For instance, in Any.pm6 only exists this routine with the following signature:

    multi sub    reverse(*@list  --> Seq:D)
    multi method reverse(List:D: --> Seq:D)

As discussed here with @moritz, $.subkinds should get the value routine but it is set to sub.

The code handling this part is here:

  • This repo goto
  • In the official repo goto

So, what should be the correct subkind for routines?

Delete functions from Pod::Convenience in Utils.pm6

There are two functions used by this module (first-code-block.pod-lower-headings), defined in Pod::Convenience. That module is not published yet so that functions will be temporarily stored in Utils.pm6.

Refactor tests

Ok, I have been a little naughty with the tests and I have been adding them without any order o explanation. In addition, there are quite functions with 0 tests. I am going to change that and tests are going to follow a name convention:

  • From 0 to 99: basic tests, not related with the core functionality of the module.
  • From 100-199: Perl6::Utils related tests.
  • From 200 to 299: Perl6::Documentable related tests.
  • From 300 to 399: Perl6::Documentable::Registry related tests.
  • From 400 to 499: Perl6::Documentable::Registry::To::HTML related tests.

At the same time I make this test I hope to close #2, with a more useful and complete documentation.

Add indexing rules

You refer back to this repository from Raku/doc#2575 but, in fact, there's no document detailing how indexing actually works. For instance, there are index entries like this X<BEGIN|Phasers, BEGIN> and also like this X<return-rw|control flow, return-rw>, with categories being indistinctly before or after the comma. Also Raku/doc#2884, which generates incorrect URLs (or at least inconvenient).

Basic setup

A lot of HTML parts must be generated. I will make them in this order:

  • Full pods.
  • Indexing pages.
  • Disambiguation.

Move find-definitions here

A big part of htmilify is the find-definitions function. It handles and determines what is indexed and how. I will document that part and split a little the function to test it (currently is 174 lines long so it's hard to test).

URL handling

Well, you can see in Perl6::Utils that URL handling right now is a big mess. I have added it temporarily to work in the HTML generation.

I have to change that logic without altering the current URLs (that's compulsory), so I will leave aside for a while.

Move process-pod-dir and process-pod-source to this module

It's the only thing that we need to completely encapsulate all processing related things in this module. As @finanalyst pointed out here, I want Perl6::Documentable::Registry to be the main nexus between all pod files in the doc and its different representations.

This means that when you want to build the doc in a specific format, you only have to setup and initialize the registry and consult it.

By these reasons, these last two functions are going to me moved to this module:

Some improvements to the documentation of Perl6::Documentable.pm6

  • The synopsis could include some usage examples for the included functions; even for all the other modules in the distribution.
  • Authors should include all contributors as identified in the repository.
  • Perl6::Documentable could be used to document sets of classes or tutorials for any other distribution, so it might be more accurate to say "a single documentation unit", or something like that, without reference
    to Perl6 documentation specifically.
  • Maybe use different classes for "primary" and "secondary" documentables? The only difference would be the use of $.origin (or maybe others)
  • if there are no categories, subkinds is returned? So, are they the same?
  • enum should be used for kinds instead of strings.
  • Maybe use .gist instead of english-list? Or create a gist that calls english-list?
  • It's not clear how this documentable is created. My impression is that it's instantiated from somewhere else, when the logic thing would be that it should receive a Pod with embedded metadata or external metadata and create everything. Right now it looks like it needs everything to be precomputed to actually start.

HTML module refactoring

Right now there's only one file responsible of generating all kind of pages. This module receive the data that it must convert to html, using p2h. That function is applied to every generated file so I think all that functionality should be moved to a Perl6::Documentable::HTML::Wrapper.

As for the other functions, they should receive the $registry as argument, instead of the data, in order to make them more customizable.

In addition, generate-kind-files generate all files of a kind. If we want to only regenerate those HTML files that has been modified (see Raku/doc#2668), we need a more granular way to generate this files.

Change indexes to Hashes

Currently indexes functions returns simple Arrays objects. I have been using them to create the index-to-html functions and as you can see it is quite messy and not so comprehensible.

I think if Hashes are returned instead, its meaning will be more clear.

Refactor Utils

Utils is quite a mess. I have added there all kind of functions that I did not know where to put.

I'm going to create two modules:

  • Perl6::Utils::Filesystem => to get pod names, read dirs and that kind of things.
  • Perl6::Utils::Logger => a good logging system.

Add TypeGraph logic

In the processing of some pods, information from Typegraph module is necessary (in types, for instance, $category attribute is modified by that information).

Page order should follow the new page specification.

In general, we will try to keep metadata as close to the file as possible, so in the file specification document it states that page order (actually, grouping in classes) should be specified as file metadata.
Maybe transitionally a flag for using one or the other should be used, but the file order document right now is in a different format, so that is that one that should be used.

Clarification in the documentation

It's not clear what "generate the documentation" really means. If it's generating the static HTML files used for the doc site, it should be clarified.

Also, it would be interesting to know what setup actually does and what's the API for the documentation set up that way, if it can be used for p6doc, Pod::To::HTML and things like that.

Indexing logic

Currently indexes and its HTML representation are generated at the same time, so they need to be split up to let multiple representations.

Make subtest more readable

As @ugexe has suggested me. I should change the way subtests are written:

you probably should write subtests like this
subtest 'Foo bar' => { }
instead of subtest { }, 'Foo bar';

Warnings in some routines

Every time you generate the documentation this warnings appears:

The subkinds of routine map in List cannot be determined. Are you sure that routine is actually defined in List 's file?
The subkinds of routine EVAL in routines cannot be determined. Are you sure that routine is actually defined in routines 's file?

I should see why they are triggered.

Use mini-doc repository

In the t dir there are a bunch of random pod6 files used to test this module, but a mini-doc repo exists where there are enough pod6 files to cover the necessary cases.

Code highlight

Currently there is not code highlighting, so I will add the same process used at perl6/doc.

5to6-perlfunc

In htmlify.p6 this is done this function. I have been thinking and I have not found a good way to resolve this without change the way definitions are considered, so I will add the same workaround here and we will rethink this thing while designing the new system.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.