spox/allgems

Document everything

allgems's Issues

documentation regeneration

Need to include a way to mark documentation as needing to be regenerated. I think the easiest method would be to include a button to click and a mark made to let the daemon know that there seems to be something wrong with the given documentation. If a check reveals that the gem directory, or the doc directory within is missing, it will automatically repull and generate. If they are there, a notification should be sent out to the admin. We want to reduce the number of times we have to pull in and regenerate documentation in case we get someone that just likes to push buttons.

class/method search

Once documentation is built, grab and index all classes and methods available within the gem. Add in search support class:ClassName and method:MethodName to return any matching gems.

Database target

I was really hoping that sqlite would be okay for this project, and I guess at the beginning, it was. It really isn't anymore as we have multiple threads and processes vying for attention, and being rejected far too often. I think I'm going to fix the migration file back up to pure sequel and default out to using postgres, which means more configuration and such. However, this shouldn't be a bad thing as we will have better availability.

Dependency slow down

When generating a LID, we hit a big slowdown as the dependencies are calculated. We need to either see if we can find a way to speed it up doing our own calculations, or we need to throw the user to a landing page to wait until the LID is generated, and then redirected on.

related gems

allow the user to select a gem, and only gem documentation shown for the given gem, and any dependencies that gem has. Basically just results in setting a filter on the db query.

Link LID to gem

If a LID is generated from a gem dependency list, save the version info for the gem it was built from so we don't regenerate a new LID if one already exists.

splib prob?

E:/dev/ruby/allgems/lib/allgems/GemWorker.rb:103:in block (2 levels) in generate_documentation': private methodexec' called for Splib:Module (NoMethodError)
from E:/installs/ruby191p376/lib/ruby/gems/1.9.1/gems/actionpool-0.2.3/lib/actionpool/Thread.rb:171:in call' from E:/installs/ruby191p376/lib/ruby/gems/1.9.1/gems/actionpool-0.2.3/lib/actionpool/Thread.rb:171:inblock in run'
from E:/installs/ruby191p376/lib/ruby/1.9.1/timeout.rb:52:in timeout' from E:/installs/ruby191p376/lib/ruby/gems/1.9.1/gems/actionpool-0.2.3/lib/actionpool/Thread.rb:170:inrun'
from E:/installs/ruby191p376/lib/ruby/gems/1.9.1/gems/actionpool-0.2.3/lib/actionpool/Thread.rb:137:in start_thread' from E:/installs/ruby191p376/lib/ruby/gems/1.9.1/gems/actionpool-0.2.3/lib/actionpool/Thread.rb:35:inblock in start'

include proper rdoc

since we are requiring hanna, we might as well require the proper version of rdoc in the gemspec so it is available

undefined method error

ERROR: While executing gem ... (NoMethodError)
undefined method `include?' for nil:NilClass

A few of these seem to pop up while gemworker is working. Need to add in some extra logging and try to find the source.

User sync

The idea was floated on the ML about having a sync ability to display only docs a user has installed. Seems pretty doable, though the user would be responsible for performing an 'allgems -sync' command. Need some feedback regarding the actual interest in this feature and the fact that user information is being uploaded and collected. Since it is something that is done explicitly by the user, shouldn't be too big of a concern.

Possible implementations after a sync:

• have a URL returned that will set a cookie for the user (preference at this point)
• generate a user key to login with
• something else

However the implementation is done, it should be kept as simple as possible. An expiry should be set on how long data is stored so we don't end up with a database full of stale, unused data.

specer can die

/home/rdp/installs/ruby-1.9.1-p243_gc/lib/ruby/gems/1.9.1/gems/rubygems-update-1.3.5/lib/rubygems/specification.rb:296:in `>': comparison of String with 3 failed (ArgumentError)
        from /home/rdp/installs/ruby-1.9.1-p243_gc/lib/ruby/gems/1.9.1/gems/rubygems-update-1.3.5/lib/rubygems/specification.rb:296:in`_load'
        from /home/rdp/dev/allgems/lib/allgems/Specer.rb:43:in `load'
        from /home/rdp/dev/allgems/lib/allgems/Specer.rb:43:in`do_spec_load'
        from /home/rdp/dev/allgems/lib/allgems/Specer.rb:29:in `block in load_specs'
        from /home/rdp/dev/allgems/lib/allgems/Specer.rb:28:in`each'
        from /home/rdp/dev/allgems/lib/allgems/Specer.rb:28:in `load_specs'
        from /home/rdp/dev/allgems/lib/allgems/Runner.rb:37:in`sync'
        from /home/rdp/dev/allgems/lib/allgems/Runner.rb:29:in `do_sync'
        from ../bin/allgems:301:in`'

Link runners option to number of processes

We need to make sure that we have a solid link between runners and rdoc processes running as well as making sure our default number is pretty low. Probably something like 5 or 10.

sdoc indexing

Currently, class and method indexing is only available for documentation built with rdoc or hanna. Sdoc does not provide an easy to parse file to pull the information out.

wizard

make configuration wizard to spit out needed apache configuration.

Load gemspecs, versions info on start

Creating LIDs makes a problem when we are still fetching and unpacking gems. The first thing we should do is cycle through our index and creating gem/version info in the database as well as saving the serialized gemspecs. At this point, users can access the /gem/GEMNAME page, but no documentation will be available.

Use database to tag documentation creation

We need an easy way to figure out who has documentation already generated. A docs and docs_versions table will help this out as the docs can hold the different types (sdoc, rdoc, hanna) and the docs_versions will hold the tagging information of what has documentation completed. If there is an error or it needs to be regenerated, that tag can just be removed.