el-get-package-exists-p not tolerant to previous failed installation

[dimitri]

el-get-package-exists-p not tolerant to previous failed installation

[dimitri]

Specific checks is what I began implementing with el-get-dpkg-package-status, but I'm not sure if that's the way to go. How to recognize a broken git clone?

There's also the problem than double-install ain't a bug with some methods, where update is in fact an install.

Will think about it.

[Niluge-KiWi]

I don't know how to check if a git clone is broken or not...

That's where the idea of a file flag created when the installation process returned without error comes, and it seems much easier to do. Because at install time we know if it went well or not, without any hack.

[dimitri]

True, but the idea is then do we need to forbid installing over an already installed package? Or do we need a way to say that "installing over" needs a remove first?

[Niluge-KiWi]

Installing over an already installed package is "reinstalling", and we could prompt the user if it's really what it wants, if he just asked for an install and not for a reinstall.

The functions to provide are like all packages managers: install, remove, update, and reinstall.

If at install time there is already a directory for the package, but no flag "install is ok", then we should just remove the half installed package, maybe prompt user, and finally install our package.

I understand that prompting the user can be an issue, since the goal is to call el-get non-interactively, from .emacs. But here again, like other packages manager, we could cancel new installs if they request a user action, notify the user, and just load already installed packages, to avoid errors.

[dimitri]

Well I could add a reinstall function for each and every current supported source type, by default it would be remove+install. Adding an interactive question for reinstalling when "installing over" is doable also.

But depending on the type of the package I don't have a directory where to create a dot file, all I have could just be a symlink (apt-get, fink, elpa). What about a el-get-dir/status file which contains a list of properly installed packages? Either text/plain or elisp so that I can `read' it.

[Niluge-KiWi]

Indeed this is an issue with apt-get where no file is installed inside .emacs.d/el-get/. An elisp file status seems to be the right solution.

For the reinstall function, it seems also ok, but there is still the case when you think you do a simple install, because the old package has not been fully installed.

[dimitri]

Good, we have a design now. Will work on it when I have some free time. Install over half-installed is handled by checking as you proposed: the package dir is there but the package is not in the status file.

[Niluge-KiWi]

Ok. This works for types like https and git, but what to do with apt-get and the others? Can they be in a "half-installed" status?
For apt-get, it is possible: an issue with post install scripts leaves the package half-installed.

However it may not be an issue, the goal of detecting half-installed packages is to cleanup the place for the install to work correctly. So for http and the others it is needed and we know how to handle it. And for apt-get and the others, I'm not sure, but we may suppose the called installers can manage this...

[dimitri]

Half installed is detected because we only update the status when all the command did succeed. Install over half-installed will first remove the package calling its type remove function (which will e.g. apt-get remove package) etc.

Design seems ok now for me to try at implementing :)

[Niluge-KiWi]

To detect we have an half-installed package, we check on status file to know it is not fully installed, but we also need to know if it has already been installed, or at least tried to. For that we can check the directory for types http and others, but for apt-get I don't know.

That's the issue I tried to raise in my previous message.

[dimitri]

Let's add a "selected" status, ala dpkg?

[Niluge-KiWi]

I don't know well enough dpkg to know what really means a selected status in dpkg.

But for el-get, the meaning would be "has been selected for installation, but is not fully installed" (it could be an installation failure, but it could also be something else...).
And to install a selected package, we have to do a reinstall.

Maybe the name is not appropriate (but if it's the same meaning for dpkg, it's ok), but otherwise it seems ok to me.

[dimitri]

Please check the code as of tonight, there's initial support for what you ask. I didn't go as far as allowing el-get to actually remove the package before attempting to install again, but I could see an option for this behavior.

Read the last README section to init your current status before going further.

[cbbrowne]

I'm running into pretty severe problems that seem a result of this...

Debugger entered--Lisp error: (error "Package magit failed to install, remove it first.")

Magit isn't installed right now; I expressly removed it. But el-get-remove magit complains that its directory isn't there. So I've got "Catch-22".

[Niluge-KiWi]

I havn't tested the last commits yet, but a remove --force seems to be needed.
Or do we really need a remove --no-force: do we need a check on remove?

For a hypothetical reinstall function, --force will be required.

[dimitri]

Added a "removed" status to the packages, so that the magit situation should not happen any more --- forgot to think people would actually use the remove feature :)

On the api, C-u M-x el-get-install could run the remove itself I guess. The remove breaking was a bug, I don't think we need --force as I don't see which problem it would solve here.

[cbbrowne]

Aside from the immediate bug, there's a broader issue here that problems in installing anything lead to one's Emacs startup basically falling over.

As cool as it is, el-get seems to be a bit too vulnerable to externalities.

Let me suggest a further sort of feature that might make it WAY better..

I think the recipes should combine with "deployment plans," in such a way that things can get dropped out happily should el-get run into problems.

Thus...

(setq el-get-sources
(list
(:name scratch ;;; And let's suppose the recipe mostly comes from a recipe file
:after (lambda ()
(require 'scratch)))
(:name bbdb
:after (lambda ()
(require 'bbdb)
(require 'bbdb-hooks)
(add-hook 'gnus-startup-hook 'bbdb-insinuate-gnus)
(bbdb-initialize 'gnus 'message 'sc)
;;; and all the rest of my BBDB configuration!
))
))

Now, each of the package installations gets run inside the context of a condition-case, thereby allowing us to catch exceptions along the way. Let's suppose "BBDB" fails to install, for some reason. If that's so, then we don't bother trying to run the :after hook, and can continue to process subsequent packages, with hopefully minimal mucking up of the environment.

Question: Am I right in thinking that I probably ought to stow my BBDB configuration in the :after hook? That would be a nice way to "modularize" config even more.

[dimitri]

:after hook is made for you to put your setup there, that's what I'm doing now

The condition-case is a good idea, will look on it Sunday or next week. Thx!

[dimitri]

On the other hand that only applies to ELPA packages I think, all other install methods are async now, so that only the async process would error out, the other concurrent ones should continue, shouldn't they?

[cbbrowne]

Things are looking much smoother now, with the recent commits. It's still worth looking at the condition-case idea, but things are otherwise working reasonably well.

[dimitri]

Ok been looking.

All the install/update methods are now async ones except for ELPA, because that's pure elisp code running. HTTP support code is elisp, but I don't know about its implementation, except that it's async. It's used by emacswiki and http-tar.

It seems like the only package installing method that would 'kill' a multi-install from el-get would be ELPA.

Then there's el-get-init which can go wrong for any package and is synced and pure elisp of course. I'll protect only that for starters, then I'll close this ticket.

A new one will be in order for a larger condition-case usage in el-get! :)

[dimitri]

Commited something easy and effective, I think.