appimagecommunity/appimageupdate

AppImageUpdate lets you update AppImages in a decentral way using information embedded in the AppImage itself.

AppImageUpdate

AppImageUpdate lets you update AppImages in a decentral way using information embedded in the AppImage itself. No central repository is involved. This enables upstream application projects to release AppImages that can be updated easily. Since AppImageKit uses delta updates, the downloads are very small and efficient.

This is an implementation of AppImageUpdate in modern C++ (C++11, to be precise).

This is beta-level code. It works, and most features available work fine. However, in this state, there is not much real world experience with the application. Please report any issues on the bug tracker.

Try it out

• Download an AppImage that contains update information (not all AppImages do).
• Wait a couple of hours/days until they have a new continuous build (happens frequently).
• Download AppImageUpdate from here and make it executable.
• Run AppImageUpdate and select your "old" Subsurface AppImage.
• Only the parts that have changed since the original version are downloaded.

Notice how quick the update was. Combined with fully automated continuous or nightly builds, this should make software "fluid", as users can get the latest development versions very rapidly.

If you have the optional appimaged daemon installed, then it can use AppImageUpdate enable right-click updates in the launcher:

Components

• AppImageUpdate: GUI application to update AppImages with
• appimageupdatetool: Command line tool to update AppImages with (AppImageUpdate does the same)
• validate: Command line tool to validate the integrity of the signature built into an AppImage (AppImageUpdate has this built in)

Motivation

Use cases

Here are some concrete use cases for AppImageUpdate:

• "As a user, I want to get updates for my AppImages easily, without the need to add repositories to my system like EPEL or ppa. I want to be sure nothing is touched on my system apart from the one application I want to update. And I want to keep the old version until I know for sure that the new version works for me. Of course, I want to check with GPG signatures where the update is coming from.
• As an application developer, I want to push out nightly or continuous builds to testers easily without having to go though complicated repository setups for multiple distributions. Since I want to support as many distributions as possible, I am looking for something simple and distribution independent.
• As a server operator, I want to reduce my download traffic by having delta updates. So that users don't have to download the same Qt libs over and over again with each application release, even though they are bundled with the application.

The problem space

With AppImages allowing upstream software developers to publish applications directly to end users, we need a way to easily update these applications. While we could accomplish this by putting AppImages into traditional deb and rpm packages and setting up a repository for these, this would have several disadvantages:

• Setting up multiple repositories for multiple distributions is cumbersome (although some platforms like the openSUSE Build Service make this easier)
• As the package formats supported by package managers are based on archive formats, the AppImages would have to be archived which means they would have to be extracted/installed - a step that takes additional time
• Delta updates would be complicated to impossible which means a lot of bandwidth would be wasted (depending on what you consider "a lot")
• Users would need to configure repositories in their systems (which is cumbersome) and would need root rights to update AppImages

AppImageUpdate to the rescue.

Objectives

AppImageUpdate has been created with specific objectives in mind.

1. Be Simple. Like using AppImages, updating them should be really easy. Also updates should be easy to understand, create, and manage.
2. Be decentral. Do not rely on central repositories or distributions. While you can use repositories like Bintray with AppImageUpdate, this is purely optional. As long as your webserver supports range requests, you can host your updates entirely yourself.
3. Be Fast. Increase velocity by making software updates really fast. This means using delta updates, and allow leveraging existing content delivery networks (CDNs).
4. Be extensible. Allow for other transport and distribution mechanisms (like peer-to-peer) in the future.
5. Inherit the intensions of the AppImage format.

Update information overview

In order for AppImageUpdate to do its magic, Update information must be embedded inside the AppImage. Update information is what tells AppImageUpdate vital data such as:

• How can I find out the latest version (e.g., the URL that tells me the latest version)?
• How can I find out the delta (the portions of the applications that have changed) between my local version and the latest version?
• How can I download the delta between my local version and the latest version (e.g., the URL of the download server)?

While all of this information could simply be put inside the AppImage, this could be a bit inconvenient since that would mean changing the download server location would require the AppImage to be re-created. Hence, this information is not put into the file system inside the AppImage, but rather embedded into the AppImage in a way that makes it very easy to change this information should it be required, e.g., if you put the files onto a different download server. As you will probably know, an AppImage is both an ISO 9660 file (that you can mount and examine) and an ELF executable (that you can execute). We are using the ISO 9660 Volume Descriptor #1 field "Application Used" to store this information.

Projects using AppImageUpdate

• Open Build Service automatically injects update information into every AppImage built there
• linuxdeployqt, a tool that makes it easy to build AppImages, automatically injects update information into every AppImage built on Travis CI

Building

You should be able to add this repository (with this branch) as a submodule in your own repository. When using CMake, call add_subdirectory() on the submodule path. The header directories should then be added globally. All you need to do is link your application against libappimageupdate. For example by calling target_link_libraries(mytarget libappimageupdate).

Note that as mentioned previously, this project uses C++11. However, care has been taken to make public headers work in older versions of C++ as well. Therefore, it should not be necessary to set CMAKE_CXX_STANDARD, -std=c++11 etc. in projects using this library. If you notice that this stops working, please open an issue.

Licensing

For details on the licensing of the libraries AppImageUpdate uses, please see their documentation.

The code of AppImageUpdate is licensed under the terms of the MIT license. Please see LICENSE.txt for details.

The sample AppImageUpdate GUI is based in part on the work of the FLTK project (http://www.fltk.org). We are interested in adding Qt and Gtk+ versions.

TODO

• On this page, describe or link to a description of how to add update information and how to upload the files, similar to the old version located at https://github.com/AppImage/AppImageUpdate/blob/master/README.md

Contact

If you have questions, the developers are on #AppImage on irc.libera.chat.

appimageupdate's Issues

Investigate brotli, butler, and wharf from itch.io

Looks like they have a very elaborate and well-thought-out solution in place, what could it offer over zsync for AppImageUpdate?

itchio/itch#683

Support type-1 AppImages

Should be rather easy to implement since the update information is at a fixed offset.

New version does not end up in directory of old version

Can be reproduced with AppImageUpdate version 1-alpha (commit 6fbc9e9), build 122 built on 2017-11-12 00:58:28 UTC like this:

1. Download https://github.com/Subsurface-divelog/subsurface/releases/download/v4.7.3/Subsurface-6f65f5191-x86_64.AppImage (v4.7.3) to $HOME/Downloads (where the browser puts it by default)
2. Use AppImageUpdate on it
3. Expectation: we should now have the new version in $HOME/Downloads, next to the old version
4. Actually it landed somewhere else, namely in $HOME

Updating AppImageUpdate with itself does not work as expected

Self-updating AppImageUpdate does not work as expected when using an older version of AppImgageUpdate (from earlier today) - I can see there is a newer pre-release available on GitHub Releases (since 26 minutes or so).

The update information inside this AppImage is gh-releases-zsync|AppImage|AppImageUpdate|continuous|AppImageUpdate-*x86_64.AppImage.zsync

Self-updating AppImages

Continuation of the discussion in AppImage/AppImageKit#133.

It would be nice to offer some tools allowing AppImages to check for updates and eventually update themselves, which can be used in special cases where this feature is required.

Therefore, a binary called AppImageSelfUpdate (a modified variant of the new GUI AppImageUpdate) has been implemented, making self-updates as easy as calling a subprocess.

The idea is that AppImageSelfUpdate can configure itself from the environment so that the binary just needs to close itself and call that binary using an exec() call.

A feature that is missing at the moment is to offer AppImages an "update check" using the algorithms implemented in AppImageSelfUpdate anyway.

We are working on such a concept at the moment in a wiki page here:
https://github.com/AppImage/AppImageUpdate/wiki/Self-updating-AppImages. This page is going to be updated with a more final concept soon.

AppImageUpdate icon missing

AppImage contains:

me@host:~$ find /tmp/.mount_AppImaZ67Gwk/ | grep icons
(...)
/tmp/.mount_AppImaZ67Gwk/usr/share/icons/hicolor/scalable/AppImage.svg

Desktop file says:

me@host:~$ cat /tmp/.mount_AppImaZ67Gwk/AppImageUpdate.desktop 
[Desktop Entry]
Name=AppImageUpdate
Type=Application
Icon=AppImage
Exec=AppImageUpdate

Probably need to set the _NET_WM_ICON property of toplevel window? In Qt, QApplication::setWindowIcon() is known to do the trick.

Full download when local filename differs from the remote one

AppImageUpdate tends to completely download a new file from the server when the server's and the local filename don't match.
The problem is that zsync (which is called internally) looks for a file matching Filename: in the .zsync file. This is totally fine behavior for "normal" download jobs, but does not work very well with the way we use it.
Thus, we should make sure that Filename: matches argv[1], by replacing that (using sed or a comparable alternative).

"Could not find update information in the AppImage" despite it being there

me@host:~$ Downloads/linuxdeployqt-continuous-x86_64.AppImage --appimage-updateinformation
gh-releases-zsync|probonopd|linuxdeployqt|latest|linuxdeployqt-_*-x86_64.AppImage.zsync
me@host:~$ Downloads/AppImageUpdate-x86_64.AppImage Downloads/linuxdeployqt-continuous-x86_64.AppImage 
AppImageUpdate version 1-alpha (commit 513949c), build 108 built on 2017-11-04 00:22:04 UTC
Checking for updates...
Could not find update information in the AppImage! Please contact the author to embed update information!

Fails on type-1 AppImages without embedded magic bytes

me@host:~$ xxd /isodevice/Applications/Audacity-2.0.5.glibc2.15-x86_64.AppImage | head -n 1
00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000  .ELF............

me@host:~$ strings /isodevice/Applications/Audacity-2.0.5.glibc2.15-x86_64.AppImage | grep CD001
CD001

As per the AppImageSpec, this is a valid type-1 AppImage (as is any ISO9660 file that is a Linux ELF file at the same time).

Nevertheless, AppImageUpdate fails:

me@host:~$ Downloads/AppImageUpdate-164-6e435c2-x86_64.AppImage /isodevice/Applications/Audacity-2.0.5.glibc2.15-x86_64.AppImage 
AppImageUpdate version 1-alpha (commit 6e435c2), build 164 built on 2017-11-21 11:07:44 UTC
Checking for updates...
Invalid magic bytes: 00
Parsing AppImage failed. See previous message for details. Are you sure the file is an AppImage?

Clicking the Run app! button is running the old rather than the new version

• Pop!OS pop-os_amd64_intel_45.iso
• Old version stored at read-only location /isodevice/Applications/Subsurface-x86_64.AppImage
• Running AppImageUpdate-x86_64.AppImage 6fbc9e9
• Gui seems to do something and suggests application has been updated to the latest version
• Where is the latest version stored? It does not show the filename nor location of the new file
• Clicking the Run app! button shows that is not the latest version (screenshot) - it is running the old rather than the new version

me@host:~$ Qulo/AppImageUpdate-x86_64.AppImage /isodevice/Applications/Subsurface-x86_64.AppImage 
AppImageUpdate version 1-alpha (commit 6fbc9e9), build 122 built on 2017-11-12 00:58:28 UTC
XOpenIM() failed
Checking for updates...
Fetching release information for tag "continuous" from GitHub API.
... done
Starting update...
Fetching release information for tag "continuous" from GitHub API.
Updating from GitHub Releases via ZSync
zsync2: Reading seed file: /isodevice/Applications/Subsurface-x86_64.AppImage
zsync2: Usable data from seed files: 32.641444%
zsync2: Renaming temp file
zsync2: Fetching remaining blocks
zsync2: Downloading from https://github-production-release-asset-2e65be.s3.amazonaws.com/2319498/25016776-c70a-11e7-82a9-201d837e0259?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20171112%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20171112T063052Z&X-Amz-Expires=300&X-Amz-Signature=be5edc9f6eeba986778e15c31dbb3ade0d1b75fb0f3cdeb5c220918b38550ee2&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3DSubsurface-4d04f7431-x86_64.AppImage&response-content-type=application%2Foctet-stream
zsync2: Verifying downloaded file
zsync2: checksum matches OK
zsync2: used 23252992 local, fetched 47984640
Update successful

Update zsync2 to latest

As fixed compile error.

ocs-manager imported AppImageUpdate repos when build time.

Add method to read total file size of update from .zsync file

Required for @probonopd's UI proposal in https://github.com/opendesktop/ocsstore/issues/8#issuecomment-340486795 (second screenshot) to work.

It should be possible to query the total file size of the new file to be queried using the Updater API. This is fairly easy to implement, and, in combination with the updater.progress() method, allows for displaying the amount of data that is available locally and/or has been downloaded already.

appimageupdatetool does not work in systems without objdump

Using the tool as appimageupdatetool.AppImage <some appimage> without objdump installed results in the following output:

Checking for updates...
sh: 1: objdump: not found

and an indefinite hang.

Increase verbosity in error messages

As the user, I would like to know

• Did the AppImage provide no update information this version of AppImageUpdate can understand? In this case, I may want to update to the latest AppImageUpdate
• Was there no Internet connectivity? In this case, I may want to switch on WLAN
• Was there an issue with the HTTPS certificates? In this case, I may want to check my clock
  etc.

Add Travis build ID to AppImage filenames

I think it might make sense to add the travis build ID, a monotonically increasing unsigned integer, to the AppImage filenames. This would allow users to see which binary is newer by comparing those numbers by size, which is something the Git commit hash doesn't provide.

CC @probonopd.

The result of this discussion will be applied to linuxdeployqt's and AppImageKit's AppImage releases, too.

Consistent naming

Let's use AppImageUpdate as the user-facing name for the user-visible GUI application. (You will never see "GUI" in the user-facing name of commercial applications; grandma doesn't know what that even means.)

appimageupdate is the name of the command line tool, following the macOS convention that GUI applications are upper-case while command line tools are lower-case. If this distinction is not sufficient, we could also call the CLI tool appimageupdatetool.

rename: Invalid cross-device link

/isodevice/Applications/ is on a FAT32 partition.

me@host:~$ sudo /isodevice/Applications/AppImageUpdate-162-2720e18-x86_64.AppImage /isodevice/Applications/Quassel_Client-61fc76a-x86_64.AppImage 
AppImageUpdate version 1-alpha (commit 2720e18), build 162 built on 2017-11-17 01:19:19 UTC
Checking for updates...
Fetching release information for tag "continuous" from GitHub API.
... done
Starting update...
Fetching release information for tag "continuous" from GitHub API.
Updating from GitHub Releases via ZSync
zsync2: Target file: /isodevice/Applications/Quassel_Client-773227a-x86_64.AppImage
zsync2: Reading seed file: /isodevice/Applications/Quassel_Client-61fc76a-x86_64.AppImage
rename: Invalid cross-device link
zsync2: Usable data from seed files: 80.501089%
zsync2: Renaming temp file
Update failed

Possibly it would be a good idea to have the temp file in the same directory as the target file?

Automatically elevate to root if necessary

Despite applying AppImage/AppImageKit#364 in 1ffe5aa we still get

when trying to update an AppImage in a location that the current user has no write rights for (e.g., /opt).

Perhaps @abhay44 can have a look?

`appimageupdatetool-179-30cb1ce-x86_64.AppImage` fails

I tried to use appimageupdatetool-*.AppImage from the commandline, but in a very non-standard way:

1. I call it via a symlink named aiu-tool.
2. I run it against a symlink named im pointing to the real AppImage of ImageMagick.

It all goes well until about 69%. Then it falls down with a chain of messages saying:

 [....]
-1 returned
-1 returned

zsync2: failed to retrieve from ImageMagick-bc5b257-gcc-x86_64.AppImage, status -1
100.00% done (19.40 of 19.40 MiB)...
Update failed!

I'm aware that my AppImage should really carry a proper version information (which it doesn't do this yet) -- but I also like to "kick the tires" in order to test stuff.

The ImageMagick AppImage so far works as expected, even though the Travis build at the end said "Your build exited with 1." and I had to "force" the upload to the GitHub releases page.

However, I submit this issue because you may gain interesting insights into your tool when it has to deal with a possibly faulty AppImage.

---

I also ran aiu-tool --self-update and ./aiu-tool ./aiu-tool, which both worked, despite of using symlinks only.

The updated aiu-tool symlink meanwhile points to appimageupdatetool-180-d89753f-x86_64.AppImage, but this also fails on my ImageMagick AppImage. This most likely means that something is wrong with my ImageMagick AppImage and its .zsync file.

---

Files:

• My ImageMagick GitHub releases page
• ImageMagick-xperimental.AppImage
• ImageMagick-bc5b257-gcc-x86_64.AppImage.zsync

"Run app" for CLI apps

When you update a CLI app (desktop file has Terminal=true), then "Run app" should open the app in a terminal.

wget https://download.opensuse.org/repositories/home:/nandub:/MyAppImages/AppImage/aria2-latest-x86_64.AppImage
# Run AppImageUpdate GUI on the just downloaded AppImage
# "Run app" --> Nothing happens in the GUI

Note that this particular AppImage has True instead of true:

me@host:~$ /home/me/Downloads/aria2-1.24.0-1.1.Build12.15.glibc2.17-x86_64.AppImage --appimage-extract '*.desktop'
squashfs-root/aria2.desktop
me@host:~$ cat squashfs-root/aria2.desktop 
[Desktop Entry]
Type=Application
Name=Aria2
Exec=aria2c
Icon=aria2
Categories=Network;Download Manager
Comment=aria2 is a lightweight multi-protocol & multi-source, cross platform download utility operated in command-line. It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink
Terminal=True

me@host:~$ desktop-file-validate squashfs-root/aria2.desktop 
squashfs-root/aria2.desktop: error: value "Network;Download Manager" for string list key "Categories" in group "Desktop Entry" does not have a semicolon (';') as trailing character
squashfs-root/aria2.desktop: error: value "True" for boolean key "Terminal" in group "Desktop Entry" contains invalid characters, boolean values must be "false" or "true"

Evaluate efficiency of zsync2

I'd like to spend some time (2+ days) on evaluating the current use of zsync2 with type 2 AppImages, i.e., compare how much has changed file wise (before calling mksquashfs, that is), and the difference ratio that zsync2 calculates. I feel like it tends to download more data than what has actually changed, but I'd rather perform some measurements before speculating in any way.

User stories:

• I as a user expect AppImageUpdate to download only the blocks for a file that is added between two releases.

• I as a user expect AppImageUpdate to be able to download blocks for an entire file at max if that file has been changed, and nothing else.

• I as a developer hosting releases of my application expect AppImageUpdate to minimize the traffic generated for updates.

At the moment, this is promised, and might be true, but it'd be nice to create a little, meaningful study on that, which we can show to people asking about this. Also, it's a great way to find potential for optimizations. And, as we plan to keep zsync2 as our core functionality (even when using alternatives to the classic server-client architecture, such as peer to peer networks (see AppImage/AppImageKit#175)), now seems to be the right time to investigate these issues.

Factors that might have to be optimized are unequal block sizes for zsyncmake2 and mksquashfs, compression after generation of the squashfs image (as far as I know, mksquashfs pads files to fill up the remaining bytes in the last block of a file), etc. Anything that could lead to equal files being stored in a way so that the hash sums for the occupied blocks are different, basically.

Rule of thumb for block sizes: mksquashfs block size >= zsyncmake2 block size, mksquashfs block size mod zsyncmake2 = 0, block sizes should be powers of 2.

To my knowledge, such measurements haven't been performed yet, or at least haven't been set up in a more scientific way, capturing and visualizing the results in a repeatable way.

I think we could e.g., use any random Qt application (bundled with linuxdeployqt and the exact same Qt version) with changes in the main binary only, or some CLI applications bundling just a few but rarely changing libraries.

The goal is to find potential optimizations in our use of squashfs which would decrease the aforementioned difference ratio to save on bandwidth. I think it should be possible to find an acceptable trade-off of higher file space versus more efficient updates.

TODO:

• set up test infrastructure (i.e., a zsync2 flag only calculating the difference ratio, without actually downloading the file (an extension to -j), and some repository containing the AppDirs of which we generate AppImages for with appimagetool, eliminating external dependencies)
• describe test methods
• perform first measurements representing the current state
• experiment with settings, generating more data, to be compared with first measurements
• make up list of parameters used initially and the ones gained from the experiments
• generate final data for all parameters ever used, plot data

Only show progress bar if there is progress to show

In this case, there is no download progress to show, hence we should only show the topmost window:

In other words, only show the progress bar window if the progress is > 0%.

Investigate xdelta3

https://github.com/jmacd/xdelta

What are its pros/cons vs. zsync?

Says I have the latest version, but I don't

me@host:~$ ./OldImageMagick-git.7a48608-x86_64.AppImage --appimage-updateinformation
gh-releases-zsync|probonopd|ImageMagick|continuous|ImageMagick*-x86_64.AppImage.zsync

AppImageUpdate version 1-alpha (commit 48d9713), build 119 built on 2017-11-07 02:37:53 UTC says that I already have the latest version, except that it isn't:

(Zipped for GitHub Issues)
OldImageMagick-git.7a48608-x86_64.AppImage.zip

Add some tests on Travis CI

Let's add some tests on Travis CI, e.g.,

• AppImageUpdate tries to update itself (using an outdated version of itself)
• AppImageUpdate tries to update itself (using the recent version of itself)
• AppImageUpdate tries to update another application that was only half-downloaded

[rewrite] Do not leave .zs-old files around

When the file was already 100% downloaded and up to date and the user then tries to update it, a .zs-old file is left behind:

me@host:~$ ls
appimaged-x86_64.AppImage	  Desktop    Downloads	Pictures
appimaged-x86_64.AppImage.zs-old  Documents  Music	Videos
me@host:~$ rm appimaged-x86_64.AppImage.zs-old 
me@host:~$ ls
appimaged-x86_64.AppImage  Desktop  Documents  Downloads  Music  Pictures  Videos
me@host:~$ Downloads/appimageupdate-x86_64.AppImage appimaged-x86_64.AppImage
Starting update...
Updating from generic server via ZSync
Update URL: https://github.com/AppImage/AppImageKit/releases/download/continuous/appimaged-x86_64.AppImage.zsync
zsync2: appimaged-x86_64.AppImage found, using as seed file
zsync2: Reading seed file: appimaged-x86_64.AppImage
zsync2: Usable data from seed files: 100.000000%
zsync2: Renaming temp file
zsync2: Fetching remaining blocks
zsync2: Verifying downloaded file
zsync2: checksum matches OK
zsync2: used 245760 local, fetched 0
100% done...
Update successful!
me@host:~$ ls
appimaged-x86_64.AppImage	  Desktop    Downloads	Pictures
appimaged-x86_64.AppImage.zs-old  Documents  Music	Videos

Check the gpg signature when updating

This issue is created just for reference so that this is not forgotten.

At the moment, the gpg_check method is commented out as you can see in the current implementation:

gpg_check()
{
  set +e
  if [ "$(which gpg_currently_disabled)" != "" ] ; then
    echo "GPG is installed; hence attempting signature verification"
    ASC_URL=$( echo "${ZSYNC_URL}" | sed -e 's|.zsync|.asc|g' )
    # curl -k -L -O -# "${ASC_URL}" && gpg --verify $(basename "${ASC_URL}") && echo "Signature verified"
  else
    echo "GPG is not installed; hence skipping signature verification" >&2
  fi
  set -e
}

After reading AppImage/AppImageKit#238, I came up with the following solution:

gpg_check()
{
  set +e
  if [[ "$(which gpg2)" != "" && "$(which validate)" != "" ]]; then
    echo "GPG is installed; hence attempting signature verification"
    validate "${ISO}"

    if [[ $? -eq 0 ]]; then
      echo "GPG signature verified"
    else
      echo "GPG verification failed. Please close and retry"
      mv -f "${ISO}.zs-old" "${ISO}"
      exit 1
    fi
  else
    echo "GPG is not installed; hence skipping signature verification" >&2
  fi
  set -e
}

All that's needed is to include the validate binary into the PATH so that appimageupdate can run it. In my case, I created a self update-able AppImage adding those two plus appimageupdategui and zsync_curl.

Tries to make wrong file executable

AppImageUpdate works for non-executable AppImages, which is fine, but when you press Launch, also non-executable files are run. That's some kind of odd behavior, it should be changed to either changing the permissions of the AppImage so that the executable bit is set (I'd prefer this way) or showing an error message if it is not executable (the safer way).

Write How-To/Guide explaining how to consume libappimageupdate in other applications

Two use cases should be covered:

• How to update foreign AppImages (app store workflow, see https://github.com/opendesktop/ocsstore/issues/8).
• How to update own AppImage (will surely be useful for @dirkhh and https://github.com/Subsurface-divelog/subsurface/)

The former has the higher priority, as @akiraohgaki needs this for OCS-Store.

Improve zsync-curl integration

At the moment, the Vala GUI is just a wrapper for the Bash script, and it can be tricky to make such process wrappers reliable and stable. It works, but to make it become the official updater for AppImages and reach a larger user base, it needs to be improved.

I experience problems with my Red Eclipse AppImages due to this. They are very large, and the UI doesn't really work well together with my AppImages and zsync files. It gets stuck, gets into inconsistent states etc. I fixed one of these states already, but with the current code base, it's quite a PITA due to how it works at the moment.

I think to solve this properly, we have to directly use some kind of zsync_curl API instead of relying on the bash script. This probably improves the experience a lot, and should make the whole thing more reliable and stable. I believe the AppImage project would benefit a lot from such a tool.

My approach would be to write an AppImageUpdate C(++) framework based on the zsync-curl source that has two or more frontends, a CLI and a GUI application. This would make the bash script obsolete, as a side effect.

While working on this, we might also find a solution to integrate it within AppImages directly in a better way. Ideally, you'd specify a url: "https://..." key in a YML recipe, and the rest happened automagically.

It's probably a good chance to improve my Vala knowledge, so I'd like to contribute. But this is probably going to take some time, I'm busy for the next weeks. If someone else finds this useful though, I'd try to help as good as I can.

Assemble "Release notes" by looking at the GitHub API

Sparkle Framework on macOS shows these information windows:

But they have an extra feed (AppCast) for this kind of metadata, which is extra work to produce.

QGitHubReleaseAPI seems to be able to put together something similar from the information on GitHub:

https://github.com/velnias75/QGitHubReleaseAPI

Move permissions logic from GUI to libzsync2

Move permissions logic from GUI to libappimageupdate so that users of the library don't have to deal with this in their own code:

https://github.com/AppImage/AppImageUpdate/blob/0f51e5c4aee19b3db651bd69cfb41c3b7b67f4ae/src/gui_main.cpp#L323-L331

Does not handle partially downloaded files correctly

me@host:~$ dd if=Downloads/OpenSCAD-Nightly-0-Build1967.1.glibc2.14-x86_64.AppImage of=partial.AppImage bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00188044 s, 558 MB/s

Expected result:

Downloads the rest of the file.

Actual result:

When trying to update this with AppImageUpdate-x86_64.AppImage via the GUI:

Of course, running this fails:

When trying to update the same partial AppImage via the CLI using the same AppImageUpdate-x86_64.AppImage:

me@host:~$ Downloads/AppImageUpdate-x86_64.AppImage partial.AppImage 
Checking for updates...
Parsing failed! Are you sure the file is an AppImage?

Handle special characters in path to AppImage

Need to gandle special characters (such as brackets and whitespace) in the path to the AppImage when supplied on the command line:

--version

It would be cool if we could somehow find out the version of the locally downloaded AppImageUpdate and appimageupdatetool.

The "-j" option does not report any error message by zsync client

Since for the "-j" option the zsync client is created on the stack (https://github.com/AppImage/AppImageUpdate/blob/rewrite/src/updater.cpp#L520) and not assigned to zsyncClient, there's no way to currently pull for error messages https://github.com/AppImage/AppImageUpdate/blob/rewrite/src/updater.cpp#L618.

Get rid of the dependency on the curl command line tool

This functionality could be handled by zsync_curl as well

FLTK does not scale on HiDPI systems

Follow common GUI guidelines

In line with other GUI applications (example: Firefox), let's have

• The default (OK) button in the bottom right
• The cancel button to the left of the default button
• No icons inside the buttons
• No punctuation (!, .) inside buttons
• No exclamation marks in text in dialog boxes

This will make the UI look more familiar and in line with most applications.

Screenshot: How Firefox asks when opening files marked as executable (e.g., by appimaged). Note: This is a Firefox dialog, not something from AppImageKit.

Screenshot: AppImageUpdate

Out of source builds no longer work

A line introduced in 0d169e4 won't work when building out of source tree. The file build-appimages.sh is meant to be used out of source however, that's why it is written with absolute paths. It even builds out of source in a RAM disk automatically, reducing disk IO and speeding up builds.

https://github.com/AppImage/AppImageUpdate/blob/0d169e4c0f06aac8a531025589a872c4ed255436/resources/build-appimages.sh#L25-L29

It should be moved to the appropriate location, which should fix local builds.

Move "run app" logic to shared module.

Currently, the "run" logic is available in the GUI application only, but it might be a good idea to move it out to a module shared between all applications. It might even make sense to move it to the Updater class, to allow other libraries to make use of its functionality.

Related to #31.

Add "appimage-help" to display all possible arguments for a particular AppImage

Quite a few times I've had to go to the documentation on the web to check for the proper argument to use. I think it would be useful if we had an argument appimage-help that shows you all the possible commands you can use for that particular AppImage.

What do you think?

Help integrating libappimageupdate into existing GUIs

As we want AppImage(Self)Update to look and feel as native as possible on every application platform, we might prefer to work with existing Qt, Gtk+, Electron, etc. updater projects rather than writing all of these GUIs and integrations (for self-updating) on our own.

Objective is to achieve a user experience roughly similar to the Sparkle Framework for macOS, but using AppImage and binary delta updates, for Linux.

Let's collect a list of promising candidates here.

Qt

• https://github.com/alex-spataru/QSimpleUpdater
• https://github.com/Skycoder42/QtAutoUpdater

Electron

• https://www.npmjs.com/package/electron-simple-updater
• https://www.npmjs.com/package/electron-installer-appimage
• https://www.npmjs.com/package/electron-updater

Python

• https://github.com/google/omaha (Google Updater)

Windows

• https://houbysoft.com/libhul/

Cross-platform

• https://github.com/Mendeley/Update-Installer

objdump missing on Kubuntu 17.10

A critical bug. objdump command is missing, hence AppImageUpdate fails to parse the update information.

The call to objdump needs to be replaced with the ELF related code @probonopd wrote for AppImageKit asap.

Do not assume certain filename endings

Do not assume certain filename suffixes. AppImages can have any or no suffix.

Investigate Beaker and Dat

Investigate Beaker and Dat as a possible way to marry p2p distribution/hosting with binary delta updates.

https://twitter.com/mafintosh/status/925258772406206464

Dat supports variable chunking so you can use the chunker best fitted for your needs. Diffing is not yet impl to focus on basic features 1st

Add support for Mirrorbrain features

CC AppImageKit, AppImageSpec, @probonopd, @adrianschroeter

Some projects like KDE or openSUSE use Mirrorbrain to publish their releases. To make it easier for such projects to add update information to their AppImages, we could add an update type supporting this platform.

Mirrorbrain differs from the random server by:

• providing a list of mirrors of the files, sorting them by a few factors like "distance" etc.
• provides integrity information, like file hashes, GPG signatures, etc.
• while some of the mirrors support encrypted connections (HTTPS), Mirrorbrain does not (according to the KDE sysadmin team), and advertising those with priority would require patches to Mirrorbrain, and since the project seems unmaintained, that'd require a fork

I already asked in #kde-sysadmin on Freenode about their infrastructure. My initial intention was to make them enforce HTTPS in their mirror infrastructure (in days of Let's Encrypt, this is a valid request), as that also solves the problem "people observing data stream can easily see what files are being requested" (because that's what people don't think of when talking about HTTPS), and obviously, people being too lazy to actually check the files they download, so having the integrity and (server) authenticity checks HTTPS enforces for those initial downloads would be awesome.

I think if we wanted to help them provide updatable AppImages from such an infrastructure, we should add a Mirrorbrain update information type which at least performs the update checks, because I think it Mirrorbrain can redirect automatically if one requests the right URL from the beginning.

Thinking about it again, IIRC, Mirrorbrain returns at least the hashes as HTTP headers, so, to achieve integrity checks based on these (because GPG signatures without verifying the keys aren't anything but pure hash-based integrity checks), we could make use of those by default, if available. This is primarily useful when downloading the .zsync file from an unauthenticated server (after the redirect from the HTTPS enabled mirrorbrain master): When it's downloaded, check its integrity to make sure the right file has been downloaded. This is similar to how APT works -- they sign and verify the central file which contains hashes of all the files in the repository, and we do the same, as the .zsync file contains integrity information (file hash) of the target file, hence one can ensure that the target file's integrity check is reliable after verifying the .zsync file's integrity.
@adrianschroeter once said in the Mirrorbrain configs it was possible to serve certain files from the master server, but we can't/shouldn't rely on this to be configured correctly.

What do you guys think? Personally, I believe we have to improve this situation, especially since we're advertising [OBS], whose artifacts storage is regularly redirecting to HTTP-only servers (I think I posted that somewhere in the OBS repository, if anyone's curious, I don't have the link any more), and therefore we should at least fix one of the problems due to the lack of HTTPS.

Side note: I was analyzing the Kdenlive infrastructure initially, having me analyze the behavior. I verified this with openSUSE's installation because that service is hosting a large share of AppImages. Kdenlive don't provide updates yet, this will be subject of a second issue.

Funny characters inside the progress bar

There are funny characters inside the progress bar

Also, the GUI would look a bit less "alien" if the progress bar was not yellow but e.g., grey.

Launch button can be clicked while update is running

The launch button can be clicked while the update is still running. It doesn't actually launch the AppImage, but it is kind of odd that it is clickable at all. It should be disabled until the update has finished.

[rewrite] Segfault when appimageupdategui-x86_64.AppImage tries to update itself

me@host:~$ wget https://transfer.sh/Kljvv/appimageupdategui-x86_64.AppImage
me@host:~$ chmod a+x appimageupdategui-x86_64.AppImage
me@host:~$ /appimageupdategui-x86_64.AppImage appimageupdategui-x86_64.AppImage 
Checking for updates...
Speicherzugriffsfehler