canonical/snapcraft.io

The official website's repository for the Snap store

snapcraft.io codebase

Python Coverage | JavaScript Coverage

Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully. This repo is the application for the snapcraft.io website.

If you are interested in Snaps, Snapping and Snapcraft, there is an active discourse forum that we encourage developers to join.

The site is largely maintained by the Web and Design team at Canonical. It is a stateless website project based on Flask and hosted on a Charmed Kubernetes cluster.

Bugs and issues

If you have found a bug on the site or have an idea for a new feature, feel free to create a new issue, or suggest a fix by creating a pull request. You can also find a link to create issues in the footer of every page of the site itself.

Bugs in snaps and tools

If you have found a bug elsewhere in the snap world:

• For issues with an individual snap - you can run snap info and use the contact information to find where you can get help.
• In the snapcraft tool - that builds and publishes snaps, file it here
• In Snapd, the daemon that manages snaps on the client, file it here

Local development

The simplest way to run the site locally is using the dotrun snap:

dotrun

Once the server has started, you can visit http://127.0.0.1:8004 in your browser. You stop the server using <ctrl>+c.

For more detailed local development instructions, see HACKING.md.

Brand stores

This codebase can be modified to setup branded stores that represent specific brand or devices, giving the brand full control over the store content, reviewing process and identity.

• For companies looking to develop a brand store with Canonical ›
• For developers to learn more about developing a brandstore, see BRANDSTORES.md.

Deploy

You can find the deployment config in the deploy folder.

License

The content of this project is licensed under the Creative Commons Attribution-ShareAlike 4.0 International license, and the underlying code used to format and display that content is licensed under the LGPLv3 by Canonical Ltd.

With ♥ from Canonical

snapcraft.io's Issues

Handle someone choosing to create snapcraft.yaml before name is registered

We recommend that someone register the snap name before adding a snapcraft.yaml — because for publishing, snapcraft.yaml needs to include the registered name.

But you might want to add a snapcraft.yaml before registering the snap name, for example if you want to see whether your software works as a snap at all.

Therefore, we should customize the contents of the template snapcraft.yaml in this case, to tell you that it won’t publish until you register the name, and tell you how to proceed.

Snap icon and name should use bigger vanilla media pattern (not grid)

Initial 404 timeout takes a long time

I didn't measure, but it appears to take a long time to get the first 404 page to display. FWIW, the second is nearly instant.

Consider populating snapcraft.yaml with contextual config

Given b.d.i is talking to github and can see a project source and meta data, it seems to make sense to use that data to pre-populate the yaml rather than give a generic boilerplate example.

The metadata such as project name, most recent version, short name and description could potentially be gleaned easily and injected directly into the initial yaml.

Via some simple heuristics we could determine whether a project uses the python, nodejs, autotools, go or cmake/qmake project and populate a part with some appropriate data including the source url. With the use of some comments we could indicate to the developer that this is not guaranteed to work as-is, but is a template built based on the available code.

Look into integration testing

We should provide some tests for the snap details page functionality, but not necessarily unit tests.

Query user agent string to display/hide install link

Create snap page template and URL endpoint

Create a snap page template with dummy data, and a route & view so that requests for snap details URLs will display the dummy page.

404 page

Display the 404 page for:

• Erroneous URLs
• Snaps that don't exist
• Snaps that are "private"

Can’t add private repositories

1. Choose “Set up in minutes” or “Add repos…”.
2. Try to choose a private repo.

What happens: You can’t.

What should happen: You can.

(Not to be confused with canonical-web-and-design/build.snapcraft.io#132, the equivalent for organization repos.)

Footer: Text improvements

Update homepage to latest vanilla framework

To use latest vanilla and the same HTML frame as the snap details pages.

Organisations: Notify the user on registering organisation snaps to explain how to add collaborators [3]

Create Flask skeleton application

Lay out the files in the basic Flask application structure.

Fall-back "install" text for macOS: Build

Expose geodata from API to view

Fetch data from API for snap details page

Wire up the back-end to the API to retrieve live data for each snap.

Add social icons to footer

Add social icons (like on snapcraft.io) to footer.

Create it as a pattern that could be proposed to Vanilla.

Add vanilla styles to snap summary and description text

Designs: https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-326014685

Implement footer (as on snapcraft.io) for snap details page

Designs: https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-326014685

Investigate map chart libaries

Geodata shape proposal and wireframes:
https://docs.google.com/document/d/1dACEHFER9Z0S1lOdm_vyOnpT3uMo87-Ak7Hf5zrX3wo/edit

Snap details map visual design:
https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-324969516

How to handle user authorization in a backend-first architecture

In this application, we've opted at the moment to put the core application functionality for building the Markup for presentation in the Flask application and the related templates on the server-side, rather than pushing that responsibility to a Single Page Application (SPA) in the client browser by using a frontend framework like React - which is how build.snapcraft.io was built.

@cprov raised some concerns about this choice, partly in #11 (review). Here I'll first explain the decision and then discuss the one downside that I don't quite know how to address yet.

Benefits of a backend-first architecture

There are a few reasons that we've made this decision:

• Our experience with build.snapcraft.io has taught us that it is difficult to encapsulate and manage complexity in a SPA. Seemingly simple additions or removals require changes in many parts of the application.
• Similarly, it is difficult to keep track of the application's state. It is easy to end up with subtle bugs where very specific flows through the application lead to a broken state.
• The webteam at large has more experience building traditional back-end focussed applications, and so we believe we have successful patterns for doing so
• Similarly, as we have many applications that use a similar architecture, this structure makes it easier to share knowledge, modules and patterns with other projects

The choice between this traditional architecture vs a SPA architecture certainly depends on the type of application. For an application that is for, the most part, an interactive experience with complex flows and interactions (like, for example, the Juju GUI) an SPA is definitely the way to go. In contrast, an app that is mostly made up of pages/views - where the user moves from one view to the next - is probably more suited to a traditional architecture. We believe that, as snapcraft.io is intended to be the home of the snap store, this would certainly fits closer to the latter type of application.

Neither is this a binary choice. An SPA will certainly push certain bits of functionality to the server, and similarly, a backend-first architecture can push as much interactive functionality to the client as makes sense.

User authentication and authorization

The most significant downside to this architecture that I'm aware of is that of how to manage permissions for users when talking to the API.

In the traditional server-first model, the server-side application would simply be given fairly permissive access to the API, allowing it to access all the data it could possibly need. The application itself would then be responsible for authenticating users and managing their permissions.

An SPA model offers the benefit that the user's browser (the client) can communicate directly with the API, which allows the API to be the one that verifies end-users directly and delegates permissions for accessing data etc. This is particularly beneficial for cases like the Snap Store, because it means these permissions can be more easily managed in the same way and the same place for users of the website as they are for users of the CLI.

We would certainly want to continue this pattern here - to delegate management of user permissions to the Store. However, our architecture choices do mean that we would like the server-side application, rather than the client's browser, to be retrieving the data from the API and crafting it into a response.

I believe this is eminently possible, that macaroons should help us to manage this delegation of authentication from the user to the backend application. We should continue to discuss how this solution could look, here and elsewhere.

Add Sass building infrastructure & vanilla framework

Activity component

A graph to illustrate the most important statistic to a user. We're not quite sure what this is yet. Some things that were mentioned:

• Daily builds / commits (like GitHub)
• Successful / failing builds ratio

---

Copied from Trello card: https://trello.com/c/fFMHZQZy/77-activity-component
Labels: design

Can't build from a branch other than the default branch

I'm sure you know about that one, it's more for a tracking purpose.

You can't build to a different branch than master, which could be handy in that you have maintainance and release branch. Seems like a good opportunity for build.snapcraft.io

Add ./run script

• Setup ./run script
• Document properly in README

Wording for snap details page title

We should define how HTML <title> for snap details page should look like.

Currently it simply says: {{ snap_title }} snap details

As snap details page is meant to be part of snapcraft.io site we should probably have a consistent titles. Snapcraft.io currently appends Snaps are universal Linux packages. to the title of every page.

So maybe title of snap details page should be: {{ snap_title }} - Snap details - Snaps are universal Linux packages.

Do you have any suggestions @matthewpaulthomas?

Define final requirements for geodata and download data

From @nottrobin on September 11, 2017 13:43

Agree with OLS the final proposal for the geodata and download data

Copied from original issue: #373

Implement header with snapcraft logo for snap details page

Designs: https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-326014685

Get sign-off for geodata and download data plan

From @nottrobin on September 11, 2017 13:43

Copied from original issue: #374

Jenkins job for releasing to production

Create a Jenkins job to take the name of a docker image and release it to production.

How to delete your account?

As discussed in the workshop, build.snapcraft.io doesn't store any state for the user. Therefore, you can effectively "delete your account" by simply:

• deleting all your added repositories
• revoking the app's permissions in GitHub settings

However, we may, for users' peace of mind, want to either:

• Explain the above steps to "deleting your account" in an FAQ or other documentation page
  or
• Provide a "delete your account" link somewhere which simply deletes your repos for you - although I don't know if its possible to revoke GitHub permissions from our end.

Header should be full width

Can’t easily see contents of snapcraft.yaml for a failed build

When a build failed because of a problem with the snapcraft.yaml, it would be helpful to see what the file looked like at that stage.

Currently this requires using Git to find the revision of the file that was current when the build was attempted.

If we can remember and display the build log for each build, we should be able to remember and display the snapcraft.yaml contents for each build as well.

[Suggested by Heroku’s Jeff Dickey.]

Remove IRC from the "Get crafting!" paragraph

Please update the last paragraph of the "Get crafting" paragraph on the homepage to:

Read how to create a snap, join the snap-crafting community on our forum or ask questions real-time on Rocket Chat.

Design fallback "install" text for non-Linux visitors

Implement snap details component (icon, name, install button)

Designs: https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-326014685

[Pattern] Update media object as per designs

Create a new modified version of the media object pattern to display similarly to @spencerbygraves's designs.

Respect publishing rights for shared snaps

build.snapcraft.io doesn't seem to respect publishing rights for snaps when they are shared from another Ubuntu One account (via dashboard.snapcraft.io) with the currently logged in account.

Specific user story:

For our organization, we are using the sharing feature from dashboard.snapcraft.io to share access for our packages to the individual maintainers (and their own, individual Ubuntu One accounts). So, I'm able to add an organization repository without any problems. Although, when it comes to the step "register a snap name for publishing", I have to sign in to Ubuntu One, which I do with my individual account the snap package is shared with via dashboard.snapcraft.io. However, unfortunately, the site doesn't seem to respect the fact that the package is shared with my account and denies access with the message: "Sorry, that name is already taken".

See this forum topic for more information.

Build snaps for pull requests

Today we support building snaps for every commit that gets pushed to the default branch on GitHub. This allows for a workflow where you:

1. Open a pull request
2. Have your code automatically tested by Travis or CircleCI
3. Have your code reviewed by humans who will clone your branch and build locally
4. Land your pull request on master
5. Have your code automatically built and published by build.snapcraft.io

We could help earlier in that workflow by giving the reviewing developers prebuilt binaries to test with. These would be close to the binaries created off master after the branch lands.

This would allow for a workflow where you:

1. Open a pull request
2. Have your code automatically tested by Travis or CircleCI
3. Have your code automatically built by build.snapcraft.io and published to a channel branch (e.g. latest/edge/sergiusens:default-options)
4. Have your code reviewed by humans who will install from the channel branch
5. Land your pull request on master
6. Have your code automatically built and published by build.snapcraft.io

Add our standard set of labels to snapcraft-flask repository

Add vanilla table with snap package data

Designs: https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-326014685

Define requirements for geodata and download data

From @nottrobin on September 11, 2017 13:42

@carlaberkers @nottrobin @bartaz @matthewpaulthomas meet to discuss what questions need answering so we can understand the shape of geodata and download data that should be exposed by the snap store API.

Answer as many of those questions as possible.

Copied from original issue: #372

Footer: Implement improvements

Header should be white

Decide how extensively to cache requests

As @cprov mentioned in #11 (comment), we may want to reconsider whether we want to cache API responses in the way that we currently do. From @cprov:

I would be more concerned about retrying failed requests than caching results in the context. Sure, the load will be not neglectable on the store services but we are in a better position to cache it properly, specially when authentication comes to play. Also, considering the possible topology of these services, both deployed in PS4.5, request round-trips won't be expensive.
OTOH, retrying 5xx request has proven to be relevant to avoid network glitches to prevent external request to be fulfilled.

I don't feel comfortable removing caching entirely however:

I like the idea of retrying requests - that certainly makes sense. I'll also add a total timeout (2 seconds, probably), as I want to avoid this application hanging on a delayed API response.

I still have concerns about not providing some redundancy in this layer. For these public API requests, it seems to me to be perfectly valid to return cached data on error, and I think the public facing details pages should be especially resilient.

If it doesn't make sense to cache the authenticated API responses, then that's fine - I think an authenticated user would be less surprised to find that an API call was required in their requests - and it would be slightly less impactful if those pages broke.

I'd be happy to set up a different cache which honours caching headers rather than just caching for a set time (probably https://pypi.python.org/pypi/CacheControl) so that your application can choose how the response is cached if that would help. But for public pages at least, I'd still like to return cached data rather than a 500 page if we have an API error. We will still notify the user that the API response failed.

Let's continue the discussion here so it doesn't block the PR.

Jenkins code build and staging release

Create a jenkins job to respond to a github webhook whenever a merge happens into master, builds the docker image and releases it to staging kubernetes.

Add Vanilla responsive layout to snap details page

Designs: https://github.com/CanonicalLtd/snapcraft-design/issues/54#issuecomment-326014685

Publicly documented webhook for build with publication

Consider when:

• I want to trigger some follow-on test process to help inform whether the revision just landed in edge is ready for a more stable channel.
• I want to notify my team chat that a new build is available for testing.

In both cases I want the process to wait for a publishing attempt and send a single status that tells me whether the build succeeded and what channel and revision to find it under.

We should support and document (under https://snapcraft.io) how to set up webhooks for these cases.

[Pattern] Update key-value table

Create a modifier pattern for a key-value table, as per designs.

Offer support for GitLab.com

Using Snapcraft Build with GitHub seems amazing (though my builds aren’t working yet, but it’s on the user end). However, I’ve been starting to use GitLab more, and I have seen other Snapcraft projects on GitLab. While it’s great that Snapcraft Build works with GitHub, I think it’d also be useful if it works with GitLab as well.