dshoreman/servidor

A modern web application for managing servers

servidor's Issues

Setup travis

There are currently no tests, but it'd be nice if travis was working for when there are.

Besides, worst case it can probably figure out if something's messed up in composer/node package installation.

Add backend for system users and groups

Certain sites, applications or services may be better ran on their own user.

Frontend/General Checklist

Groups

• List all groups in new System/Groups page
• Selecting a group should open editor in right pane
• Include group users in a list under each group
• List users separately on the edit form with 'Remove from group' option
• Ability to add a user to a group, from Users or Groups page

Users

• Create 'Users' endpoint under new System section
• Display all users (including root and system users) in a list [of cards?]
• Ensure [system-]users can be added via backend+gui
• On edit form, display groups in a dropdown using new API endpoint
• Add a multiselect for the user's secondary groups

Misc

• Add a 'System' area to primary nav

API Checklist

Users

• GET /system/users
• POST /system/users
• PUT /system/users
• DELETE /system/users

Groups

• GET /system/groups
• POST /system/groups
• PUT /system/groups
• DELETE /system/users

Sample 'Create User' API Request

Flags respective to adduser command, useradd may differ.

POST /api/users
Request:
  username: string|required
  group: string|appears_in:groups   // -g
  groups: list[group]               // -G
  is_system: // -r / --system
  has_homedir: bool|default:true    // -m | -M
  homedir: path|default:null        // -d


### Sample Flow for Creating a User
[Enter a username]

<Which type of user is this?> [normal] [system]
<Can this user login?> [yes] [no]
<Set primary group?> [match username]  [select from existing] [custom]

Add 'Browse' button in Sites list

Now that there's a file manager, it'd be useful if we had a button directly in the site list (and/or editor) that takes us directly to the current Site's document root in the File Browser.

Validation errors don't show in site editor

When editing a site, submission fails when there are errors but the errors don't actually show up in the page so you have to check the network tab in devtools which isn't really ideal.

Need to update the form to show the first error for each input when submission fails. Alternatively could show each input's error combined in one big message component above the form, but individual errors are preferable.

Can't view folders/files unless www-data has permission

When you browse to somewhere like /root which doesn't have read permissions for www-data, it just throws a 500 and does nothing. If you try to load the page at a path without permission, it'll show a blank file list.

There are probably other ways around it (at least I hope so), but for now we could do with manually parsing output from the ls command running as sudo. Same goes for opening files, too.

Eventually paths should be checked for permissions before attempting to load anything within them. That'll have to happen when db users are linked to actual system users though - for now we assume superuser status.

Maintain symlink in sites-enabled relative to is_enabled

As it says on the tin. If and only if a Site's is_enabled flag is TRUE, we should automatically create a symlink to /etc/nginx/sites-available/some-site.conf in /etc/nginx/sites-enabled/.

If the Site later gets disabled, the symlink should be removed.

Server should not restart unless manually triggered somewhere, as such an action will affect all Sites on the server. In an ideal world, said action should use systemctl reload ... rather than restart as it's slightly less destructive.

Use CodeMirror for syntax highlighting

While the file viewer works, a plain <pre> element is a bit... plain. There are easier options for highlighting alone but at some point we'll also need to edit, so we might as well add CodeMirror in the first place to save replacing it later.

It also has the benefit of supporting different colour schemes and editing modes (Vim, emacs and sublime) which would be useful to implement eventually. Until the API supports editing files though, we can keep it read-only.

Form errors persist on resubmission

If there are errors when you save a site, errors for all fields are correctly displayed including the top "Could not save Site!" alert.

However when you fix the error (for field validation) or resubmit the form (for general errors when saving), the errors don't get cleared even if the latter submission is successful.

We could add frontend validation that deals with all the per-field errors, but we'd still need to deal with the top alert box in case of server errors causing the save to fail. It's probably easier to just reset the errors var in Vue before the ajax call is fired...

Add a way of setting home directories

Currently there's no way to create/set a home directory for a user when you're adding or updating it.

The field should be optional and, to save adding an extra switch, the respective useradd/usermod flags to enable/create the home directory should only be passed when the field isn't blank.

New Site form

Probably just one input for the site name.

Must create a new site - eventually in the database once API is available - and update the list to include the new entry. Could sort alphabetically or by ID, doesn't really matter. Alpha is probably best.

The domain could be set on the next screen once added, since it's not a certainty that a site will even have a primary domain.

Add ability to set the branch for source repos

Cloning the source repo works well when you're using master, but if you want to use any other branch you're out of luck.

Add some logic that gets a list of branches on the repo which can be shown in a dropdown to the right of the source repo input. For the list, use git ls-remote <repo> 'refs/heads/*' to just get branches. Could be nice to get tags too though with 'refs/tags/*'.

If branch is set, the site update handler should append -b <branch> to the clone command. We'll probably need to checkout before/after pulling when it already exists too, just in case the branch gets changed.

Update url when navigating between directories

Currently if you view a file in the file manager, then click the '<' button to return to the containing folder it will update the URL with the path of that folder.

This needs to apply when you're clicking through the file system, including when using the button to go up a level.

Stats bar should update periodically

CPU and RAM usage are prone to changing frequently, so if the stats bar never updates after loading the page then it won't really be of much use.

Update the axios call such that it re-runs every so often. Some details are likely to change more than others (CPU changing constantly vs disk once in a blue moon) but for now it's probably enough to just ping every 30 seconds or minute.

Travis is too slow

Most of the time it doesn't really matter, but when it comes to finalising a PR ready for merge it can be annoying waiting a minute or two for Travis to do its thing.

Luckily there are some things we can do to improve it:

• Disable XDebug
• Enable caching for composer packages
• Switch to running commands in parallel

Switch placeholder out for real system hostname

In the StatsBar Vue component, 'Carbon' should be replaced with the hostname of the, well, host.

If FQDN is availble, short version is preferred. FQDN could optionally be displayed in a popup.

For now, it would suffice to pass values directly to the Vue component from Laravel in the layouts.app Blade template.

Old file flickers before showing the current file

If you browse to and open a supported file in the File Manager, it displays fine.

If you then click the < button to go back to the containing folder and open a different file, you'll occasionally see the first file flash for a moment before it's replaced with the file you intended to see.

This is likely just a case of clearing the file content in the VueX store's state. This could be done either before loading a new file in the viewer, or before leaving the editor route.

`mpstat not found`

When running PHPUnit tests in Travis, it spits out an sh: 1: mpstat: not found error on 3 of the tests.

The tests still pass (maybe that's an issue in itself? Should we be requiring that mpstat is available in the code using it?) but it makes the output difficult to read, so it'd be nice to get this fixed.

Replace "Welcome to nginx!" Page

Currently when you visit a site that points to Servidor but hasn't been enabled, you're greeted with this standard Nginx page:

This is fine, but it's a bit naff. Replace it with something better, even if it's just in Vagrant's bootstrap script for now. That'll be used to build an installer later.

Guests must not be able to create Sites

According to tests/Feature/CreateSiteTest.php, guests can create sites. This is bad. VERY bad.

Test needs to be changed to GuestCanNotCreateSite with a separate test added to make sure it works for authed users (which can utilise the RequiresAuth trait added previously).

Naturally, the controller/routes will also need to be updated to fit this behaviour.

Check for MIME types before opening files

When you try to open an image or some other binary file, it dies with an encoding error.

A list of supported MIME types should be added and referenced when loading files. If the file exists and is readable, but its MIME isn't a supported type, we should return a 'Unsupported filetype' error.

Example Supported Filetypes

While we're just dumping the file contents into a <pre></pre> block, we can't really display images or anything other than text. There are a few varieties though, so it's worth making sure we have all of 'em. Just a couple examples are:

• text/csv
• text/css
• text/plain
• text/markdown
• application/x-sh
• application/json
• application/javascript
• text/javascript

Further Reference: Incomplete list of MIME types

Sites Create box should also filter list

When typing in the input on the Sites list, it should filter the list similar to how the System Users/Groups pages function.

It should start off as a Search icon, then once there are no results, change to the Plus icon.

For now, hitting enter should simply create a Site as before. Unless it's an exact match, then we could possibly have it redirect to the edit page for that Site instead.

Primary domain validation is in the wrong order

Validation for the primary domain field seems to run the FQDN check before the required rule, so when it's left empty the user is prompted that it should be an FQDN rather than it being required.

In addition, the FQDN validation message is not capitalised, so we need to apply a healthy dose of ucfirst() to it. See below:

Error prevents creation of new System Users and Groups

When you type in the name of a user/group that doesn't exist and click Add User/Group, an errors gets thrown in the console about users/groups keys (user.groups and group.users).

This in turn prevents creation of the user or group... Not ideal, really.

Path parts should be clickable

When the File Manager's current path is several levels deep (eg /var/servidor/.vagrant/machines/default/virtualbox) it can be a bit tiresome constantly clicking the up button to get back somewhere else.

If we add a computed property that spits out the path as an array (split on /), it should be pretty easy to link each individual part of the path to that point in the structure.

The hard part would be joining up the half-path for the link and ensuring leading/trailing slashes stay the same.

Handle errors properly when viewing files

Currently if you attempt to view a file you don't have permissions for, or perhaps doesn't exist, it throws a 500.

The 500 is caught fine in the frontend with the error displayed in place of the file contents, but it's not ideal. We need to be checking if the file exists and is readable before we do anything, or at the very least handle the exceptions so that we can return less cryptic error messages.

Server Templates

Need to have some concept of server templates for use in the Site Editor.

The template itself should be a Blade or Twig view, with placeholders where needed that can be used to "inject" values from the Site config.

Since the placeholders (and thus form fields in the Site Editor) will vary between templates, we'll also need some kind of Class/Interface system to define everything.

Templates should provide:

• Name of template file
• Method to get fields as array

Example:

class RedirectTemplate implements ServerTemplate
{
	protected function template()
	{
		return 'redirect.blade.php';
	}

	protected function fields()
	{
		return [
			'destination' => [
				'type' => 'text',
				'order' => 2,
				'rules' => 'required|url',
			],
			'type' => [
				'type' => 'dropdown',
				'order' => 1,
				'default' => 301,
				'options' => [
					301 => 'Permanent',
					302 => 'Temporary',
					307 => 'Permanent (Maintain POST)',
					308 => 'Temporary (Maintain POST)',
				],
			],
		];
	}
}

Show true system hostname in app layout

Instead of displaying the machine name, the StatsBar component is currently displaying 'host' information from the HTTP request. In many cases, this will simply be localhost, which is less than useful.

Replace request()->getHost() (added in #10) with something equivalent to the output of hostname or cat /etc/hostname on Linux/OSX.

The best solution would be a new GET /api/system/status API (with tests) which returns a simple key-value array. This could then be fetched to update all information in the StatsBar component, rather than only the hostname.

Colour the icons

Eventually it would be great to have an icon for each of the main file/MIME types to aid in distinguishing between them, but that could be a while off yet.

For now we should have folder icons one colour, with a different colour used for file icons.

Force php-cs-fixer

I already have php-cs-fixer configured to run locally when saving files, but if anybody else were to send a PR then it doesn't help us.

Add php-cs-fixer to the Travis CI config to make sure all future code is going to match the rules already defined in .php_cs.dist.

Can't edit user or group while editor is already open

Once you click on a User or Group to open it in the editor, the only way to edit a different one without reloading the page is by first clicking the Cancel button in the editor.

If no changes have been made in the editor, you should be able to click a different User/Group to edit it without closing the existing editor.

Missing tests for System User editing/creation

Multiple server block support

Rather than being restricted to a single server template, it should be possible for a user to add as many as needed.

This would allow for defining subdomains in the same place as the primary domain, eg for staging/testing versions.

Display text permissions instead of octal

775 works great and all, but it would be much easier to understand for a lot of people if it was represented as text, eg:

0600 -> rw-------
0664 -> rw-rw-r--
0775 -> rwxrwxr-x

If it doesn't clutter the UI too much, it could be useful to have a toggle switch somewhere so both representations can be supported.

Frontend doesn't redirect to login when session expires

Occasionally when attempting to reload the page, it'll redirect to the Dashboard which acts as if you're logged in but without displaying any user data.

To replicate, simply destroy and recreate the Vagrant VM (vagrant destroy && vagrant up). It might be sessions being invalidated, but there are likely other causes too.

Add ability to filter out system users/groups

Add a toggle input below the search field on both the Users and Groups System sections, with a label like 'Include system users' or 'Include system groups'.

Normal users/groups will generally have a UID/GID >= 1000. System U/GIDs will be below 1000.

There's a bit more to it than that, but for now a simple system_user = user.id < 1000 should suffice.

Add a "Pull" button

We can force a site's repo to get pulled again by changing and saving the site config, but this is far from ideal.

Add a button somewhere to request a git pull for a site. Will probably be better off on more of an "overview" page that summarises the site details rather than cluttering the editor too much.

Site Editor

Route

/sites/{$primary_domain} or /sites/{$id}

Fields

	# [My First Site]			[status]
	
	Primary Domain:  [my-first-site.com]

	Server Template: [Laravel    ▼]
					 [PHP		  ]
					 [Plain HTML  ]
					 [Redirect    ]

	[Template Fields]

Template Fields

• PHP
  • Docroot (relative to project root)
  • Clone URL
• Laravel (extends PHP)
  • Clone URL
  • Docroot (hidden, set to /public)
• Redirect
  • Redirect Type (Dropdown/switch: temporary or permanent)
  • Destination (url)

Server templates should be either Blade or Twig templates, probably saved in resources/views/server-tpl but they might be better in their own folder somewhere.

Can't update site without changing name

The unique:sites,name rule in Site validation is preventing updates to a site unless you also change the name.

Need to update this to have the current Site as an exception.

Clearing search doesn't work

Like creating users/groups, if you type something in that doesn't exist so that the placeholder is shown, the Clear Search button doesn't work.

The console warns that "Computed propertysearch was assigned to but has no setter", but who knows why that's gone missing. It used to work just fine...

Empty groups show Group Members section with no content

Either need to remove the section entirely when it's empty, or show a message in place of the user list:

Replace branch input with a dropdown

It'd be awesome if we could have some kind of dropdown menu that lists all the branches for the repo in Site Editor to avoid memorising them or typing them in. This consists of two parts:

Backend / API

We'll need a new endpoint that accepts the value from source_repo and returns a list of branches. The source repo should be validated using the same rules as when editing a Site.

To get the actual branches we can run git ls-remote --heads <repo_url> via exec(). We don't care about the commit SHAs, so everything before and including refs/heads/ can be stripped:

$ git ls-remote --heads https://github.com/dshoreman/servidor.git | sed 's^.*refs/heads/^^'
develop
master

Frontend

The source_branch input should be replaced with a SemanticUI Search Selection, which could also allow for setting a specific commit or tag in the future using the search field's value. To get data we'll need to add a listBranches or similar action to the Site VueX module, using axios to call the API.

If there are no branches it might be best to revert to a text input in case the repo hasn't been pushed to yet. The store action should be triggered both when the page loads and when the repo URL changes, otherwise we might end up using branches that don't exist on the repo we're saving.

Checklist

The tl;dr. Each one of these could be a separate PR.

• Add an API endpoint that takes a repo URL and returns its branches as an array
• Update the setSiteEditor mutation to load branches from the API endpoint
• Replace the branch input with a sui-dropdown component
• Make the dropdown filterable by typing partial branch names
• Ensure the branch list is updated when the source repo is changed
• Display a spinner when the branches list is being updated

FontAwesome icons are not loading

Instead of the icons defined in code, all are loading as the 'invalid' placeholder.

Project is using SemanticUI for CSS, with scripts from Semantic-UI-Vue (installed via npm) which is probably why it lacks FA.

May just need to add the FA script either to package.json, or layouts/app via CDN.

Create System 'dashboard'

Add a System dashboard with 'tiles' linking to each page under the System category (currently only users and groups).

• Create new single-action System\Dashboard controller
• Add new Link the /system route to the new controller
• Create the system.dashboardSystemDashboard view component extending layouts.app
• Add links to Users and Groups

Eventually it might be better to use the system layout with meaningful dashboard widgets, but for now there's not much to display.

Can't delete sites via frontend editor

Deletion works via the API, but there's no button for it in the frontend.

Add a new Delete button that triggers a DELETE /api/sites/{site.id} request through axios, and make sure VueX removes the affected site from the main state.sites array when complete.

Vertical scrollbar is always present

The default Semantic UI styles set the body height to 100%, but the 5 ems of margin-top applied to the .main container are pushing the height beyond the viewport.

Either put the margin somewhere else, or calc(100% - 5em) on the body to counter it.

Missing is_enabled switch in Site Editor

There's currently no way of toggling the enabled status of a Site within the UI.

Preferably use a toggle switch for this, to the right of a header. Whether it's added by the actual "Editing Foo" page header or instead in the site card to the left doesn't really matter. The page header will probably be easier though.

Site/Application Management

Won't be much good if a site is only a name and domain.

Add some dummy UI for adding config blocks to an nginx template.

Should also have spaces for setting other top-level server rules, along with separate sections for 'custom' blocks - both in the main template section and each individual config block.

Primary domain shouldn't be a URL

Validation in SiteController defines primary_domain to be a URL, but really we want a domain/FQDN. Laravel doesn't support that by default, so it's gonna need a custom rule.

Example of a custom rule here: https://stackoverflow.com/questions/47122338/laravel-eloquent-rule-for-domain-name

The primary_domain should be a plain FQDN because it'll be used for various things including Nginx configs where it won't have the https:// or any other protocol/path with it.

Saving a Site should create/update nginx

If a Site uses a Server Template (not all applications will have a web element) then we need to make sure the site is maintained as an Nginx configuration file in /etc/nginx/sites-available/site-name.conf.

Updates to the actual config file should preferably only happen when the user specifically triggers it by way of a Save button or similar.

Applications/Sites API

• Site model+migration (start with name+domain)
• Ability to add and list sites
  • Basic validation rules
  • Proper Request validation
• Ability to update sites
• Site deletion
  • Soft deletes? Maybe later...