components-web-app/api-components-bundle

Creates a flexible API for a website's structure, reusable components and common functionality.

Components Web App Template ⚠️ [WIP] ⚠️

Live Demo

• API (API Components Bundle)
• Front-end (Nuxt.js application with cwa-nuxt-module)

Introduction

This repository provides a starting template which consists of API Component Bundle for the API, and a Nuxt application configured to use cwa-nuxt-module for the application.

This setup will include CI integration with GitLab by utilising a bash script file so that it is easier to switch between different CI tools.

The integration will assume deployment to a Kubernetes cluster for the API which will also include a Helm chart.

The primary API stack is as follows:

• PHP application
• Nginx
• Varnish
• Vulcain

This setup will also include Mercure for the front-end application to subscribe to updated resources.

We also want testing to be implemented by default. This means we will have both PHPUnit and Behat configured for the API and Jest for the front-end application.

Sponsors

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Pascal Borreli 🤔 🚇

Antoine Bluchet 🐛

This project follows the all-contributors specification. Contributions of any kind welcome!

See also contributors to

• API Components Bundle

• CWA Nuxt Module

api-components-bundle's Issues

Dynamic Page Features

Implement features to allow for Dynamic Pages

• Page Data Provider service (get the page data based on the original route requestedby the front-end application)
• Component Resource Placeholder

User class serialization groups for different tasks

It seems prudent to address the serialization groups on the AbstractUser class. E.g. the oldPassword property only needs deserializing during a password update. Validation groups applied already as we use the form components to create the forms for the front-end to use. But serialization groups not applied yet. Maybe good to do with the context builder. TBD.

Security functionality to implement

• Login forms
• Mailer notifications for user updates (new user, updated password/username, user newly enabled)
• Password reset
• Create new User resource, persist password, secure behind API token to limit abuse and allow front-end to implement anti-bot checks
• config options to enable/disable some emails and define path to email twig templates
• change email address process with email address validation / confirmation
• email address verification, adjust database value. User entity listener
• option to disable user login if their email is not verified

Components can specify whether they must be explicitly permitted

You may have a component which can only be added to specific component groups. If there is a boolean on components whereby if the value is true the component can only be added to component groups that specifically allow it, then the user-interface becomes cleaner by not listing these components to be added to every component group, and the restriction will mean the component must be used properly.

Testing & Bug Fixes

We need to introduce tests to ensure code is working as expected and to prevent regression.

This includes unit and functional tests. We must also ensure that we get a good mutation score for the tests.

Target: 90% test coverage + 80% mutation score

^^ These targets are a little arbitrary at the moment due to a lack of experience writing tests. I'm not sure whether they will be achievable within budget for the current stage, but it is important to make a start. We can adjust these 1st stage targets and we progress and add a less important issue to increase coverage and mutation score at a later date if required.

• Implement exception best practices as unit tests are created Fixes #17

Docs

Docs need a lot more attention and work. Many features not documented, e.g. groups added to components and many options and guidance on each of the resources/objects.

Dynamic pages needs documenting too with examples. Dynamic content resources can be created and properties added which are components. A component position can refer to the property in the dynamic content and will overwrite an existing component in that location when outputted. Dynamic content are routable resources. Further functionality should be added with data transformers.

Add a /me endpoint for logged in user to fetch their object

This is required to populate a user on most front-end applications including the Nuxt auth module we are implementing in CWA Nuxt Module.

Custom Actions should handle data in other formats than json

We are already detecting the format type on custom actions (e.g. for forms). But currently we deserialize the submitted data using json_decode. We could easily deserialize the data using the detected format instead so that the formats can be handled in line with whatever we have configured our API to accept.

Create a form listener that can be wired in to simply persist the object to database

Often it will be needed that when a form is submitted, we simply need to persist that object to the database. We are doing this already for the user registrations and new email address form types.

I have seen this done before on other packages but cannot remember where.

So the config would be something like:

services:
    'app.listener.contact_form':
        class: Silverback\ApiComponentBundle\EventListener\Form\EntityPersister
        arguments:
            $formType: App\Form\ContactType
            $dataClass: App\Entity\ContactSubmission

Review of Collection component implementation

Full review required

Handling parent/child (nested) pages and routes

Sub routes may be nested. E.g. /about/contact and /about/history could use the same hero and tab navigation on the hero and the nested page changes.

This works on v1 but the structure and functionality of this needs to be worked out for v2. v1 is very messy and unclear how this functionality is really working.

Complete 'FileInterface' functionality

• Check and complete data transformations on components using FileInterface. Check the extended data is being outputted.
• Ideally add to API Platform metadata the output class of these resources automatically if not specified by the user so that the Data Transformers are run

Discuss usage of Message Handler

Possibly use Message Handler as described here for tasks we have custom actions for at the moment. Not urgent right now as we would need to add instructions on configuring RabbitMQ or something as the transport.
https://api-platform.com/docs/core/messenger/

Refresh token tests required

Minimally these are needed

• when I log in, I should receive an access-token in a cookie http+secure+samesite (<= should already exist) and a refresh-token should have been generated
• given I have a valid refresh-token when I log in with an expired access-token, I should receive an access-token and the refresh-token has been expired and a new refresh-token has been generated
• given I have an expired refresh-token when I log in with an expired access-token, I should get a 401
• given I have a valid refresh-token and I am authenticated when I log out, all my refresh-tokens should expire

Migrations storage configuration

Should look into storing doctrine migrations in configured Flysystem storage.

Senior dev to advise on location of assets for emails

Currently put assets within the views directory in the bundle. I feel this is wrong but was simple way to reference them and possibly allow css and image to be overridden?

I'm not sure, best practices advice required.

PageData classes should allow auto-generated routes

If a resource is added to the database that extends AbstractPageData it will require a PageTemplate to be set and if the Route is not set, it should be generated from a value from a 'getter'/property and slugified.

Test with strict Symfony version in GitHub workflows

This bundle should follow Symfony supported versions: 3.4 (LTS), 4.4 (LTS) and 5.0 (latest).

• Update composer.json require dependencies to support those versions
• Update .github/workflows/ci.yml to configure extra.symfony.require version and test for each supported version (using matrix)

Review of Form component implementation

Full review required.

Security functionality

We must re-introduce the functionality from version 1 for standard security features. This includes:

• Login (forms and functionality)
• Forgotten password process
• Registration functionality possibly?
• Change username/email with confirmation emails and notifications to previous email address when completed
• Change password with notification to users that the account password has been changed
• Usage of new Mailer functions to send emails

Implement best practices for exceptions

Some exceptions thrown are vague or inaccurate. With little experience of throwing the proper exception messages and how best to structure this, it'd be best for a more experienced developer to take a look through and quickly create exception classes with best-practices.

'Soft' Validation for drafts

If a resource that is being edited is a draft, validation should take place, however the data should still save in the database where possible.

The only exception to this should be if a value is NULL and the database will not allow this.

When a resource is being changed to 'published' or a 'publish date' is being set, the validation will run and prevent this change if it fails.

If a resource is in a 'published' state validation will prevent the resource from being saved.

Implement new Symfony Mailer

As all our applications will have user functionality, and therefore require emails, it makes sense to include a Symfony Mailer extension so that we have an easy to customise template that will be used for security procedures, but the main application can also use to send branded emails to customers or their own email (e.g. for contact form).

Respect OSS bundle best practices

• Migrate configuration from YAML to XML
• Create a route loader
• Create a Symfony recipe on https://github.com/symfony/recipes-contrib
• Replace DoctrineSubscriber and EntitySubscriber by Doctrine EntityListeners
• Remove useless travis.php.ini
• Use controler.service_arguments tag on Action services
• (optional) Use @author on each class or nowhere (configure php_cs.dist accordingly)

Route security implementation [A discussion]

I have been battling with my own thoughts on the best way to implement this. Please take a look at how the bundle configuration can currently be used to determine route security and perhaps we can discuss the flexibility of this for applications.

The results need to be able to be the following:

1. Collections only return routes for which the current user (or anonymous user) has access to.
2. Route resource items which are not authorised must return 401/403 (not just a 404 not found).

Any input and discussion is greatly appreciated.

Draft/Published Functionality

Any resource should allow to be linked with a 'draft' version. Drafts should be able to have a 'publish' date/time. Database time should be UTC, but front-end will set local time with full timestamp stating the timezone that it is being set in.

Draft versions should be created automatically as a resource is being modified.

When a draft component needs to be published, we want to copy the draft component into the original component so the original ID will stay the same.

Applies to:

• Components
• Component Locations

Other resources will simply edit. Perhaps apply this with a trait/interface in case we want to apply this to other resources in future?

This issue needs some thought and discussion. It is a large feature of this bundle.

Test email verify process

Process for verifying an email address not yet tested. Need to test this functionality works with behat scenarios.

Update login docs at the same time

PHP Infection Mutation Failing on Travis

Not failing locally, locally uses higher phpunit version - possibly the issue. Need to check.

Implement security features for API resources

In version 1 of ACB, some routes and components could be configured so that they are only available to users with specific roles. The ability to restrict routes and even which components/resources can be modified by which users needs to be addressed.

Perhaps we can primarily be using the API Platform security features once the main security firewall is passed by knowing the user is an admin? There are also some default serialization groups put into the context based on the logged in user's roles. The issue arises when we need to determine which individual resource in the database should be able to be accessed by a user. E.g. a navigation item.

We may even want to add a property to the component's output so that the front-end knows that it will need reloading if the user's login status is changed...

Big feature to TBD...

Passing querystring parameters to collection components (via route request??)

When a collection component needs to be filtered on SSR loading we need to be able to pass the querystring to the collection. Need to think about how to handle this where it will be able to support multiple collections on a page so we know which filter parameters we want to pass to each collection component.

ComponentPosition auto sort value

When adding a ComponentPosition it should automatically populate the sort value if not defined with the next number available sequentially.

On create or update, if the sort value is defined, the API should prevent a duplicate sort value by also adjusting all other resources sort values which would be affected. So the existing position sort value that is the same as the new/updated entity will increase and this will cascade to the next entity etc.

Refresh tokens

We should implement best practices with refresh tokens, no exposing them to the end user and securely regenerating new JWT tokens. Invalidate refresh tokens once used and create a new one, and remove on logout.

Symfony Best Practices

• Services should be given names and the classes as aliases.
• Deprecations should be addressed
• Double check auto-wiring. We should not be auto-wiring in a bundle

Feature: Allow user with draft access to modify published resource directly when a draft is available

By default the published feature means a user will automatically be editing a draft resource if available. It may be needed to make another change to the published resource. If we allow this using the 'published=true' querystring with the request then we would also need to ensure the 'publishedAt' date cannot be modified, possibly with a validator.

Use friends-of-behat components

composer require --dev friends-of-behat/mink friends-of-behat/mink-browserkit-driver friends-of-behat/mink-extension

behat/symfony2extension is replaced by friends-of-behat/symfony-extension.

Also, dama/doctrine-test-bundle will help to reset the database on each Behat scenario using doctrine transaction.

[Publishable] Add permission option to override global publishable permission (requires symfony/security)

Review & test User functionality

Some user actions should possibly not be actions and adhere to REST API standards for their endpoints.

Review required and testing of many user functionalities needed as well.

Component Groups should allow restricted components

A component group should be able to restrict which components are allowed within it. This is low priority because to begin with, web developers/trusted workers will be adding the components - so would not add a "Hero" to a navigation (for example).

Test with supported versions of PHP

PHP supported versions are: 7.2, 7.3 and 7.4. Update GitHub workflows to test with each of those versions to ensure the bundle is bootable of each of them

Uploadable: Private files

Need to add tests for using Amazon S3 with private files to check how it is handled. Perhaps a temporary token is generated which expires shortly afterwards... I hope! If not, then we need to ensure it acts in this way.

For the actual uploaded file (not imagine) we will be hitting an endpoint for the given resource. This means an application developer can hook into events to prevent access to the resource and therefore prevent access to the download....

Maybe address in alpha...

Timestamped fixes

• serialization groups to ensure they are always returned... or groups can be configured... implement same as publishable.
• hook into requests instead of doctrine events (purpose of API is to handle events). The helper can have a method available should people want to trigger events in commands etc. Similar to Uploadable

Collection data transformer should use sub-request.

Should probably use a sub-request for collections instead of current functionality. E.g.

function getCollection() {
        $subRequest = Request::create($path, $method, [], [], [], [], $body);
        try {
            return $this->kernel->handle($subRequest, HttpKernelInterface::SUB_REQUEST);
        } catch (\Exception $e) {
            // handle
        }
}

Files support rework

Currently there is basic support for handling files with this bundle using FileTrait and FilterInterface. This is limited for a number of reasons. We also use LiipImagine bundle for resizing images.

In v2 we should be using FlySystem v2 - it is in Alpha 3, but as this is a new bundle, and we will write many tests, this is the best option for longer term stability. To support the new Silverback\Files annotation the user must have a FlySystem adapter configured. We will NOT be using a bundle for this. The FlySystem implementation is very straight forward and another bundle would over-complicate this. It would also prevent us from supporting the latest version when released as the packages have to handle many more use-cases than this bundle. We can create a simple interface and tag where the application developer is able to have a factory to provide the FlySystem adapter.

We will not support ANY other implementation of file systems for the time being. FlySystem is flexible enough, handles local files as well as the ability to create adapters for any system. However for Silverback use cases, S3 and Swift object storage is supported and all we required other than local storage.

The downside is that LiipImagineBundle required OneUp FlySystem Bundle. However, implementing our own Resolver and Loader is trivial.

Another bundle option is VichUploaderBundle - while it is well documented and many people use it, the failing builds, especially in Symfony v5 are a concern. It has a number of features ACB v1 implemented in different ways such as v1 uses a metadata object to return extended information about image files. We also detect whether the files are supported by Imagine, generate thumbnails/image versions and return an array of image data objects for each image variation. For these reasons we will be implementing our own File handing system. We also do not need to be quite as flexible as VichUploaderBundle and want to implement a single file upload endpoint to handle all upload requests for API resources, rather than an endpoint for each resource as documented in API Platform.

The Files feature will begin to be re-worked in the next week and this issue is a point of reference for the work to be undertaken.

Configure gitattributes

/.gitattributes export-ignore
/.github export-ignore
/.gitignore export-ignore
/.scrutinizer.yml export-ignore
/Api Component Bundle Work Overview.docx export-ignore
/.php_cs.dist export-ignore
/docs export-ignore
/behat.yml.dist export-ignore
/infection.json.dist export-ignore
/phpunit.xml.dist export-ignore
/travis.php.ini export-ignore
/tests export-ignore
/features export-ignore

Configurable user management

It would be good to allow a bundle configuration of a security expression defining those that are able to manage other users.

Email address changed notifications

New notification email if email address changed with auto password reset link added. Otherwise the new user can change the user's password. May need to be a new backup password change key so that this cannot be overwritten by a hacker changing email and then doing a password reset request...

Change of plan. See PR.

Detect if resource persisted to database

The front-end application should be able to detect whether a component/resource is persisted to a database. It could be that a DataPopulator has added pseudo-resources to the output (perhaps some components created automatically using some other data). E.g. if there are a list of events and they need to appear in a navigation, a DataPopulator may add some navigation link resources to the output programatically. The front-end should not be able to do anything to modify this component/location.

Serialzation groups security vulnerability to resolve

vulnerability highlighted by @vincentchalamon - thank you.

• you have 2 entities linked by a ToMany association. using serialization groups, you just want the root resource and only the iris of the collection associations
• but I'm authenticated as admin, so the admin serialization group is automatically added to the related "normalization_context.groups"
• because the admin serialization group is also setter in the collection association entity, you'll get a collection of objects instead of a collection of iris
• to prevent it, it's recommended to use the Class:Action syntax for serialization groups instead of read or write basic serialization groups
• for instance: User:Read or User:Write, or even User:AdminRead and User:AdminWrite
  with this syntax, there is no risk to change the response structure by accident, and it respects more strictly the serialization groups process

Security of components and restricted pages

The security implementation around restricted pages and the components within the pages needs to be addressed.

• Routes will be configurable in Symfony security as to the users which can have access.
• A route fetched by an ID needs the same permissions.
• Page data resources can be configured by the app developer within the application with security policies as with any other API Platform resource
• Should we just guide that if a page needs to be restricted the user simply needs to add a 'dummy' page data resource and locate all the components within this using other methods to populate the page? Probably not.
• We would still need to check every component and which page data or route it resolves to being within and if it is restricted it should not be accessible by unauthorised users.
• Does this need to apply to a page template, component group or component location? (maybe rename to component locator...). Probably not as they hold no useful data other than strucutral data about the page.

...

• Components need to be restricted based on
  • The route they resolve to and if that is accessible for the current user given the Symfony security policy
  • The PageData resource it may be within and if this is loadable for the current user by the data provider. The most secure way to restrict resources in API platform is by ensuring the data provider does not return the item which is why we should check this way.
  • If the component is accessible via ANY route or PageData that the current user can access, then it is allowed to be loaded by the user.

Possibility to extract publishable feature into separate bundle

Thank you for the suggestion/idea @vincentchalamon

There may a demand for the publishable feature to be extracted and functionality enhanced in a separate bundle.

Features may include:

• simple boolean publication: is_published
• drafting publications (as implemented in ACB)
• startedAt / endedAt publication (with or without draft)
• etc.

This way it could be used with any Symfony project also using Doctrine to easily configure entities that should be publishable.

This feature is a big part of the ACB and would require a lot of work to be as flexible as required outside of this bundle. TBD