stefandoorn/sitemap-plugin

Sitemap Plugin for Sylius eCommerce platform

Sylius Sitemap Plugin

Big thanks

Goes out to the Sylius team. The core code of this plugin is created by the Sylius team. Unfortunately it got removed from the Sylius core. Luckily the Sylius team approved the extraction to a separate bundle.

Features

• Creates a sitemap index file to point to sub sitemap files per type of data
• Default providers: taxons, products & static content (homepage & contact)
• Easily add your own providers
• Product provider supports locales (hreflang) & is channel aware
• Taxon provider supports locales (hreflang)

Installation

1. Run composer require stefandoorn/sitemap-plugin.
2. Add to app/config/bundles.php:

  SitemapPlugin\SitemapPlugin::class => ['all' => true],

3. Add to app/config/packages/_sylius.yaml:

    - { resource: "@SitemapPlugin/Resources/config/config.yaml" }

4. Add to app/config/routes.yaml:

sylius_sitemap:
    resource: "@SitemapPlugin/Resources/config/routing.yml"

5. Add to app/config/packages/sylius_sitemap.yaml:

   • Default configuration

6. Forcing HTTPS on Generated URLs, see Symfony Docu. In console commands, where there is no HTTP request, URLs use http by default. You can change this globally with these configuration parameters:

# config/services.yaml
parameters:
    router.request_context.scheme: 'https'

Usage

Generate your sitemap from the CLI:

 $ bin/console sylius:sitemap:generate

Add this command to your cronjob to regularly generate a new sitemap, e.g. once a day.

If you only want to generate the sitemap for a specific channel, use:

 $ bin/console sylius:sitemap:generate --channel=US_WEB

The plugin defines three default URI's:

• sitemap.xml: redirects to sitemap_index.xml
• sitemap_index.xml: renders the sitemap index file (with links to the provider xml files)

Next to this, each provider registeres it's own URI. Take a look in the sitemap index file for the correct URI's.

Default Configuration

Get a full list of configuration: bin/console config:dump-reference sitemap

sitemap:
    providers:
        products: true
        taxons: true
        static: true
    template:             '@SitemapPlugin/show.xml.twig'
    index_template:       '@SitemapPlugin/index.xml.twig'
    exclude_taxon_root:   true
    hreflang:             true
    images:               true
    static_routes:
        - { route: sylius_shop_homepage, parameters: [], locales: [] }
        - { route: sylius_shop_contact_request, parameters: [], locales: [] }

The request context is also important for generating the URLs inside the sitemap:

• The hostname is defined per channel, if nothing set it defaults to localhost
• Other request context settings can be adjusted as mentioned in the Symfony docs

Default storage

By default the sitemaps will be saved in %kernel.root_dir%/var/sitemap. You can change this setting by adjusting the parameter sylius.sitemap.path.

Feature switches

• providers: Enable/disable certain providers to be included in the sitemap. Defaults are true.
• exclude_taxon_root: Often you don't want to include the root of your taxon tree as it has a generic name as 'products'.
• hreflang: Whether to generate alternative URL versions for each locale. Defaults to true. Background: https://support.google.com/webmasters/answer/189077?hl=en.
• images: Whether to add images to URL output in case the provider adds them. Defaults to true. Background: https://support.google.com/webmasters/answer/178636?hl=en.

Default providers

• Products
• Taxons
• Static content (homepage & contact)

Add own provider

• Register & tag your provider service with sylius.sitemap_provider
• Let your provider implement UrlProviderInterface
• Use one of the default providers as an example to implement code

sitemap-plugin's Issues

Example of how to use parameters

Hi, thanks for the plugin, all looks great except can't quite work out how to use the parameters on static routes effectively.

We have, for example, a single route for static pages that takes a parameter "page" and accepts several strings in order to generate some static content pages... imagine "/terms , /about , /info", those kinds of things. Could you please give some pointers on how would it be possible to specify in the configuration that these pages should be added to the sitemap?
The closest I've got to it working with the configuration is something like

parameters:
    - ['page', ['terms', 'about', 'info']]

But this gives:

InvalidConfigurationException
Unrecognized options "0, 1" under "sylius_sitemap.static_routes.0.parameters.0"

Thanks!

Performance improvements

Release v0.1

Can't change providers option

When trying to set taxons provider to false I get an exception:

InvalidConfigurationException
Unrecognized option "providers" under "sylius_sitemap"

My config.yml:

sitemap:
    providers:
        taxons: false

Change storage folder for sitemaps

Should probably be put in var/storage/sylius-sitemap or something to keep of with conventions

Originally posted by @loevgaard in #99

Problem on install with Sylius 1.10

composer require stefandoorn/sitemap-plugin

Executing script cache:clear [KO]
[KO]
Script cache:clear returned with error code 255
!! PHP Fatal error: Uncaught ArgumentCountError: Too few arguments to function Symfony\Component\Config\Definition\Builder\TreeBuilder::__construct(), 0 passed in /var/vendor/stefandoorn/sitemap-plugin/src/DependencyInjection/Configuration.php on line 18 and at least 1 expected in /var/vendor/symfony/config/Definition/Builder/TreeBuilder.php:26
!! Stack trace:
!! #0 /var/vendor/stefandoorn/sitemap-plugin/src/DependencyInjection/Configuration.php(18): Symfony\Component\Config\Definition\Builder\TreeBuilder->__construct()
!! #1 /var/vendor/symfony/config/Definition/Processor.php(50): SitemapPlugin\DependencyInjection\Configuration->getConfigTreeBuilder()
!! #2 /var/vendor/symfony/dependency-injection/Extension/Extension.php(111): Symfony\Component\Config\Definition\Processor->processConfiguration()
!! #3 /var/vendor/stefandoorn/sitemap-plugin/src/DependencyInjection/Si in /var/vendor/symfony/config/Definition/Builder/TreeBuilder.php on line 26
!!
Script @auto-scripts was called via post-update-cmd

Installation failed, reverting ./composer.json and ./composer.lock to their original content.

Only support absolute URL's in sitemap

Generate sitemaps from CLI

Especially on bigger sitemaps, it makes more sense to generate it and store it in the public folder instead of generating it on each request. As a backup real-time generation could still be an option, but it might be something user can configure.

Proposal:

• CLI command to generate sitemap index and store it on /sitemap_index.xml
• CLI command to generate specific sitemap and store it on /sitemap/{provider}.xml
• CLI command to generate all the above at once

Optional:

• Allow default routing to still work with on the fly generation, which can be disabled/enabled via a setting
• If used, call the generator similar to the CLI would do

Investigate / discuss:

• To store files in the public root, it might need too much rights. Might be better to store in sitemap/index.xml and let the default route sitemap_index.xml perform a redirect.
• By default the CLI isn't aware of the request context, so https://symfony.com/doc/current/console/request_context.html should be applied. Find out whether that might be difficult for certain users. Might be needed to be overridable for this specific plugin.

Enable/disable providers based on config settings

Should be possible e.g. to disable the taxon provider if wanted. Switch service config loading via config settings. Default to true, to not introduce BC.

customer can see the sitemap on the website

like this:
https://www.reiss.com/rw/sitemap/

Hreflang support is incorrect

Each URL version should be present at top level, defining all the other languages. Not just once an URL with the alternatives, as the bundle generates now. Suggestion is to keep the providers as is, would make them very complicated else. Best is to handle this on the output. We still only need an URL (always one atleast) with the alternatives from a provider to make the XML valid according to Google.

https://support.google.com/webmasters/answer/189077?hl=en

Extend configuration tests

Drop PHP 5.6 support

As soon as Sylius hits v1 or v1 RC to keep up with. In there PHP >= 7.1 will be supported.

Rename to follow plugin standards

Add support for locales (URLs)

Remove defaults from services definition

This is default in Symfony and can therefore be removed

Originally posted by @loevgaard in #99

update recipes

Hello,

You should update the recipies with the good config file name :)

I stuck for 1 hour to undersntand it is missing the "a" in config.yaml

https://github.com/symfony/recipes-contrib/edit/master/stefandoorn/sitemap-plugin/1.0/config/packages/sitemap_plugin.yaml

AND Update Readme with good command

It is

$ php bin/console sylius:sitemap:generate

not with "-index" in the alpha5 for me.

Also mention about interface renaming in UPGRADE

Not only about models, also mention interfaces.

Upgrade test infrastructure to new Symfony directory structure

Before we can bump the minimum Sylius version to e.g. 1.8 (latest supported version) the whole test infrastructure needs to be updated to the latest version similar to PluginSkeleton. Current version is e.g. using the old Kernel.

See which models don't need setters

It might be good to move away from model setters where possible, in case the data can be provided via the constructor. It improves immutability, as long as we keep extensibility in the implementation layer.

• Identify models / setters
• Adjust models

Sitemap shows disabled products

Idea: abstract url generator or use absolute URLs

sitemap-plugin/src/Provider/ProductUrlProvider.php

Lines 143 to 146 in 3858e2c

	$location = $this->router->generate('sylius_shop_product_show', [
	'slug' => $translation->getSlug(),
	'_locale' => $translation->getLocale(),
	]);

Using the UrlGeneratorInterface::ABSOLUTE_URL option will ensure the right protocol (http or https) is used.

An alternative might be to abstract the code to generate an URL to a generator class that users could overwrite/replace.

Add support for images in sitemap

#51 & #61

• Make sure paths are browser URL's (cache)
• Proper DI in ProductToImageSitemapArrayConverter
• Make image support optional (default = on)
• Support localisation (optional)
• Support product provider
• Support taxon provider
• Add specs for ProductImagesToSitemapImagesCollectionGenerator

Only support XML out of the box?

Why localhost as domain ?

Hello,

I am doing the generation of the sitemap with a cron every days but the domain is 'localhost' :
<sitemapindex> <sitemap> <loc>http://localhost/sitemap/products.xml</loc> </sitemap> <sitemap> <loc>http://localhost/sitemap/taxons.xml</loc> </sitemap> <sitemap> <loc>http://localhost/sitemap/static.xml</loc> </sitemap> </sitemapindex>
What am i doing wrong? I would like 'https://mydomain.com/sitemap/...'

Move to final classes

Incompatibility with Symfony 5

Hi!
It's impossible to use this plugin with Symfony 5.2. The issue is due to the use of the service 'templating' that is no more available in Symfony 5 (Check for example Sylius upgrade guide).

Support Symfony 3.3

Adding locales to static routes causes exception

Config.yml:

sitemap:
    providers:
        taxons: false
    static_routes:
        - { route: sylius_shop_homepage }
        - { route: sylius_shop_login, locales: ["sv", "en"] }

InvalidConfigurationException
Unrecognized options "0, 1" under "sylius_sitemap.static_routes.1.locales"

Probably related Sylius/Sylius#8187

Fix packagist version badge

Priority for static routes

Hi!

Thanks for this very useful plugin!

While setting up my static pages, I realized that the default priority for this provider is 0.3. That seemed a bit low to me, especially for instance in the case of a home page.

Would it be possible to leave 0.3 as a default and allow customizing the priority per route in the configuration?

Thanks!

Split big sitemaps

According to https://www.sitemaps.org/faq.html:

Sitemaps should be no larger than 50MB (52,428,800 bytes) and can contain a maximum of 50,000 URLs. These limits help to ensure that your web server does not get bogged down serving very large files.

It would be good that sitemaps have a default of maximum 50.000 items (configurable) after which they'll be split up. The first file of a specific provider should have the default name (as now) and the consecutive files should have a follow up number appended. Also the sitemap index should point to the proper files.

E.g.: sitemap/products.xml & sitemap/products_2.xml.

Alternative:

sitemap_index.xml refers to sitemap/products.xml which itself is a sitemap index again which points to sitemap/products/1.xml & sitemap/products/2.xml. Not sure though that is allowed, I guess there is a maximum inheritance to index files.

Relates to #78.

Use channel hostname in url generated in sitemap file

When the file is generated, the hostname used in the absolute URL is not provided by the channel.

Ex: in my channel chan1, the hostname is set to shop.local. In the generated file var/sitemaps/chan1/products.xml all URLs have localhost hostname.

Make sure URL's can only have one alternative for each locale provided

Currently it's possible to provide multiple alternative URL's with the same locale, which I think should not be possible.

CLI Generator: separate builders from provider collections (also for index)

Handle writing and reading of sitemaps via streams

Issue with routing when we make a mistake in our own app

Hi!

In your routing you have:

sylius_sitemap_providers:
    resource: .
    type: sitemap

This causes a problem when we make a mistake in our own routing. If so, it returns always this error:

Exception thrown when handling an exception (Symfony\Component\Config\Exception\LoaderLoadException: Do not add the "sitemap" loader twice in . (which is being imported from "/sylius/vendor/stefandoorn/sitemap-plugin/src/Resources/config/routing.yml"). Make sure there is a loader supporting the "sitemap" type.)

If we comment out the route configuration above, we get the correct error: (as example, in my case)

Exception thrown when handling an exception (Symfony\Component\Config\Exception\LoaderLoadException: The routing file "/sylius/config/routes.yaml" must not specify both the "resource" key and the "path" key for "app_custom". Choose between an import and a route definition in /sylius/config/routes.yaml (which is loaded in resource "/sylius/config/routes.yaml").)

And this is the same issue everytime we make an error in our config.
This is kind of very annoying actually.

Can we try to find a solution to avoid this behavior?
What do you think?

Thanks!

How to reproduce

Of course with the sitemap bundle loaded.

In your config/routes.yaml, add these lines:

example:
    resource: "toto.yaml"

Do not create the toto.yaml file.

Reload and you'll get the error …Do not add the "sitemap" loader twice in….

Now, comment the lines in the routing of the bundle like written above in the description.
Reload and you get this error: …The file "toto.yaml" does not exist…

This is much better don't you think?
Thanks!

Symfony 3.2 deprecation

User Deprecated: The "sylius.sitemap_builder" service is private, getting it from the container is deprecated since Symfony 3.2 and will fail in 4.0. You should either make the service public, or stop using the container directly and use dependency injection instead.

I believe caused by $providers = $this->container->get('sylius.sitemap_builder')->getProviders(); at row 41 in SitemapLoader.php.

I suggest us to inject the sylius.sitemap_builder service into the sylius.sitemap.routing_loader service, no need to have it public that way.

Accept custom providers in the configuration

Allow to enable/disable custom providers in the config sitemap.providers. Only core providers can be enabled/disabled now.

InvalidConfigurationException

Unrecognized option "cms_sections" under "sylius_sitemap.providers". Available options are "products", "static", "taxons".

It will be useful for plugins which works with the sitemap plugin. E.g. https://github.com/BitBagCommerce/SyliusCmsPlugin/blob/master/doc/sitemap.md

CLI Generator: sitemap renderers should return filenames, which index builder should use

How to use Redis to save sitemap files ?

My Sylius is installed on multiple servers and I cannot run the command on each web server.

Who to configure the plugin to use Redis for storage?

how to generate the sitemap.xml

Use generators in providers

CLI Generator: Split into services

setChangeFrequency casts to string

Should we cast it on setting, or should we cast it on getting (or parsing)?

See

sitemap-plugin/src/Model/Url.php

Line 63 in 673fcd6

public function setChangeFrequency(ChangeFrequency $changeFrequency): void

Upgrade lakion/api-test-case to 2 or higher for sylius 1.1.3

So, the fact that I can't update to sylius 1.1.3 if I use your plugin. If it's easy to do - do it please. Thank in advance!

Add support for channels (filter products)

Sitemap should only show the products that are applicable for that channel (based on URL). For taxons there is no channel awareness it seems, but for products there is.

Model setters with optional values (nullable)

I think we should get rid of this kind of setter parameters:

  public function setter(?string value): void;

If such setters still exist after #92, we might make it non-nullable. Either provide a string, or not use the setter at all in the implementation layer.

Pass parameter to static route

Hi, I'm using bitbag CMS plugin and just want to pass slug parameter to the static route bitbag_sylius_cms_plugin_shop_page_show in your plugin. But still get "unrecognized option slug". Any hint how to properly pass slug to bitbag cms route in yml sitemap config?

Thank you!

Do not make controllers public

You should tag controller with controller.service_arguments: https://symfony.com/doc/current/service_container/3.3-di-changes.html

Originally posted by @loevgaard in #99