cdeleeuwe/netlify-plugin-submit-sitemap

Automatically submit your sitemap after production build

netlify-plugin-submit-sitemap's Issues

Show the result in the deploy summary

The final result:

netlify-plugin-submit-sitemap/index.js

Line 41 in 7d80db4

console.log(`\n \u2713 DONE! Sitemap submitted succesfully to ${provider}\n`);

could be shown in the deploy summary by using utils.status.show() (documentation here):

utils.status.show({ summary: `Sitemap submitted successfully to ${provider}` })

Plugin is not installed when using netlify configuration file

Hello.

So, I've setup submit-sitemap to automate the task of sending Google and Yandex my website's sitemap.

Here is how I configured netlify.toml:

# It is supposed to run only in production context
[[context.production.plugin]]
package = "netlify-plugin-submit-sitemap"
    [context.production.plugin.inputs]

    # Time in second not to submit after succesful submission
    ignorePeriod = 0

    # Which providers should receive the sitemap - Currently supporting only
    # Google and Yandex.
    providers = ["google","yandex"]

I then installed set the plugin as a dependency using npm install -D netlify-plugin-submit-sitemap code.

The site is built with no error, but I noticed there is no mention of submit-sitemap in the Deploy log. The section Netlify Build in the log shows:

❯ Version
11:29:33 PM:   @netlify/build 27.4.2
11:29:33 PM: ​
11:29:33 PM: ❯ Flags
11:29:33 PM:   baseRelDir: true
11:29:33 PM:   buildId: <not sure I can share>
11:29:33 PM:   deployId: <not sure I can share>
11:29:33 PM: ​
11:29:33 PM: ❯ Current directory
11:29:33 PM:   /opt/build/repo
11:29:33 PM: ​
11:29:33 PM: ❯ Config file
11:29:33 PM:   /opt/build/repo/netlify.toml
11:29:33 PM: ​
11:29:33 PM: ❯ Context
11:29:33 PM:   production

Installing the plugin by using Netlify UI, this same section appears differnt in the log:

11:29:33 PM: ❯ Version
11:29:33 PM:   @netlify/build 27.4.2
11:29:33 PM: ​ 
11:29:33 PM: ❯ Flags
11:29:33 PM:   baseRelDir: true
11:29:33 PM:   buildId: <not sure I can share>
11:29:33 PM:   deployId: <not sure I can share>
11:29:33 PM: ​
11:29:33 PM: ❯ Current directory
11:29:33 PM:   /opt/build/repo
11:29:33 PM: ​
11:29:33 PM: ❯ Config file
11:29:33 PM:   /opt/build/repo/netlify.toml
11:29:33 PM: ​
11:29:33 PM: ❯ Context
11:29:33 PM:   production
11:29:33 PM: ​
11:29:33 PM: ❯ Loading plugins
11:29:33 PM:    - [email protected] from Netlify app and package.json

So I think only installing through the UI did work. What am I doing wrong?

Add Yandex support

One user tried to use Yandex as a provider. However, at the moment, only Google and Bing are supported. Would it be possible to add support for Yandex? Thanks!

Windows support

curl is not guaranteed to be available on a Windows machine.

Build plugins can be run locally with the Netlify CLI, so they can run on Windows.

The HTTP request could instead be performed using node-fetch, got or similar libraries. This would ensure it works on Windows.

What do you think?

"Invalid date" error with Google Search Console

When I put generated sitemap, google search console show sitemap contains error.
Error is "Invalid date".

Could not import plugin: Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'data-uri-to-buffer'

Just got the above error when trying to deploy my site.
Here's the more detailed log from Netlify:

11:25:04 PM: ────────────────────────────────────────────────────────────────
11:25:04 PM:   Plugin "netlify-plugin-submit-sitemap" internal error         
11:25:04 PM: ────────────────────────────────────────────────────────────────
11:25:04 PM: ​
11:25:04 PM:   Error message
11:25:04 PM:   Could not import plugin:
11:25:04 PM:   Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'data-uri-to-buffer' imported from /opt/build/repo/node_modules/node-fetch/src/index.js
11:25:04 PM: ​
11:25:04 PM:   Plugin details
11:25:04 PM:   Package:        netlify-plugin-submit-sitemap
11:25:04 PM:   Version:        0.3.0
11:25:04 PM:   Repository:     git+https://github.com/cdeleeuwe/netlify-plugin-submit-sitemap.git
11:25:04 PM:   npm link:       https://www.npmjs.com/package/netlify-plugin-submit-sitemap
11:25:04 PM:   Report issues:  https://github.com/cdeleeuwe/netlify-plugin-submit-sitemap/issues
11:25:04 PM: ​
11:25:04 PM:   Error location
11:25:04 PM:   While loading "netlify-plugin-submit-sitemap" from netlify.toml and package.json
11:25:04 PM:       at new NodeError (node:internal/errors:371:5)
11:25:04 PM:       at packageResolve (node:internal/modules/esm/resolve:932:9)
11:25:04 PM:       at moduleResolve (node:internal/modules/esm/resolve:978:18)
11:25:04 PM:       at defaultResolve (node:internal/modules/esm/resolve:1080:11)
11:25:04 PM:       at ESMLoader.resolve (node:internal/modules/esm/loader:530:30)
11:25:04 PM:       at ESMLoader.getModuleJob (node:internal/modules/esm/loader:251:18)
11:25:04 PM:       at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:40)
11:25:04 PM:       at link (node:internal/modules/esm/module_job:78:36)
11:25:04 PM: ​
11:25:04 PM:   Error properties
11:25:04 PM:   { code: 'ERR_MODULE_NOT_FOUND' }
11:25:04 PM: ​
11:25:04 PM:   Resolved config
11:25:04 PM:   build:
11:25:04 PM:     command: npm run build
11:25:04 PM:     commandOrigin: config
11:25:04 PM:     environment:
11:25:04 PM:       - GOOGLE_CLOUD_PLATFORM_API_KEY
11:25:04 PM:       - GOOGLE_SPREADSHEET_ID
11:25:04 PM:       - NODE_VERSION
11:25:04 PM:       - ONEGRAPH_AUTHLIFY_TOKEN
11:25:04 PM:     processing:
11:25:04 PM:       skip_processing: true
11:25:04 PM:     publish: /opt/build/repo/out
11:25:04 PM:     publishOrigin: config
11:25:04 PM:   functionsDirectory: /opt/build/repo/netlify/functions
11:25:04 PM:   plugins:
11:25:04 PM:     - inputs: {}
11:25:04 PM:       origin: config
11:25:04 PM:       package: '@netlify/plugin-lighthouse'
11:25:04 PM:     - inputs: {}
11:25:04 PM:       origin: config
11:25:04 PM:       package: netlify-plugin-submit-sitemap

Support for DuckDuckGo

Hello, I'd use this if support for DuckDuckGo were to be added. To be clear, I don't know if DuckDuckGo has a similar process/system, so if they don't please let me know and close this out. Thanks.

Add error handling for invalid `sitemapPath`

When sitemapPath is not a valid URL, the following line throws:

netlify-plugin-submit-sitemap/index.js

Line 58 in 6257766

const sitemapUrl = (new url.URL(sitemapPath, baseUrl)).href;

A try/cache/utils.build.failBuild() block should be added to handle this and report it as a user error, instead of a plugin error.

Question about frequency

Hi There,

I would love to use this plugin, but im concerned about the frequency of updates. We do 50 - 100 builds a day on production, as content is being updated all the time. Does this plugin have the ability to limit when the sitemaps are submitted? Or how many times in a set time period they are submitted?

Thanks,

-Josh-

Runing the plugin from a CI environment falsely skips submitting sitemap

When deploying the site from CI environment other than netlify(eg. github workflow), constants.IS_LOCAL returns true when the plugin runs causing sitemap submission to be skipped.

Support for Kagi

It would be wonderful if the sitemap could be submitted to Kagi, too.

Microsoft Bing drops anonymous sitemap submission due to spam issues

For more information, you can visit this link

Error handling

Errors should be using either the utils.build.failPlugin() or utils.build.failBuild() method (documentation here, instead of console.log() in:

netlify-plugin-submit-sitemap/index.js

Line 37 in 7d80db4

console.log(`\n \u274c ERROR, was not able to submit sitemap to ${provider}`);

And:

netlify-plugin-submit-sitemap/index.js

Line 26 in 7d80db4

console.error(`Provider ${provider} not found!`);

This ensures that errors are correctly shown to users in the Netlify app.

Sitemap is not live when it's pinged to the services

In my output the ping to google/yandex/etc. is done before the site is put live. Because there is post processing involved the resulting site is "live" 6 minutes after the ping which in some cases is also after the Bots react to the ping.

Is there any way to configure a sleep period or set the plugin to run after the "live" mark? We are submitting the link to the resulting sitemap, not the sitemap content itself?

Services are pinged for local builds as well

The isProduction check should also look for the NETLIFY env variable. In local builds, this is undefined. The full check should be if (NETLIFY && CONTEXT == 'production')

[ERROR] "sh" is not a valid Prism.js language for eleventy-plugin-syntaxhighlight

Everytime I tried to build & deploy an eleventy blog app with the submi-sitemap plugin, this error happened:

4:03:18 PM: src/main.js → js/min.js...
4:03:19 PM: created js/min.js in 530ms
4:03:21 PM: Language does not exist: sh
4:03:21 PM: Problem writing Eleventy templates: (more in DEBUG output)
4:03:21 PM: > Having trouble rendering liquid (and markdown) template ./.netlify/plugins/node_modules/node-fetch/README.md
4:03:21 PM: `TemplateContentRenderError` was thrown
4:03:21 PM: > "sh" is not a valid Prism.js language for eleventy-plugin-syntaxhighlight
4:03:21 PM: `Error` was thrown:
4:03:21 PM:     Error: "sh" is not a valid Prism.js language for eleventy-plugin-syntaxhighlight
4:03:21 PM:         at module.exports (/opt/build/repo/node_modules/@11ty/eleventy-plugin-syntaxhighlight/src/PrismLoader.js:11:11)
4:03:21 PM:         at Object.highlight (/opt/build/repo/node_modules/@11ty/eleventy-plugin-syntaxhighlight/src/markdownSyntaxHighlightOptions.js:21:35)
4:03:21 PM:         at Object.default_rules.fence (/opt/build/repo/node_modules/markdown-it/lib/renderer.js:50:27)
4:03:21 PM:         at Renderer.render (/opt/build/repo/node_modules/markdown-it/lib/renderer.js:326:38)
4:03:21 PM:         at MarkdownIt.render (/opt/build/repo/node_modules/markdown-it/lib/index.js:543:24)
4:03:21 PM:         at /opt/build/repo/node_modules/@11ty/eleventy/src/Engines/Markdown.js:72:38
4:03:21 PM:         at processTicksAndRejections (internal/process/task_queues.js:93:5)
4:03:21 PM:         at async Template.render (/opt/build/repo/node_modules/@11ty/eleventy/src/TemplateContent.js:199:22)
4:03:21 PM:         at async Template._getContent (/opt/build/repo/node_modules/@11ty/eleventy/src/Template.js:547:12)
4:03:21 PM:         at async Template.getTemplateMapContent (/opt/build/repo/node_modules/@11ty/eleventy/src/Template.js:732:19)
4:03:21 PM: Copied 59 files / Wrote 0 files in 0.42 seconds (v0.11.0)
4:03:21 PM: npm ERR! code ELIFECYCLE
4:03:21 PM: npm ERR! errno 1
4:03:21 PM: npm ERR! [email protected] build: `npm run js-build && eleventy && npm run test`
4:03:21 PM: npm ERR! Exit status 1
4:03:21 PM: npm ERR!
4:03:21 PM: npm ERR! Failed at the [email protected] build script.
4:03:21 PM: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
4:03:21 PM: npm ERR! A complete log of this run can be found in:
4:03:21 PM: npm ERR!     /opt/buildhome/.npm/_logs/2020-12-19T08_03_21_227Z-debug.log

Avoid deprecated `url.resolve()`

url.resolve() is deprecated since Node 11.

const sitemapUrl = url.resolve(baseUrl, sitemapPath);

Since any new major release might remove support for url.resolve(), should it be converted to the WHATWG URL API as recommended by Node.js?

const sitemapUrl = (new url.URL(sitemapPath, baseUrl)).href

Run HTTP requests in parallel

The HTTP requests are currently run serially:

for (const provider of providers) {
  await submitToProvider({ utils, provider, sitemapUrl });
}

Would it work to run them in parallel instead? This would be faster:

await Promise.all(providers.map(provider => submitToProvider({ utils, provider, sitemapUrl })))

What do you think?

Support for Mojeek

It would be great if the sitemap could automatically be submitted to Mojeek, too!

Installing as plugin on Netlify but no way to set sitemap filename

Hello,

I wanted to use this Plugin directly from Netlify. I am building my Site with Astrojs and have a package which generates 2 sitemap files. sitemap-index.xml and sitemap-0.xml.
I see that I could do this via file based, but is it not possible to configure this on Netlify via Plugin?

Allow `sitemapPath` to omit `https://`.

Some users are passing the following values to sitemapPath: activepure-biz.netlify.app and www.activepure.biz . This fails becomes it lacks the scheme (https://).

Would it be a good idea to prepend a default scheme when the scheme is missing?