bartholomej/svelte-sitemap

Sitemap generator for SvelteKit. Small helper which scans your SvelteKit routes and generates static sitemap.xml

Svelte sitemap.xml generator

Small helper which scans your Svelte routes and generates sitemap.xml

• Designed for SvelteKit adapter-static with prerender option (SSG)
• TypeScript, JavaScript, CLI version
• Useful options for customizing your sitemap
• Support for submiting sitemap to Google Search Console
• Support for Google sitemap index. Useful for large sites (more than 50K pages)
• Also compatible with Vercel hosting
• Workaround for this official SvelteKit issue

Install

npm install svelte-sitemap --save-dev
# yarn add svelte-sitemap --dev

Usage

Use this library as a postbuild hook in your package.json file.

File: package.json

{
  "name": "my-awesome-project",
  "scripts": {
    "postbuild": "npx svelte-sitemap --domain https://myawesomedomain.com"
  }
}

It scans your routes in build/ folder and generates build/sitemap.xml file.

Alternative usage: TypeScript or JavaScript method

Sometimes it can be useful to call the script directly from JavaScript or TypeScript. Of course there is also this option, but in most cases you will need the CLI method as a postbuild hook.

File my-script.js:

import { createSitemap } from 'svelte-sitemap/src/index.js';

createSitemap('https://example.com', { debug: true });

And now you can run your script like this: node my-script.js

⚙️ Options

Option	Description	Default	Example
--domain, -d	Use your domain [required]	-	-d https://mydomain.com
--out-dir, -o	Set custom build folder	build	-o dist
--ignore, -i	Ignore files or folders	[]	-i '**/admin/**' -i 'my-secret-page'
--trailing-slashes, -t	Add trailing slashes	false	--trailing-slashes
--reset-time, -r	Set lastModified time to now	false	-r
--change-freq, -c	Set change frequency Option	-	--change-freq daily
--help, -h	Display this usage info	-	-h
--version, -v	Show version	-	-v
--debug	Show some useful logs	-	--debug

🙋 FAQ

How to exclude directory?

Let's say we want to ignore all admin folders and subfolders + just one exact page pages/my-secret-page

npx svelte-sitemap --domain https://www.example.com --ignore 'pages/my-secret-page' --ignore '**/admin/**'

Ping Google Search Console

Every time I deploy a new version, I want to inform Google that there is a new update.

See this discussion with very useful tips.

Error: Missing folder

× Folder 'build/' doesn't exist. Make sure you are using this library as 'postbuild' so 'build/' folder was successfully created before running this script.

• Make sure your output folder exists. If it has other name than the default build, you can use the outDir (--out-dir) option.

Vercel apdatper

• If you are using Vercel hosting and adapter-vercel you'll probably want to use it like this:

npx svelte-sitemap --out-dir .vercel/output/static --domain https://www.example.com

Or check out other solutions and join the discussion.

Cloudflare adapter

• If you're using @sveltejs/adapter-cloudflare to deploy your app to Cloudflare Pages, you'll need to add some options to your adapter in svelte.config.js:

-import adapter from '@sveltejs/adapter-auto';
+import adapter from '@sveltejs/adapter-cloudflare';

/** @type {import('@sveltejs/kit').Config} */
const config = {
    kit: {
-       adapter: adapter()
+       adapter: adapter({ routes: { include: ['/*'], exclude: ['<all>', '/sitemap.xml'] }})
    }
};

export default config;

Error: Missing html files

× There is no static html file in your 'build/' folder. Are you sure you are using Svelte adapter-static with prerender option?

This library is intended for the static adapter and prerender option (SSG). So if there are no static files, then my library will not work for you :/

⭐️ Show your support

Give a ⭐️ if this project helped you!

Or if you are brave enough consider making a donation for some 🍺 or 🍵 ;)

🕵️ Privacy Policy

I DO NOT STORE ANY DATA. PERIOD.

I physically can't. I have nowhere to store it. I don't even have a server database to store it. So even if Justin Bieber asked nicely to see your data, I wouldn't have anything to show him.

That's why, with this library, what happens on your device stays on your device till disappear.

🤝 Contributing

I welcome you to customize this according to your needs ;)

Pull requests for any improvements would be great!

Feel free to check issues page.

Developing and debugging this library

git clone [email protected]:bartholomej/svelte-sitemap.git
cd svelte-sitemap
yarn
yarn start

Run demo locally

You can find and modify it in ./demo.ts file

yarn demo

🙏 Credits

• svelte-sitemap is workaround for this official SvelteKit issue
• Brand new version is inspired by Richard's article
• Thanks to @auderer because his issue changed the direction of this library

📝 License

Copyright © 2023 Lukas Bartak

Proudly powered by nature 🗻, wind 💨, tea 🍵 and beer 🍺 ;)

All contents are licensed under the MIT license.

svelte-sitemap's Issues

`change-freq` is not documented

Hi!

First eval thanks for the wonderful package!

Just noticed a small thing: change-freq is implemented but not documented in the README.

Does not recognize svelkit pages

Hello, I am trying this great tool but I have a problem and it is that it does not recognize the subdirectories. What am I doing wrong? Thank you

Custom output directory option produces double slashes in sitemap.xml

Hi, first of all, thanks for your package, it just works out of the box, which is great.

The small issue I have, as described in title, is that -o option produces double slashes in sitemap.xml

My package.json:

    "postbuild": "svelte-sitemap --domain https://mydomain.com -o output",

My svelte.config.js

import preprocess from 'svelte-preprocess';
import adapter from '@sveltejs/adapter-static';

/** @type {import('@sveltejs/kit').Config} */
const config = {
	preprocess: preprocess(),

	kit: {
		adapter: adapter({
			pages: "output",
			assets: "output"
		}),
		target: '#svelte',
		trailingSlash: 'always'
	}
};

export default config;

Resulting sitemap (simplified):

<?xml version="1.0"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <!-- This file was automatically generated by https://github.com/bartholomej/svelte-sitemap v2.0.1 -->
  <url>
    <loc>https://mydomain.com</loc>
  </url>
  <url>
    <loc>https://mydomain.com//about</loc>
  </url>
</urlset>

(check the //about URL)

Env info:

  System:
    OS: Windows 10 10.0.17763

  npmPackages:
    @sveltejs/adapter-static: ^1.0.0-next.18 => 1.0.0-next.18
    @sveltejs/kit: next => 1.0.0-next.164
    svelte: ^3.34.0 => 3.42.4
    svelte-sitemap: ^2.0.1 => 2.0.1

SvelteKit new default adapter-static configuration makes Sitemap point to invalid URL

Hello!

So, after this PR was merged, my static website started building pages in the folder like /my-page.html instead of /my-page/index.html. This works fine on the hosting side, but I've found that the Sitemap is now generated with the ".html" in the end of the URL. However, that URL is invalid as it will throw a 404.

Sample generated sitemap:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <!-- This file was automatically generated by https://github.com/bartholomej/svelte-sitemap v2.3.0 -->
  <url>
    <loc>https://fantinel.dev/2019-year-in-review.html</loc>
  </url>
  <url>
    <loc>https://fantinel.dev/2020-year-in-review.html</loc>
  </url>
</urlset>

The project I use it on is this: https://github.com/matfantinel/matfantinel.github.io
You can currently reproduce it on main branch by running npm run build.

I'm using svelte-sitemap v2.3.0. Happy to offer more details if needed

Make "This file was automatically generated..." text optional

For security reasons we may not want to share what technology is being used to generate the site.

Add a no-attribution switch to be used like: svelte-sitemap --domain https://stlouis.closedparishes.com --no-attribution

Which will stop this line from being run:

svelte-sitemap/src/helpers/global.helper.ts

Lines 35 to 37 in ab8f294

	sitemap.com(
	` This file was automatically generated by https://github.com/bartholomej/svelte-sitemap v${version} `
	);

`npm run svelte-sitemap --domain https://example.com` doesn't work

npm run svelte-sitemap --domain https://example.com doesn't work, output:

npm ERR! Missing script: "svelte-sitemap"
npm ERR! 
npm ERR! To see a list of scripts, run:
npm ERR!   npm run

I want to use as a post build hook, but need this command to test it locally. I used option with file, from example, that works well. would be good to fix it.

Fails to ignore pages when using custom build folder

For some reason, this will work:

svelte-sitemap -d https://coolwebsite.com -i 404

while this won't 😕:

svelte-sitemap -d https://coolwebsite.com -o dist -i 404

Consider filtering out 404

I noticed my sitemap.xml contains a /404 route. It might be worth filtering that out of the final result.

To repro:
I used the cloudflare-adapter (steps detailed in my other issue), and all worked as expected, afaict.

Add lastmod

Google recently deprecated the sitemap ping (1-6-2023) in favor of the <lastmod> element. Any chance that could be integrated?

https://developers.google.com/search/blog/2023/06/sitemaps-lastmod-ping

Does not recognise adapter-vercel build directory

Issue

svelte-sitemap does not recognise the build directory when using adapter-vercel, regardless of what output directory is provided.

Details

Error

× Make sure you are using this script as 'postbuild' so '.vercel_build_output' folder was successfully created before running this script. See https://github.com/bartholomej/svelte-sitemap#readme

Scripts

Relevant excerpt from package.json:

"scripts": {
    "postbuild": "yarn generate:sitemap",
    "generate:sitemap": "svelte-sitemap -d https://www.example.com -o .vercel_build_output"
}

Versions

"svelte-sitemap": "^2.1.1"
"@sveltejs/adapter-vercel": "next" (version "1.0.0-next.28" in yarn.lock)
"@sveltejs/kit": "next" (version "1.0.0-next.159" in yarn.lock)

I've tried adding ls -a in the postbuild script prior to running generate:sitemap and the output includes .vercel_build_output, so the directory does exist

Referencing the sitemap.xml in a Netlify build

Hey there,
maybe I'm really stupid here but I'm quite new to Sveltekit and has struggled with getting this plugin to work on a Netlify deploy. It get's added correctly to the build folder, which I can also see when I download the deploy from Netlify, but when visiting the mysite.com/sitemap.xml url, I get a 404 and I interpret it like Sveltekit expects a route, but nothing found.

Do I need to exclude the sitemap from the routes somehow?

I've seen some other solutions online where people create their own loader and custom build function for the sitemap which I could do of course, but I was under the impression this was not needed for this plugin? Or how are people actually being able to "visit" their sitemap without Sveltekit interfering and trying to render a route there?

Thanks!

// Jens.

Custom build dir

I am using the vercel adapter with its own build output folder. Would be great if we can customize the build dir.

– Originally requested by @tonprince in this issue #1 (comment)

Scan output build folder instead of src/routes

Currently this package scans the src/routes folder to generate the sitemap. This works fine for normal pages, but it doesn't work for dynamically generated pages like [slug].svelte, etc.

Instead, we should scan the build folder. Here's a demo script that does this:

import fs from "fs";
import fg from "fast-glob";
import { create } from "xmlbuilder2";
import pkg from "./package.json";

const getUrl = (url) => {
	const trimmed = url.slice(6).replace("index.html", "");
	return `${pkg.url}/${trimmed}`;
};

async function createSitemap() {
	const sitemap = create({ version: "1.0" }).ele("urlset", {
		xmlns: "http://www.sitemaps.org/schemas/sitemap/0.9"
	});

	const pages = await fg(["build/**/*.html"]);

	pages.forEach((page) => {
		const url = sitemap.ele("url");
		url.ele("loc").txt(getUrl(page));
		url.ele("changefreq").txt("weekly");
	});

	const xml = sitemap.end({ prettyPrint: true });

	fs.writeFileSync("build/sitemap.xml", xml);
}

createSitemap();

We should use a method like this instead of routes, as it will capture the actual output from Svelte, which can differ greatly from the source files.

Cannot find module 'fast-glob'

Apparently, there is something wrong with the fast-glob dependency that throws this error when running the postbuild script. I'm using pnpm, so my node_modules folder is flat. I think that's because fast-glob is a devDependency, instead of a normal dependency, but idk

Logs

internal/modules/cjs/loader.js:905
  throw err;
  ^

Error: Cannot find module 'fast-glob'
Require stack:
- C:\Users\user\svelte\node_modules\.pnpm\[email protected]\node_modules\svelte-sitemap\src\helpers\global.helper.js
- C:\Users\user\svelte\node_modules\.pnpm\[email protected]\node_modules\svelte-sitemap\src\index.js
- C:\Users\user\svelte\node_modules\.pnpm\[email protected]\node_modules\svelte-sitemap\index.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at Object.<anonymous> (C:\Users\user\svelte\node_modules\.pnpm\[email protected]\node_modules\svelte-sitemap\src\helpers\global.helper.js:7:37)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:14)
    at Module.require (internal/modules/cjs/loader.js:974:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    'C:\\Users\\user\\svelte\\node_modules\\.pnpm\\[email protected]\\node_modules\\svelte-sitemap\\src\\helpers\\global.helper.js',
    'C:\\Users\\user\\svelte\\node_modules\\.pnpm\\[email protected]\\node_modules\\svelte-sitemap\\src\\index.js',
    'C:\\Users\\user\\svelte\\node_modules\\.pnpm\\[email protected]\\node_modules\\svelte-sitemap\\index.js'
  ]
}

Command not found

Following

yarn add svelte-sitemap --dev

It says I get [email protected] version

When I try to run svelte-sitemap in some way in terminal on mac I get:

zsh: command not found: svelte-sitemap

It's the same with installing the npm package

Auto-detect build adapter

If possible, this would remove one more aspect for devs to think about and allow it to be used more easily in automated set up scripts.

Add option for additional pages not within SvelteKit

I use SvelteKit for my blog but not my main site. I'd like to add some additional pages by hand if possible.

I'll make a PR for adding the support!

Small script changes needed for SvelteKit 1.0 on Vercel using adapter-static

Hi,

Just wanted to document here that the correct (AFAIK - it works for me!) script in package.json for a Vercel deployment is:

npx svelte-sitemap --out-dir .vercel/output/static --domain https://www.example.com

Cloudflare Pages solution

Wanted to share how to get this working with Cloudflare Pages successfully.

1. Use import adapter from '@sveltejs/adapter-cloudflare'; instead of adapter-auto
2. Add the following within the kit property within svelte.config.js

		adapter: adapter({
			routes: {
				include: ['/*'],
				exclude: ['<all>']
			}
		})

3. In package.json, use "postbuild": "npx svelte-sitemap --out-dir .svelte-kit/cloudflare --domain https://example.com --debug"

Actually can't run the script

Right now, my dependencies are like this:
"@sveltejs/adapter-auto": "next",
"@sveltejs/adapter-static": "^1.0.0-next.29",
"@sveltejs/adapter-vercel": "^1.0.0-next.47",
"@sveltejs/kit": "next",

scripts:
"build": "svelte-kit build",
"postbuild": "npx svelte-sitemap --domain https://danieleavolio.it"

Whenever i try to run the command I get:
× Make sure you are using this script as 'postbuild' so 'build' folder was successfully created before running this script. See https://github.com/bartholomej/svelte-sitemap#readme

I don't know what to do

Exclude pages from sitemap

First of all, thank you very much for this project and your work, I use it in my project and it's great!

Second, is there any way to exclude certain pages from the sitemap? For example, auth pages or maybe "secret" pages that you don't want to be indexed?

What about <priority>?

Awesome package!
Great work!

But what about adding the option to add attribute for the sitemap.xml?

Add option for including or excluding trailing slashes in the sitemap

Continuing from #6 (comment)

Perhaps you could add a command line argument where you can specify whether the URLs in the sitemap should contain a trailing slash or not.
It's up to you which should be the default, if it were my call I would not include a trailing slash by default and expect the developer to specify that they want this option.

Thanks again for your work on this project, I really appreciate it.

[New Feature Request] Automatically submit sitemap to search engine APIs

Please excuse me if this is out of the scope of this project, but it would be really cool if this library, in addition to generating the sitemap, also had an option to ping search engine APIs that the sitemap had been updated (e.g. Google's Search Console API). Right now I'm doing this manually.

[New Feature Request] Automatically set lastmod in sitemap

It'd be great if this library could automatically fill in the <lastmod> time in the generated sitemap based on either the datetime the source route file was modified (probably won't work if you clone a fresh git repo since all files will have the same datetime?) or maybe the datetime the file was last modified in git or something.

Parameter -t does not work

Hey there 👋 I have tried this project for my small Svelte Kit website and found a tiny documentation or code bug.

Docs state:

But command results in following:

> npx svelte-sitemap -t --domain https://#####/

⚠ Those arguments are not supported: -t
Use: `svelte-sitemap --help` for more options.

I think the solution is simply to add a new alias to the list:

https://github.com/bartholomej/svelte-sitemap/blob/master/index.ts#L16-L31

Let me know if the project is open to contributions and I will gladly submit a PR 😇

Large Sites

Hello, great tool, one problem though, sitemaps are limited to 50k entries per page.

[New Feature] Load config from file `svelte-sitemap.cjs`

Feature idea

I want to try the possibility of loading the config from a file

• This should become the main and recommended method
• The CLI version should remain untouched

Proposal

./svelte-sitemap.cjs

module.exports = {
  domain: 'https://www.example.com',
  resetTime: false,
  debug: true
};

Let's discuss here ;)

Move site name to a config

Instead of having the site name in the command which lives in package.json:

"postbuild": "npx svelte-sitemap --domain https://eecue.com"

Could we move it somewhere else that can be populated dynamically at runtime? I use a config file for things like this:

import { config } from '$lib/config';

Ideally I could use that to populate the domain.

[bug] sitemap couldn't be read by the google search console

I just used your tool and it does a great job with indexing the site 🙏. It also integrates very neat into the build progress, thank you for that.

But as I submitted the sitemap to the Google Search Console, it couldn't get read (screenshot is in german, sorry for that):

Is this issue known?