Provide a sitemap for search engines

[dogecoin]

Provide a sitemap for search engines

[cluzier]

If this has not already been started, i will give it a go @jeremiahisaacson @freelancing-solutions

[freelancing-solutions]

give it a go

…

On Wed, May 12, 2021 at 7:00 AM Conner Luzier ***@***.***> wrote: If this has not already been started, i will give it a go @jeremiahisaacson <https://github.com/jeremiahisaacson> @freelancing-solutions <https://github.com/freelancing-solutions> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#167 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHJMAVXBIP7COQAD7ULKXTTNIDQLANCNFSM437WIUSQ> .

-- *Thank you* *Justice Ndouhttps://justice-ndou.site <https://justice-ndou.site>*

[freelancing-solutions]

[jeremiahisaacson]

This would be a great addition. Is this something you're willing to put together? If not, I will.

[langerhans]

Help me understand the need please.

If this file doesn't exist, web robots assume that the website owner does not wish to place any limitations on crawling the entire site.

I mean that's exactly what we want right? We don't want to limit web crawlers so our site shows up in search results. Since no file is there, web crawlers just crawl the whole site. Or am I misunderstanding?

[freelancing-solutions]

yes you are , the same file is used to point to relevant pages so the site gets indexed properly for the subject its actually addressing rather than for subjects it isnt addressing... robots.txt and seo goes hand in hand, and a site without this file probably wont rank in google search . last i check this site doesnt rank anywhere in search ever wonder why

…

On Wed, May 12, 2021 at 8:05 PM Max K. ***@***.***> wrote: Help me understand the need please. If this file doesn't exist, web robots assume that the website owner does not wish to place any limitations on crawling the entire site. I mean that's exactly what we want right? We don't want to limit web crawlers so our site shows up in search results. Since no file is there, web crawlers just crawl the whole site. Or am I misunderstanding? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#167 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHJMAWYFGU544QVDVJQ7LLTNK7ODANCNFSM437WIUSQ> .

-- *Thank you* *Justice Ndouhttps://justice-ndou.site <https://justice-ndou.site>*

[langerhans]

You are mistaking robots.txt with a sitemap then. I have the Google Search console page open right now and the report shows all pages are indexed and no invalid pages are found.

[freelancing-solutions]

yeah the entry point to a sitemap is robots.txt

…

On Wed, May 12, 2021 at 8:16 PM Max K. ***@***.***> wrote: You are mistaking robots.txt with a sitemap then. I have the Google Search console page open right now and the report shows all pages are indexed and no invalid pages are found. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#167 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHJMARZY5XW66K542ZORM3TNLAWVANCNFSM437WIUSQ> .

-- *Thank you* *Justice Ndouhttps://justice-ndou.site <https://justice-ndou.site>*

[langerhans]

Okay then maybe we should rename the issue and focus on the sitemap part :)

[freelancing-solutions]

no you are mistaken i actually do SEO optimization beside coding and website development you cant have a sitemap without robots.txt it will just be a bunch of links in a file and useless

…

On Wed, May 12, 2021 at 8:21 PM Max K. ***@***.***> wrote: Okay then maybe we should rename the issue and focus on the sitemap part :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#167 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHJMAUEMCA53OJSUYNEQ7TTNLBKDANCNFSM437WIUSQ> .

-- *Thank you* *Justice Ndouhttps://justice-ndou.site <https://justice-ndou.site>*

[freelancing-solutions]

how would you tell google crawler which file is your sitemap file in your website without robots.txt ... robots.txt allows you to script how the crawler accesses your site..it does that by pointing to a sitemap file and also by indicating which part of the site to index,,, just naming the file sitemap.xml does nothing and its useless

…

On Wed, May 12, 2021 at 8:21 PM Max K. ***@***.***> wrote: Okay then maybe we should rename the issue and focus on the sitemap part :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#167 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHJMAUEMCA53OJSUYNEQ7TTNLBKDANCNFSM437WIUSQ> .

-- *Thank you* *Justice Ndouhttps://justice-ndou.site <https://justice-ndou.site>*

[langerhans]

Calm down, I just want to keep issues here focused. And technically you can have a sitemap without robots.txt by just telling Google where it is:

So can we get back to actually solving the problem instead of having this pointless argument?

[freelancing-solutions]

you are failing to see that robots.txt cannot be separated from a sitemap.xml
and requesting a crawler manually still wont solve the problem of not having robots.txt
it seems to me you cant figure out why its needed...
let me tell it to you like this , the crawler needs to know how to access your site.
and still needs to know this even if you submit your sitemap file...
robots.txt and map file ,,,serves the same purpose in a website...the other one is a default entry
point for the crawler..robots.txt without this file the crawler first hit on your site is a 404 error...
not the sitemap.

[langerhans]

Just drop it already. I understand we need a robots.txt to have a sitemap properly. Which I said already.

[freelancing-solutions]

so if the crawler gets a 404 it takes as an indication that your site is not working properly and will not therefore access your site normally...

[langerhans]

Added a basic sitemap in #204 . Let me know if that looks okay to you

[freelancing-solutions]

i didn't start this because i wanted to be the judge on what is appropriate for a sitemap or robots.txt i was just stating the obvious that robots.txt is needed...now a sitemap.xml file in a website without robots.txt does not constitute a sitemap as a sitemap is not made one because you named it sitemap its made one because you point to it from robots.txt file you could name it whatever you want , the only file you can't give a arbitrary name in a website is robots.txt the other files is as you see fit including sitemap . so i guess i am trying to tell you that the first file in a website is robots.txt this is a default that cant be changed

…

On Wed, May 12, 2021 at 8:58 PM Max K. ***@***.***> wrote: Added a basic sitemap in #204 <#204> . Let me know if that looks okay to you — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#167 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHJMAWI6GQTFT5BVCUBWSDTNLFULANCNFSM437WIUSQ> .

-- *Thank you* *Justice Ndouhttps://justice-ndou.site <https://justice-ndou.site>*

[freelancing-solutions]

Added a basic sitemap in #204 . Let me know if that looks okay to you

you cant get this wrong ...as long as you do it...lols

[freelancing-solutions]

just try limiting access to resource files as they contain javascript files which most certainly you dont want crawlers to crawl as the site does not offer javascript advice or content ..just a hint

[langerhans]

Closing as addressed by #204