Sitemap too big! The uncompressed size exceeds 10Mb

[kjvarga]

Sitemap too big! The uncompressed size exceeds 10Mb

[kjvarga]

That is a warning only, your sitemaps have generated fine. It looks like you are using an old version. Please update to latest (0.3.2 i think).

In newer versions we are enforcing the < 10Mb rule properly, so no sitemap should go over this limit.

Wow, that's a big sitemap! Would you mind pasting the output after you run with the new version so if there's any improvement in the speed.

Thanks,
Karl

[ghazel]

Well, they generated fine in than they exist on my webserver, but the Google Webmaster Tools say that file failed. They probably have a similar size check.

I'll upgrade, that's easy :)

The speed is certainly largely due to all the work I have to do with third-party RPCs and the database. So, don't worry that your code might be that slow.

[kjvarga]

Thanks for the report. The old version had a very crude method of checking the size. In the new version we know exactly how big everything is that goes into the file, so it's very easy to check that something can fit.

It's such a simple task it makes me want to optimize it. I haven't even had the chance to run a profiler yet. In Ruby 1.9.2 (?) there are Fibers for synchronous IO...imagine getting a record, and writing another to file happening simultaneously, so maybe a 50% boost in speed.

[ghazel]

The RPC calls are limited by the other end, actually. I've tried parallelization, but I believe they limit it to so many request per second per IP.