Crawl errors (soft errors)

[Dennis]

As Cliff suggested in his reply in the thread below, I took a screenshot of the rewrite rules inspector output.
Result for this URL: https://www.wann-is-was.de/kalender/category/sport/motorsport/indycar-series/foto/

Result for this URL: https://www.wann-is-was.de/kalender/kategorie/sport/fussball/woche/2016-11-28/

Original Post:

I have the same issue on my site. Crawl errors have gone up the last weeks. Especially soft 404 errors. Those are not real 404 errors. But crawling these URLs costs crawl budget from Google.

I have enabled daily and monthly view and disabled everything else. Still Google is crawling pages like https://www.wann-is-was.de/kalender/kategorie/sport/fussball/woche/2016-11-28/. That site looks like a 404 page, but sends out a code 200 (OK).

Is there a way to prevent the creation of these pages (weekly view, list view and so on)?

========================
PLEASE LEAVE FOR SUPPORT
Reporting the same issue as: https://theeventscalendar.com/support/forums/topic/crawl-errors/

[Dennis]

Just in case there is a problem loading the screenshots…

[Cliff]

Hi. Thanks for sharing these details.

If I am understanding everything correctly (guessing on the language a bit), https://www.wann-is-was.de/kalender/kategorie/sport/fussball/woche/2016-11-28/ is an archive of the Sport category (unsure what fussball or woche are) and Day View is generating this URL… but there are no events on Nov 28, 2016.

Is this likely the correct guess/explanation of what’s happening?

[Dennis]

Hi Cliff,

sorry, I could have translated it:)
https://www.wann-is-was.de/kalender/kategorie/sport/fussball/woche/2016-11-28/
means
https://www.wann-is-was.de/calendar/category/sport/soccer/week/2016-11-28/

So it’s the week view in soccer category. The first URL is just the same, but photo view. I was just wondering why these URLs are being generated, cause I have just enabled month and list view. From a SEO perspective it would be better if these URLs wouldn’t be generated so Google wouldn’t have to crawl them (even if they are set to noindex). It saves a lot of crawling budget. Controlling the crawl budget is a very important part of SEO these days. Focusing on the important pages of your website and not having too much sites with thin content helps.

[Cliff]

Thanks for the extra information.

On my testing site, I was able to load this week’s calendar at both /events/category/concert/week/ and /events/category/concert/week/2017-05-28 (with and without the date on the end)

Then I went to my Display tab settings and unchecked Week View. Then I tried reloading these 2 URLs and neither worked.

Therefore, this seems to be working as expected — and I’d guess operating the same on your site — except you’re saying such URLs are getting hit by Google… which leads me to believe Google’s likely following a link or you have such URLs in your XML or HTML sitemap… basically, Google doesn’t end up at a URL without a reason; it gets pointed there.

Is there anywhere on your sitemap that links here? If not, is there anything throughout your site that does?

[Dennis]

Ok, thanks. So if I uncheck a certain view, these pages are still created, right? There is no way to avoid that these pages are generated?

I have to analyze if there are internal links pointing to these pages. I’ve crawled my website with screaming frog, but just found’t links to the previous and next week. But obviously it has to start somewhere.

There are no links on my sitemap to these pages. I will also run a test with a default theme. The theme came with it’s own style for TEC. Possible that there are internal links in there.

[Dennis]

I think I found the problem.

@Cliff: Can you do a test with your URLs from above and make a header check right here: http://www.webconfs.com/http-header-check.php

When I enable week view, I can see the page and get a “Code 200 OK” with the the header check.
When I disable week view, I see my normal 404 page, but the http header still sends a “Code 200 OK” instead of 404, which should be returned to the browser, search engines and so on.

I did a test on my other page with the default free plugin of TEC and disabled the month view. My browser returned the default 404 page for that: https://www.stockcar-news.de/events/monat/
But this URL also returns a Code 200 with the header check.

[Cliff]

Thanks for your effort here.

Because they’re 404s, I’d say they’re not created. And the only way Google would know about them is if your site told them about it.

Please let me know if you find out anything else in your additional testing.

[Support Droid]

Hey there! This thread has been pretty quiet for the last three weeks, so we’re going to go ahead and close it to avoid confusion with other topics. If you’re still looking for help with this, please do open a new thread, reference this one and we’d be more than happy to continue the conversation over there.

Thanks so much!
The Events Calendar Support Team

[Author]

Posts