{"id":1354614,"date":"2017-09-25T16:37:36","date_gmt":"2017-09-25T23:37:36","guid":{"rendered":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/"},"modified":"2017-09-25T16:37:36","modified_gmt":"2017-09-25T23:37:36","slug":"crawling-problem-2","status":"closed","type":"topic","link":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/","title":{"rendered":"Crawling Problem"},"content":{"rendered":"<p>Hello, I have an issue linked to search crawling.<\/p>\n<p>My client has reported that crawling can return over 30,000 results from the events calendar. Most pages of which are redundant. For example: <\/p>\n<p>https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=50<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=51<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=52<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=53<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=54<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=55<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=56<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=57<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=58<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=59<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=6<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=60<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=61<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=62<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=63<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=64<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=65<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=66<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=67<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=68<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=69<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=7<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=70<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=71<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&amp;tribe_paged=72<\/ul>\n<p>Or ical &#8216;duplicates&#8217;:<\/p>\n<p>https:\/\/shakespeares-england.co.uk\/event\/ghost-walks\/2018-09-15\/<br \/>\nhttps:\/\/shakespeares-england.co.uk\/event\/ghost-walks\/2018-09-15\/?ical=1<\/p>\n<p>Or event extremely old monthly archives:<\/p>\n<p>https:\/\/shakespeares-england.co.uk\/events\/1970-08\/<\/p>\n<p>So my question is this: How do I prevent these (and similar) from being crawled?<\/p>\n<p><em>I am in a multisite instance of WP, and also use Yoast SEO for other SEO requirements.<\/em><\/p>\n","protected":false},"template":"","class_list":["post-1354614","topic","type-topic","status-closed","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Crawling Problem -<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Crawling Problem -\" \/>\n<meta property=\"og:description\" content=\"Hello, I have an issue linked to search crawling. My client has reported that crawling can return over 30,000 results from the events calendar. Most pages of which are redundant. For example: https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=50 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=51 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=52 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=53 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=54 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=55 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=56 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=57 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=58 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=59 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=6 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=60 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=61 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=62 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=63 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=64 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=65 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=66 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=67 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=68 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=69 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=7 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=70 [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/\",\"url\":\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/\",\"name\":\"Crawling Problem -\",\"isPartOf\":{\"@id\":\"https:\/\/theeventscalendar.com\/support\/#website\"},\"datePublished\":\"2017-09-25T23:37:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/theeventscalendar.com\/support\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Topics\",\"item\":\"https:\/\/theeventscalendar.com\/support\/topics\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Calendar Products\",\"item\":\"https:\/\/theeventscalendar.com\/support\/forums\/forum\/events\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Events Calendar PRO\",\"item\":\"https:\/\/theeventscalendar.com\/support\/forums\/forum\/events\/events-calendar-pro\/\"},{\"@type\":\"ListItem\",\"position\":5,\"name\":\"Crawling Problem\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/theeventscalendar.com\/support\/#website\",\"url\":\"https:\/\/theeventscalendar.com\/support\/\",\"name\":\"\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/theeventscalendar.com\/support\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Crawling Problem -","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/","og_locale":"en_US","og_type":"article","og_title":"Crawling Problem -","og_description":"Hello, I have an issue linked to search crawling. My client has reported that crawling can return over 30,000 results from the events calendar. Most pages of which are redundant. For example: https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=50 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=51 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=52 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=53 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=54 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=55 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=56 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=57 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=58 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=59 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=6 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=60 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=61 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=62 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=63 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=64 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=65 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=66 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=67 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=68 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=69 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=7 https:\/\/shakespeares-england.co.uk\/event\/warwick-town-centre-fun-quiz-trail\/all\/?tribe_event_display=past&#038;tribe_paged=70 [&hellip;]","og_url":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/","url":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/","name":"Crawling Problem -","isPartOf":{"@id":"https:\/\/theeventscalendar.com\/support\/#website"},"datePublished":"2017-09-25T23:37:36+00:00","breadcrumb":{"@id":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/theeventscalendar.com\/support\/forums\/topic\/crawling-problem-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/theeventscalendar.com\/support\/"},{"@type":"ListItem","position":2,"name":"Topics","item":"https:\/\/theeventscalendar.com\/support\/topics\/"},{"@type":"ListItem","position":3,"name":"Calendar Products","item":"https:\/\/theeventscalendar.com\/support\/forums\/forum\/events\/"},{"@type":"ListItem","position":4,"name":"Events Calendar PRO","item":"https:\/\/theeventscalendar.com\/support\/forums\/forum\/events\/events-calendar-pro\/"},{"@type":"ListItem","position":5,"name":"Crawling Problem"}]},{"@type":"WebSite","@id":"https:\/\/theeventscalendar.com\/support\/#website","url":"https:\/\/theeventscalendar.com\/support\/","name":"","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/theeventscalendar.com\/support\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/theeventscalendar.com\/support\/wp-json\/wp\/v2\/topic\/1354614","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/theeventscalendar.com\/support\/wp-json\/wp\/v2\/topic"}],"about":[{"href":"https:\/\/theeventscalendar.com\/support\/wp-json\/wp\/v2\/types\/topic"}],"version-history":[{"count":0,"href":"https:\/\/theeventscalendar.com\/support\/wp-json\/wp\/v2\/topic\/1354614\/revisions"}],"wp:attachment":[{"href":"https:\/\/theeventscalendar.com\/support\/wp-json\/wp\/v2\/media?parent=1354614"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}