Comments on: +John Mueller second question in a day http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/ Search Engine Optimisation Ireland Sat, 28 Mar 2015 13:21:35 +0100 hourly 1 http://wordpress.org/?v=3.9.1 By: John Muellerhttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1364276 Fri, 03 Oct 2014 12:36:39 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1364276 Great to hear, Richard! Yes, that would have made this more useful here. Luckily we don't see a lot of these cases, but maybe there's a way to aggregate that information without cluttering the UI for those who don't need it. I'll pass that on. 

]]> By: Richard Hearnehttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1363865 Fri, 03 Oct 2014 08:59:02 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1363865 FYI +John Mueller it turned out that that IP had been added to an RTBH list by the hosting provider.

Should be fine now.  Thanks for your help with this – it's very much appreciated.

It might be a nice future feature for GWT to message more info around this as it's not something that could ever be diagnosed in the logs.

Again thanks John.

]]> By: Richard Hearnehttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1363294 Fri, 03 Oct 2014 03:17:04 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1363294 Thanks John for your help.  We'll try to figure out where that IP is being blocked and why.

]]> By: John Muellerhttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362811 Thu, 02 Oct 2014 20:46:21 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362811 Something on your server or upstream of it seems to be blocking our 66.249.78.189. Those are the requests that are timing out. The other IP addresses pretty much work normally. Depending on which IP address is used for which part of the crawl, that's the bumps you're seeing.

]]> By: Joseph Mashttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362312 Thu, 02 Oct 2014 14:09:22 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362312 +Richard Hearne, hope you find the issue.  Please share your findings if you discover something other than the server :)

]]> By: Richard Hearnehttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362311 Thu, 02 Oct 2014 13:48:29 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362311 Server was rebooted 2 days ago but still getting issue reports from GWT. Network guys also checked and found nothing. Thanks for your suggestions +Joseph Mas. Hopefully John can check if anything obvious from their side.

]]> By: Joseph Mashttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362233 Thu, 02 Oct 2014 13:07:47 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362233 In nearly every case I have handled it was either the mail server being bombed or SQL server bring the entire http server down for a short periods that were undetectable if browsing the site. But for a bot during a crawl, a hickup can kill the entire crawl. Hope that helps.

]]> By: Joseph Mashttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362232 Thu, 02 Oct 2014 13:04:26 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362232 Get the server logs. They will reveal if there is an issue with the server itself. Could be as simple as needing a reboot..

]]> By: Richard Hearnehttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362166 Thu, 02 Oct 2014 11:54:32 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362166 Hi John were seeing 20% fail only over past 2 weeks. Multiple emails coming from wmt telling us also. Domain is motorcheck.ie if there's anything you can see from your site it would be great.

]]> By: John Muellerhttp://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362165 Thu, 02 Oct 2014 11:42:11 +0000 http://www.redcardinal.ie/general/02-10-2014/john-mueller-second-question-in-a-day/#comment-1362165 The times I checked these there were real issues involved. Sometimes not as much as Webmaster Tools would make you believe though. For example, if the robots.txt is unreachable briefly, then we'll stop crawling anything until we can reach it again. Also, sometimes with big sites, we crawl so much that even if 10% fail, we still have a pretty good coverage of the new & updated content. Happy to check if you give me a URL though.

]]>