Typical Google though – “it’s not a duplicate content penalty, it’s a spam penalty”…
and….
“what’s actually happening… we want to show diversity… so we omit certain pages…”
Sorry but, if your rankings are being omitted from the search results because it’s duplicated somewhere else, regardless of how they want to dress it up… it’s a penalty.
The one thing I like though was the Royal.gov site. From working with other “SEOs” on a web design level, it’s something we come across a lot. This clearly explains the issues surrounding the homepage link – I will be pointing them in this direction the next time the issue arises!
The home.aspx example is probably the most common thing I see with ASP/MS sites. The good news is that Google has handled this pretty well now for about 18 months. Same goes for index.html, index.php etc. Of course I’d still think it’s better to remove this issue than rely on Google to fix on the fly.
Hi Michael – thanks for the link. Had seen that video previously. I think this new video is probably the best so far from Google to cover the baseline stuff.
Rgds to both, and thanks for dropping by Richard
Comment by Richard Hearne — September 18, 2009 @ 9:39 am
@Richard
I can honestly say I’ve never really had an issue with it. I see it as more of a non-issue than something to worry about.
Trying to explain this to SEOs hell bent on dealing with it as if it’s a major factor in their SEO efforts is where I end up banging my head against the wall! Now thanks to you, I have some armour to throw at them
@Tom – the reason I really don’t like seeing index.xxx, and often insist that it’s fixed, is that it generally points to other issues. It’s sloppy, and more often than not you’ll find other related issues like index.aspx Vs Index.aspx, which Google may not be handle so well. Google may display the correct version, but I’d never ever rely on them combining links.
Anyhow, enough of me ranting RGds Richard
Comment by Richard Hearne — September 19, 2009 @ 5:22 am
I recently worked on a site that had 3 pages indexed for the home page including /Default.aspx and the non-www.
I’ve also recently seen the /Default.aspx listed in GWT as a duplicate title and meta description.
So I wouldn’t rely on Google to sort the issue out themselves.
@Richard – I don’t agree it’s sloppy at all. For example, some CMSs will automatically create a link to a alias in order to generate a dynamic navigation. So in some cases you don’t have an option.
Combining the links is obviously something that appears to be an issue – but let’s be honest, it’s rare that you will have a link to the URL/index.html etc.
@Michael – it can take a while for Google to pick up what the default page is. But once it has, it’s not an issue as per video.
I regularly go to websites and find that the “Home” link points at domain/index.xxx, and that’s after arriving at domain/. Quite a few of the “SEO” websites I visit have this problem, and I do think it shows either sloppiness or a misunderstanding of how duplicate content can harm a website’s SEO. If the latter then I see this as a warning. I spend a lot of time looking at issues over on Google Webmaster Support Group, and one thing I’ve learnt (even from the Googlers) is better not leave such issues in the hands of the Google gods.
But of course we can both have different opinions on this Rgds Richard
Comment by Richard Hearne — September 21, 2009 @ 2:34 pm
@Richard – of course we can!!
My point would be that the only difference it makes is the dilution of links to your homepage – as the video above demonstrates, Google does not penalise/negatively affect your rankings for this.
I personally think a lot of SEOs make too much of a deal of this, when it’s not an issue at all. Show me a case where it has caused problems for a site.
It’s very naive to think that Google can’t handle this – at the end of the day it’s been there from the dawn of the web.
You could even point to the “Skip Intro” option in SERPs where Google tells you that there is an “intro page” on a website. If it has this covered, you can bet your backside that it has “index.xxx” covered.
We’ll have to disagree again – link dilution can and frequently does lead to reduced ranking. If you think not then just look at a site: query for websites where dupe content is prevalent. By splitting your links you’re effectively trying to promote 2 pages where only 1 exits.
Can I show you empirical evidence that this is a real issue? No. But from all the conversations I’ve followed with Googlers I think you’d be daft to assume that Google handles this perfectly.
I think they may have the index.xxx covered pretty well, but I’d not rely on them to get this right in other instances (e.g. different location of homepage, different page name, canonical URL issues etc.).
My view is – why leave things to chance? Just take care, and build your architecture to be resilient to these types of issues, rather than leaving things to chance.
The “Skip Intro” thingie is more to do with usability, and I’d not be so sure that they are combining splash page links with links to the internal homepage.
Of course all the above IMHO only Rgds Richard
Comment by Richard Hearne — September 25, 2009 @ 10:05 am
Hi Richard
Yeah again, I don’t disagree with the link dilution, but that’s something that rarely happens in most cases. I’m sure the % of peple linking to “/index.html” over the main domain is extremely low therefore even that is a minor issue.
Since you can’t provide any examples of it actually affecting anyones rankings I think it’s a bit unfair to put it down to being something that can have a negative impact on your sites rankings.
As the video states, your website will not be negatively affected by having a link to your “index” page, but that it will treat links to the individual pages as seperate entities, which is fair enough as it is wide open for abuse.
It’s our jobs as SEOs to make sure that any links back to our website are directed to the URL we are trying to promote.
Let’s use a bit of common sense here – if it did affect the SERPs in anyway – a competitor could easily put links to your /index.php page and affect your rankings. It just wouldn’t happen.
There are plenty of examples of where you can see a site has not been negatively affected by having links to “index.xxx”.
Off the top of my head, try the Revenue.ie site – you’ll see that the actual homepage isn’t indexed (i.e. /en/index.html) and you can even try 2bscene – my home.html isn’t indexed either. Both sites have links to their default file.
As we both know the web is full of people (or Googlers as you call them) with many different opinions on how Google works. The bottom line is you need conclusive evidence of how it affects the SERPs before you can tell someone it affects results. It’s all about trial and error.
In this case, there is no proof other than the effect on link dilution – which is covered in the video above. In all my years of doing this – I haven’t come across a site that has been negatively affected for having links to “index.xxx” being picked up as dup content. I would love to see examples of where it has actually affected a site – that way we could establish why some sites are effected (if they are) and others aren’t.
Oh and the “skip intro” example was just to point out that Google is intelligent enough to see if your site uses a splash page. So to do a simple task like figuring out if the index.xxx page is the home page or not would be a much easier task.
Hi Richard
Great find!
Typical Google though – “it’s not a duplicate content penalty, it’s a spam penalty”…
and….
“what’s actually happening… we want to show diversity… so we omit certain pages…”
Sorry but, if your rankings are being omitted from the search results because it’s duplicated somewhere else, regardless of how they want to dress it up… it’s a penalty.
The one thing I like though was the Royal.gov site. From working with other “SEOs” on a web design level, it’s something we come across a lot. This clearly explains the issues surrounding the homepage link – I will be pointing them in this direction the next time the issue arises!
Comment by Tom Doyle — September 17, 2009 @ 5:27 pm
Matt Cutts covered similar ground in this video tutorial about the Canonical Element – http://www.youtube.com/watch?v=Cm9onOGTgeM
Comment by Michael — September 18, 2009 @ 8:55 am
Hey Tom
The home.aspx example is probably the most common thing I see with ASP/MS sites. The good news is that Google has handled this pretty well now for about 18 months. Same goes for index.html, index.php etc. Of course I’d still think it’s better to remove this issue than rely on Google to fix on the fly.
Hi Michael – thanks for the link. Had seen that video previously. I think this new video is probably the best so far from Google to cover the baseline stuff.
Rgds to both, and thanks for dropping by
Richard
Comment by Richard Hearne — September 18, 2009 @ 9:39 am
@Richard
I can honestly say I’ve never really had an issue with it. I see it as more of a non-issue than something to worry about.
Trying to explain this to SEOs hell bent on dealing with it as if it’s a major factor in their SEO efforts is where I end up banging my head against the wall! Now thanks to you, I have some armour to throw at them
Comment by Tom Doyle — September 18, 2009 @ 4:33 pm
oh armour = ammunition!
Comment by Tom Doyle — September 18, 2009 @ 4:33 pm
@Tom – the reason I really don’t like seeing index.xxx, and often insist that it’s fixed, is that it generally points to other issues. It’s sloppy, and more often than not you’ll find other related issues like index.aspx Vs Index.aspx, which Google may not be handle so well. Google may display the correct version, but I’d never ever rely on them combining links.
Anyhow, enough of me ranting
RGds
Richard
Comment by Richard Hearne — September 19, 2009 @ 5:22 am
I recently worked on a site that had 3 pages indexed for the home page including /Default.aspx and the non-www.
I’ve also recently seen the /Default.aspx listed in GWT as a duplicate title and meta description.
So I wouldn’t rely on Google to sort the issue out themselves.
Comment by Michael — September 19, 2009 @ 6:49 am
@Richard – I don’t agree it’s sloppy at all. For example, some CMSs will automatically create a link to a alias in order to generate a dynamic navigation. So in some cases you don’t have an option.
Combining the links is obviously something that appears to be an issue – but let’s be honest, it’s rare that you will have a link to the URL/index.html etc.
@Michael – it can take a while for Google to pick up what the default page is. But once it has, it’s not an issue as per video.
Comment by Tom Doyle — September 21, 2009 @ 11:52 am
I regularly go to websites and find that the “Home” link points at domain/index.xxx, and that’s after arriving at domain/. Quite a few of the “SEO” websites I visit have this problem, and I do think it shows either sloppiness or a misunderstanding of how duplicate content can harm a website’s SEO. If the latter then I see this as a warning. I spend a lot of time looking at issues over on Google Webmaster Support Group, and one thing I’ve learnt (even from the Googlers) is better not leave such issues in the hands of the Google gods.
But of course we can both have different opinions on this
Rgds
Richard
Comment by Richard Hearne — September 21, 2009 @ 2:34 pm
@Richard – of course we can!!
My point would be that the only difference it makes is the dilution of links to your homepage – as the video above demonstrates, Google does not penalise/negatively affect your rankings for this.
I personally think a lot of SEOs make too much of a deal of this, when it’s not an issue at all. Show me a case where it has caused problems for a site.
It’s very naive to think that Google can’t handle this – at the end of the day it’s been there from the dawn of the web.
You could even point to the “Skip Intro” option in SERPs where Google tells you that there is an “intro page” on a website. If it has this covered, you can bet your backside that it has “index.xxx” covered.
Comment by Tom Doyle — September 25, 2009 @ 9:23 am
We’ll have to disagree again – link dilution can and frequently does lead to reduced ranking. If you think not then just look at a site: query for websites where dupe content is prevalent. By splitting your links you’re effectively trying to promote 2 pages where only 1 exits.
Can I show you empirical evidence that this is a real issue? No. But from all the conversations I’ve followed with Googlers I think you’d be daft to assume that Google handles this perfectly.
I think they may have the index.xxx covered pretty well, but I’d not rely on them to get this right in other instances (e.g. different location of homepage, different page name, canonical URL issues etc.).
My view is – why leave things to chance? Just take care, and build your architecture to be resilient to these types of issues, rather than leaving things to chance.
The “Skip Intro” thingie is more to do with usability, and I’d not be so sure that they are combining splash page links with links to the internal homepage.
Of course all the above IMHO only
Rgds
Richard
Comment by Richard Hearne — September 25, 2009 @ 10:05 am
Hi Richard
Yeah again, I don’t disagree with the link dilution, but that’s something that rarely happens in most cases. I’m sure the % of peple linking to “/index.html” over the main domain is extremely low therefore even that is a minor issue.
Since you can’t provide any examples of it actually affecting anyones rankings I think it’s a bit unfair to put it down to being something that can have a negative impact on your sites rankings.
As the video states, your website will not be negatively affected by having a link to your “index” page, but that it will treat links to the individual pages as seperate entities, which is fair enough as it is wide open for abuse.
It’s our jobs as SEOs to make sure that any links back to our website are directed to the URL we are trying to promote.
Let’s use a bit of common sense here – if it did affect the SERPs in anyway – a competitor could easily put links to your /index.php page and affect your rankings. It just wouldn’t happen.
There are plenty of examples of where you can see a site has not been negatively affected by having links to “index.xxx”.
Off the top of my head, try the Revenue.ie site – you’ll see that the actual homepage isn’t indexed (i.e. /en/index.html) and you can even try 2bscene – my home.html isn’t indexed either. Both sites have links to their default file.
As we both know the web is full of people (or Googlers as you call them) with many different opinions on how Google works. The bottom line is you need conclusive evidence of how it affects the SERPs before you can tell someone it affects results. It’s all about trial and error.
In this case, there is no proof other than the effect on link dilution – which is covered in the video above. In all my years of doing this – I haven’t come across a site that has been negatively affected for having links to “index.xxx” being picked up as dup content. I would love to see examples of where it has actually affected a site – that way we could establish why some sites are effected (if they are) and others aren’t.
Oh and the “skip intro” example was just to point out that Google is intelligent enough to see if your site uses a splash page. So to do a simple task like figuring out if the index.xxx page is the home page or not would be a much easier task.
Tom
Comment by Tom Doyle — September 25, 2009 @ 5:10 pm