Robots.txt File Case Study: How Third-Party Directive Changes Led To Leaking URLs And Lost SEO Traffic

by Glenn Gabe
I’ve written many times in the past about how technical SEO problems can lead to a serious drop in rankings and traffic. From the meta robots tag to rel=canonical to cloaked 404s, there are a number of problems that can lie below the surface, causing serious damage to your website, SEO-wise. Well, I unfortunately came across yet another situation recently that I’m going to cover in this post.Read the full article

robots.txt and robots meta tags – Google Webmasters – English

This session is focused on robots.txt and robots meta tags; ways to control crawling & indexing.