Crawl Directives

    • SEO basics: What is crawlability?

      Ranking in the search engines requires a website with flawless technical SEO. Luckily, the Yoast SEO plugin takes care (of almost) everything on your WordPress site. Still, if you really want to get most out of your website and keep on outranking the competition, some basic knowledge of technical SEO is a must.

      Marieke Van De Rakt/ Yoastin SEO- 13 readers -
    • Ask Yoast: nofollow layered navigation links?

      If you have a big eCommerce site with lots of products, layered navigation can help your users to narrow down their search results. Layered or faceted navigation is an advanced way of filtering by providing groups of filters for (many) product attributes. In this filtering process, you might create a lot of URLs though, because the user will be able to filter and thereby group ...

      Joost De Valk/ Yoastin SEO- 14 readers -
    • Ask Yoast: should I redirect my affiliate links?

      There are several reasons for cloaking or redirecting affiliate links. For instance, it’s easier to work with affiliate links when you redirect them, plus you can make them look prettier. But do you know how to cloak affiliate links? We explained how the process works in one of our previous posts. This Ask Yoast is about the method of cloaking affiliate links we gave you in that post.

      Joost De Valk/ Yoastin Affiliate- 11 readers -
  • How to cloak your affiliate links

    … the same structure. This makes it a lot clearer for the user where the link is taking them to! Get the most out of Yoast SEO, learn every feature and best practice in our Yoast SEO for WordPress training! » $ 99€ 99 - Buy now » Info Cloaking affiliate links, the how to The basic process of cloaking affiliate links is simple: Create a folder…

    Joost De Valk/ Yoastin Affiliate SEO How To's- 25 readers -
  • Playing with the X-Robots-Tag HTTP header

    … different kinds of directives: crawler directives and indexer directives. I’ll briefly explain the difference below. Get the most out of Yoast SEO, learn every feature and best practice in our Yoast SEO for WordPress training! » $ 99€ 99 - Buy now » Info Crawler directives The robots.txt file only contains the so called ‘crawler directives’, which tells…

    Joost De Valk/ Yoastin SEO- 14 readers -
  • Crawl budget optimization

    … Google doesn’t always spider every page on a site instantly. In fact, sometimes it can take weeks. This might get in the way of your SEO efforts. Your newly optimized landing page might not get indexed. At that point, it becomes time to optimize your crawl budget. Crawl budget is the time Google has in a given period to crawl your site. It might…

    Joost De Valk/ Yoast- 15 readers -
  • robots.txt: the ultimate guide

    … this the crawl budget. By blocking sections of your site from the search engine spider, you allow your crawl budget to be used for other sections. Especially on sites where a lot of SEO clean up has to be done, it can be very beneficial to first quickly block the search engines from crawling a few sections. blocking query parameters One situation where…

    Joost De Valk/ Yoastin Paid Search- 10 readers -
  • rel=canonical • What it is and how (not) to use it

    … In February 2009, six years to the day from when this is published, Google, Bing and Yahoo! introduced the rel=canonical link element (Matt’s post is probably the easiest reading). While the idea is simple, the specifics of how to use it turn out to be complex. The basic premise is: if you have several similar versions of the same content, you…

    Joost De Valk/ Yoastin Social How To's- 17 readers -
Get the top posts daily into your mailbox!