Crawling And Indexing

    • An update on Google's feature phone crawling & indexing

      Limited mobile devices, "feature-phones", require a special form of markup or a transcoder for web content. Most websites don't provide feature-phone-compatible content in WAP/WML any more. Given these developments, we've made changes in how we crawl feature-phone content (note: these changes don't affect smartphone content): 1.

      Google Webmaster Central Blog- 11 readers -
  • Deprecating our AJAX crawling scheme

    … tl;dr: We are no longer recommending the AJAX crawling proposal we made back in 2009. In 2009, we made a proposal to make AJAX pages crawlable. Back then, our systems were not able to render and understand pages that use JavaScript to present content to users. Because "crawlers … [were] not able to see any content … created dynamically," we…

    Google Webmaster Central Blog- 14 readers -
  • The four steps to appiness

    … links: We now send a weekly clicks and impressions update to the Message center in your Webmaster Tools account. You can now track how much traffic app deep links drive to your app using referrer information - specifically, the referrer extra in the ACTION_VIEW intent. We're working to integrate this information with Google Analytics for even…

    Google Webmaster Central Blog- 8 readers -
  • Best practices for XML sitemaps & RSS/Atom feeds

    … of an Atom feed: 2011-06-27T19:34:00+01:00 ... “other tags” refer to both optional and required tags by their respective standards. We recommend that you specify the required tags for Atom/RSS as they will help you to appear on other properties that might use these feeds, in addition to Google Search. Best practices Important…

    Google Webmaster Central Blogin Google- 12 readers -
  • Testing robots.txt files made easier

    … Webmaster level: intermediate-advanced To crawl, or not to crawl, that is the robots.txt question. Making and maintaining correct robots.txt files can sometimes be difficult. While most sites have it easy (tip: they often don't even need a robots.txt file!), finding the directives within a large robots.txt file that are or were blocking…

    Google Webmaster Central Blog- 22 readers -
  • Android app indexing is now open for everyone!

    … it. As a site owner, you can show your users the right content at the right time — by connecting pages of your website to the relevant parts of your app you control when your users are directed to your app and when they go to your website. Hundreds of apps have already implemented app indexing. This week at Google I/O, we’re announcing a set of new…

    Google Webmaster Central Blog- 25 readers -
  • Directing smartphone users to the page they actually wanted

    … Webmaster level: all Have you ever used Google Search on your smartphone and clicked on a promising-looking result, only to end up on the mobile site’s homepage, with no idea why the page you were hoping to see vanished? This is such a common annoyance that we’ve even seen comics about it. Usually this happens because the website is not properly…

    Google Webmaster Central Blog- 14 readers -
  • Rendering pages with Fetch as Google

    … Webmaster level: all The Fetch as Google feature in Webmaster Tools provides webmasters with the results of Googlebot attempting to fetch their pages. The server headers and HTML shown are useful to diagnose technical problems and hacking side-effects, but sometimes make double-checking the response hard: Help! What do all of these codes mean…

    Google Webmaster Central Blogin Google- 19 readers -
  • Creating the Right Homepage for your International Users

    …. Show them the content that you consider the most appropriate. Whatever configuration you decide to go with, you should make sure all the pages – including country and language selector pages: Have rel-alternate-hreflang annotations. Are accessible for Googlebot's crawling and indexing: do not block the crawling or indexing of your…

    Google Webmaster Central Blog- 7 readers -
  • App Indexing updates

    … Webmaster Level: Advanced In October, we announced guidelines for App Indexing for deep linking directly from Google Search results to your Android app. Thanks to all of you that have expressed interest. We’ve just enabled 20+ additional applications that users will soon see app deep links for in Search Results, and starting today we’re making…

    Google Webmaster Central Blog- 3 readers -
  • More Precise Index Status Data for Your Site Variations

    … Webmaster Level: Intermediate The Google Webmaster Tools Index Status feature reports how many pages on your site are indexed by Google. In the past, we didn’t show index status data for HTTPS websites independently, but rather we’d included everything in the HTTP site’s report. In the last months, we’ve heard from you that you’d like to use…

    Google Webmaster Central Blog- 3 readers -
  • Infinite scroll search-friendly recommendations

    … isn’t -- the right example would cause crawling and indexing of duplicative content.</i></div><br><li>Structure URLs for infinite scroll search engine processing. <ul><li>Each component page contains a full URL. We recommend full URLs in this situation to minimize potential for configuration error.<ul><li…

    Maile Ohye/ Google Webmaster Central Blog- 25 readers -