There are often pages on a site that you don’t want to be indexed. If you have the same text on multiple pages, of you have a generic privacy notice or disclaimer which you use across multiple sites, using the robots “NOINDEX” meta tag on those pages will prevent a duplicate content slap from Google.

This is a particular problem when using a system such as Wordpress as a CMS, since by default there are archives (date archive, author archive etc.) which can be seen as duplicate content by the search engines.

I’ve recently been installing a plugin on all my Wordpress sites which makes managing which pages are NOINDEX and NOFOLLOW nice and easy. The plugin is simply called Robots Meta and can be downloaded from the Wordpress.org plugin directory.

The plugin is simple to install and use, allowing you to set noindex and nofollow on a per page/post basis, and also for sections of your site such as specific archives. An essential tool in the arsenal of any Wordpress CMS webmaster!