site-map

Website Index and Crawl Health

Creating and Submitting XML Sitemaps

Since I’m building this site on WordPress with Yoast’s WordPress SEO plugin, setting up my XML sitemap has essentially been done for me. Check the box in the SEO settings of the XML sitemaps page and boom. A sitemap! I also checked off categoriestags and format as page types I didn’t want to include in my sitemap. I’ll get into that later on in this post, but my objective is clear: allow Google to index every possible piece of content on my website, but index it only once.

Now that I’ve got my sitemap file location from Yoast, http://marketingchris.com/sitemap_index.xml, it’s time to submit it to Google via Webmaster Tools, which I set up earlier in the campaign. From your GWT dashboard, click Optimization > Sitemaps > Add/Test Sitemap

Excluding Categories and Tags

It might seem strange that I didn’t want to include my category and tag pages in my sitemap, but I actually don’t want those indexed by Google. I mentioned earlier that I only one want my content indexed once and unless I make modifications to my category and tag pages, they will have the same exact content as the posts themselves. I want my pages to work together, not compete for link juice or traffic.

Look at how similar the category and tag pages are below.

 

The content I really want indexed are the articles themselves, not a list of articles. Thinking of it from a user experience perspective as well as respecting the resources needed to discover and store this information may be that which separates your website from others.

Create a Sitemap for a Non-WordPress Site

If you’re using another CMS (content management system), chances are it has a way for you to create XML sitemaps easily. However, if you (or your niece) built your website in a completely custom fashion, you may need to rely on another program to do the heavy lifting.

I recommend Xenu Link Sleuth as your starting point and keep it handy – you’ll probably use it later to detect potential issues with your website. Fire it up and launch your first crawl (yep, it’s a spider!) by clicking File > Check URL.

Xenu Link Sleuth

Once it’s finished crawling your website, click File > Create Google Sitemap File and then follow the directions above to submit your new XML sitemap to Google.

During my crawl I noticed some 404, Page Not Found errors. I’ll address those in my next post. If you want to skip ahead, please feel free to contact me! Thanks for reading.

3 thoughts on “Website Index and Crawl Health

  1. Lang Brucculeri

    The first thing you need to do before anything else is to get yourself a domain name. A domain name is the name you want to give to your website. For example, the domain name of the website you’re reading is “thesitewizard.com”. To get a domain name, you have to pay an annual fee to a registrar for the right to use that name. Getting a name does not get you a website or anything like that. It’s just a name. It’s sort of like registering a business name in the brick-and-mortar world; having that business name does not mean that you also have the shop premises to go with the name.;

    Our favorite internet page
    <".http://www.caramoanpackage.com/

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>