Category Archives: My Campaign

Why am I ranking for this keyword?

A lot has happened since my last campaign update, so there hasn’t been a lot of time for my personal projects. I joined Delphic Sage; some really smart marketers told me about their mentors; Raven and others are going to stop scraping ranking data; Christmas.

So when I finally got to check out Google Analytics, I noticed that I had more traffic than I expected and for unexpected terms. Without scraped ranking data I can still see keyword performance data using either Google Webmaster Tools or Google Analytics.

I found that http://marketingchris.com/my-campaign/improving-page-speed-easily/ is appearing for a lot of speed related queries, even though I’m not intentionally optimizing for that term.

Google Analytics Query Data

	Query	Impressions	Clicks	Average Position
1.	speed	27,000	500	21	1.85%
2.	foundation	10,000	50	17	0.50%
3.	spider	1,600	5	390	0.31%
4.	keywords	1,000	0	54	0.00%
5.	hız	700	16	17	2.29%
6.	raven	400	5	490	1.25%
7.	xenu	400	0	150	0.00%
8.	keyword	320	0	66	0.00%
9.	speed pictures	250	5	4.6	2.00%
10.	google database	200	5	23	2.50%
11.	short term goals	200	0	5.6	0.00%
12.	speed images	200	5	14	2.50%
13.	spped	200	5	20	2.50%
14.	countey	170	0	12	0.00%
15.	link detective	170	5	6.4	2.94%
16.	speed image	170	5	2.2	2.94%
17.	business goals	150	5	13	3.33%
18.	foundation images	150	5	3.6	3.33%
19.	raven tools	150	5	21	3.33%
20.	speed photo	150	5	5.0	3.33%
21.	سرعة	150	5	20	3.33%
22.	data	110	0	150	0.00%
23.	free cdn	110	5	18	4.55%
24.	onion talks social media	110	5	5.5	4.55%
25.	database architecture	90	0	68	0.00%
26.	keyword analysis	90	0	84	0.00%

Let’s investigate. First I’ll check Google Webmaster Tools to ensure that I’m only looking at queries with the type of web.

On Page Factors

Next, I’ll look at the on-site factors that might be influencing my rank.

Keyword in URL

http://marketingchris.com/my-campaign/improving-page-speed-easily/

Keyword in Page Title

<title>Improve Page Speed with a Free CDN | Page Speed for SEO</title>

Keywords in Headings

H1: Improve Page Speed with CloudFlare

H2 (with strong element): What is page speed?

Elements

Strong: page speed, high priority speed issues, awesome post of page speed, page speed optimizations

Keywords in Images

File names: http://marketingchris.com/wp-content/uploads/2012/11/page-speed1-470×260.jpg, http://marketingchris.com/wp-content/uploads/2012/11/page-speed-waterfall-view.jpg, http://marketingchris.com/wp-content/uploads/2012/11/google-mod-page-speed-300×81.jpg

Image ALT Tags: Improve Page Speed with CloudFlare

Meta Keyword Data

Social Media’s Role

This data reminds me that Rand Fishkin of SEOmoz helped me test CloudFlare by tweeting http://marketingchris.com/my-campaign/improving-page-speed-easily/.

I’ve seen Rand break sites with a Tweet, sending thousands of visitors that overwhelm an under-powered server. That was not the case here, could it be possible that his Tweet gives my page an edge over others for this term for image related queries?

Link Profile

The only real backlink to this page comes from http://www.inbound.org/articles/view/improving-page-speed-easily-chris-countey. Inbound.org (domain authority: 55) is like hacker news for marketers, and it so happens that Rand started it.

I’d love to hear what you think. Was is the combination of keywords in important places, the link from Inbound or the Tweet from Rand that causes this page to rank so well for speed? Or was this just the perfect storm for the perceived search intent?

Build Links By Building Relationships

1 Reply

In the previous post of this series I used Google Adwords to do a very early stage keyword analysis. The Google Adwords Keyword tool is pretty slick, but it only tells part of the story: the paid traffic. I need to see what web pages I’m up against organically. More importantly, I want to learn more about the people behind these websites.

Analyze Ranked Pages Fast

To start my competitive analysis I’ll use the SEOmoz Keyword Difficulty Tool. Maybe it’s ironic, but I did choose keyword research as a supporting term for seo consultant, which I found via the Google Adwords Keyword Tool. I’m not as interested in the competitive score as I am the data the tool provides about the top 10 ranking pages.

Click to Enlarge

SEOmoz allows you to export to CSV or Excel. Here’s mine: keyword-keyword research

Clicking on the magnifying glass will launch Open Site Explorer. Did you know that SEOmoz has their own index of the web and they use this data to evaluate link relationships? Ahrefs, Majestic SEO and other great tools index the web, too. This is why each tool may show different quantities of links when running reports, driving you or your clients bonkers.

Analyze Backlinks Quickly

Take my advice: take it one tool at a time. The image below shows the kind of data Open Site Explorer will return. You can see them all here: Inbound Links OSE

Click to Enlarge

Before I exported the data as a CSV, I applied the following filters:

2 seemingly powerful links catch my eye immediately:

Uncover Real People, Not Links

Websites are run by people. Period.

The first post was written by Lee Odden, CEO of Top Rank Online Marketing. I’ve definitely heard of him, but I doubt he’s heard of me. I need to find out where he lives online and what he likes to read with the hopes of creating something good enough to be published on TopRankBlog.com. Because he’s an SEO he’s likely to see new links to his site from mine.

That’s my first touch point: Google Analytics

Lee has also posted some additional links and social media profiles in his Google Plus account. There might be some clues as to what he loves to read in his Google Plus and Twitter timelines.

His LinkedIn profile gives some insight into how he should be approached. “If you would like to connect with me here on LinkedIn, please include a reason, context for how we are connected. I cannot reply to generic requests. Thanks!”

Seems friendly enough, but he probably gets a lot of spam from nerds like me. But his blog seems pretty popular (and the posts are pretty damn good, too) so I’ll maintain course in this direction. It seems like stalking and maybe it is, but if a professional relationship is my end-game, I need to do my homework.

Find Local People

SEOs tend to hover more around the Twittersphere, so I’ll take a look at who he’s following using another SEOmoz tool: Followerwonk.

Click to Enlarge

Followerwonk lets me zoom in to street level, showing me real, in-person opportunities to network should I venture into Philly, like, tomorrow. As a local SEO consultant I’ll use this to help my clients connect with other businesses in their area, both online and offline.

The next chart shows me the best times to send out a Tweet that Lee is likely to see…

… and the next chart shows what my Tweets should contain.

You can see the full exported list here.

All this for a link? Nope. All this for a relationship.

I highly recommend you give these applications a try. I’ll dive into the 2nd link in my next post. Thanks for reading!

Semantic Relationships, Intent and Content Sculpting

Leave a reply

In a previous post of my new campaign series I decided on seo consultant responsibilities as my first target keyword. I recently enrolled in the Distilled U SEO course and I’ll reference their lessons during this campaign, but I recommend you give their course a try.

The keyword I chose should be easy to rank for, but we’ll see!

First, I want to see what kind of results are being returned for this query.

It’s believed that Google classifies queries based on what it thinks the user’s intent is.

What is search intent?

Do: Distilled U labels this as transactional. In other words, the person is looking to perform an action with their query, such as buying a product or visiting a restaurant.
Know: Distilled U calls this informational. Google believes this user is doing research. If you see Wikipedia as the first result, chances are Google thinks you’re looking for more information on a topic.
Go: Distilled U refers to this query as navigational. This query is very brand-focused and is probably why a search for apple will result in that brand’s site being favored over a page that defines what an apple is.

Quiet, please. I’m now going to do a Google search… incognito! (This feature is found at the top right corner of the Chrome browser.)

I expected my results to be more informational, but they look more transactional. In other words, Google thinks I’m looking for a job as an SEO consultant or that I’m looking for resumes of consultants.

And it just so happens that I have my resume handy, so I created a new page to show my job experience. The stuff I talk about next is theory and just may be complete nonsense.

I want to see what other relevant terms appear on the pages that are already ranking by creating a Venn diagram from a couple of tag clouds. This isn’t so much scientific as it is sheer curiosity.

With my trusty Link Clump Chrome extension, I can open the top 3 ranking URLs with one right-click.

Next I’ll visit Tagcrowd.com to produce tag clouds for the 3 top pages. I can drag and select the text, open Excel and paste > paste special > text each series into its own column. Behind the scenes I have a macro that’s going to make it easy to create rows in Excel from text.

Maybe an Excel expert reading this can tell me how to create a Venn diagram using 3 columns of data easily, but until then, I need to convert this cell of text to rows and use a free online Venn diagram creator.

Here’s the code for the macro. Create it and then assign it to a button in Excel:

Sub SplitAndTranspose() Dim N() As String N = Split(ActiveCell, " ") ActiveCell.Resize(UBound(N) + 1) = WorksheetFunction.Transpose(N) End Sub

Next, I’ll paste each column into the boxes of the Venn diagram creator. I’m doing not keyword density analysis; instead I’m looking for terms that appear in 2 or 3 of the pages that are thematically related to my target.

Names	total	elements
user_list1 user_list2 user_list3	9	business marketing seo search internet engine team optimization clients
user_list1 user_list2	2	website campaigns
user_list1 user_list3	8	content web design responsibilities media social link blog
user_list2 user_list3	9	google thanks people work online manager job site lot

Based on this table, some words I should probably include on my resume page are in the first, third and 4th rows.

Interestingly enough, the sites labeled user_list1 and user_list2 don’t even have my keyword in the title. User_list3, which happens to be an SEOmoz article, has almost an exact match keyword page title. Sites 1 and 2 could be classified as transactional while the Moz post is informational.

I’ll need to look a little more closely, but with such a small dataset, I shouldn’t spend too much time thinking about why and just make some minor changes to the content of my resume page to include some of these terms.

Thanks for reading!

Recover WordPress Data

1 Reply

Earlier this year I shut down SEO Rewind, a satire website, in order to concentrate on this project. In addition to my lame posts and SEO comics, some very smart SEOs contributed to the site. In fact, the only reason anyone noticed the site was because of the content written by Joel Klettke, Nick Eubanks and Stuart McHenry.

Before I shut it down I exported the WordPress SQL database in case any of the authors asked for their work in the future. In hindsight, I should have used WordPress’s export function because it would have made restoring SEO Rewind a lot easier. My mission: Get my WordPress post data back.

Project Overview

Remove create database command from old SQL file
Use my host tool to create a new database
Import my old SQL data using phpMyAdmin
Upload a new WordPress installation to a sub-directory of my site
Connect it to my new database that contains my old data
Test!
Export from within WordPress
Import the WordPress XML file
Test!
Set up a landing page on SEO Rewind telling users what happened and provide navigation
Create permanent (301) redirects for important links to their new homes on this website
Test!
Have a mint hot chocolate

The problem with my scenario is that my SQL file includes the command to create a new database, something that isn’t allowed in a Network Solutions shared server environment.

If your host has similar restrictions, you may get an error message when trying to import a SQL file within phpMyAdmin or another database management platform. To get around this, I’ll edit the file removing the create database command and import it into a fresh database.

I’ll create a new database first using my host’s dashboard and back up my main database just in case. (If you’re not comfortable messing with databases, stop! Contact me if you’re unsure.)

Open the file, probably called localhost.sql, in a text editor, remove the create database command, save the file, fire up phpMyAdmin and click on import. (I kept the default settings.)

Make sure that database and USE are referencing the name of your new database or you’ll receive a permissions error. In this case, I created a new database with the same name as the old version. After you successfully import your old data, you’ll need to change the options table to reflect your new website.

Connect a New WordPress Site to Old WordPress Data

My new WordPress website with my old WordPress data is ready. Next I need to upload a new installation of WordPress and connect the data.

Create a new directory in your website and upload the WordPress files
Navigate to the directory to launch the setup page
Enter the database information you used above (caution: if you used something other than wp_ as the table prefix, use that version instead. In my case I used seo_ which could be seen the SQL file)
The setup is modifying the wp-config file shown below

Load your website in a browser and you should see your old data in terms of posts, categories, tags, even authors. When you first log into your WordPress dashboard you may get some plugin errors. You can ignore these for now. My primary objective was to export my WordPress data in a way that makes it easier and safer to import.

WordPress Important Tool

Now that I’ve got my XML export file, I can load the data safely using the Import Tool (Tools > Import).

Capturing Lost Link Flow

During SEO Rewind’s short life it accrued some external links and while I’m restoring the content, it wouldn’t hurt to restore some of the authority, too. Using ahrefs.com, I can see that there are some pretty good links I wouldn’t mind recovering for my new website. In order for that to happen, I need to identify what the old links were and redirect those pages to their new homes on this site.

First, I’ll set up a simple landing page on SEO Rewind to inform human visitors that it’s moved.

Next, I want to forward both humans and robots to the new addresses for these pages. 301 redirects or permanent redirects can help me recapture some of that authority I lost when the pages started returning 404 not found errors. Below is a sample of the redirects in my .htaccess file.

redirect 301 /outed-point-blank-seos-jon-cooper/719/ http://marketingchris.com/seo-rewind/outed-point-blank-seos-jon-cooper/
redirect 301 /google-unveils-ambitious-new-algorithm-update/650/ http://marketingchris.com/seo-rewind/google-unveils-ambitious-new-algorithm-update/
redirect 301 /darth-vader-talks-retargeting/802/ http://marketingchris.com/seo-rewind/darth-vader-talks-retargeting/

If you need help with anything in this post for your own site, let me know!

Laying My Content Architecture Foundation

Leave a reply

In my last post I began looking at potential keywords that I could create resources for. I split the keyword phrases into 2 lists: content for those who may be searching for consulting services and those who may be looking for training in SEO. I’ve made a lot of assumptions to this point and the only data I’ve used to make decisions was Google Autcomplete. Not a bad start, but in order to work smarter I need more data.

I’m going to use Google Adwords to test the keywords I’ve chosen based on Google Autocomplete and Ubersuggest results. By using the lowest bid I can test how likely my chosen keyword is to result in a conversion without spending a lot of money.

How to Set Up Google Adwords [VIDEO]

I chose [seo consultant responsibilities] as my target phrase so I’ll place a bid for this term in Adwords at the lowest amount. I won’t start the campaign until I’ve created my destination page. I used the Google Adwords keyword tool to see what the volume and competition numbers look like. As expected both are low, which to me indicates that I don’t need a full page specifically about seo consultant responsibilities in order to get some organic visibility. In fact, it might be overkill and building new pages for every keyword is likely to cause a Panda problem. Instead I can create a page about this keyword’s potential parent theme, seo consultant.

Using Adwords Competition to Determine Content Hierarchy

I’m not a fan of keyword density or other metrics that get too specific about how many times you need to have a keyword on a page. Instead I let the competition tell me where my keyword needs to appear at least once on the page.

Low competition: I can probably add the keyword as a heading and as a part of a paragraph, along with related terms.

Medium competition: I’ll probably need to include this term in the title and within content on the page.

High competition: I’ll definitely need a strong page, with the keyword in the title, in a heading and it will likely need supporting long-tail pages unless the page becomes extremely popular.

Creating the Page

My content about seo consultant responsibilities needs to be written for human visitors, not for Google. I know that I need the keyword on the page, but its placement and its usage need to be natural. Based on competition, I’ll be adding a section around this keyword as support for a more competitive term, such as seo consultant.

For my URL structure I want to show Google and other visitors how the pages fit together. I prefer the /category/post name/ architecture to the tld/post name format. Using Yoast’s SEO plugin and the permalink settings you can achieve this structure easily.

First, click on the permalink settings in the Yoast plugin and select the options shown below.

Next, click Settings > Permalinks and use the custom URL structure /%category%/%postname%/

Campaign Update

My new marketing campaign is about 3 weeks old so it’s time for a progress report. Google Webmaster Tools shows that the content on my website is starting to impact the SERPs by appearing for 26 search queries.

There isn’t anything too exciting in Google Analytics, but I am happy to see some “popular” keywords appearing on the radar.

Social traffic is currently the highest source with organic search coming in second. I expect this to shift in the next few weeks as new content gets indexed.

What do you think about the graphs below? What other information would you like to see?

I’ll provide the results of my Adwords test next. Thanks for reading!

How do I find keywords?

3 Replies

Choosing the right keywords is possibly the most important step of my campaign. If I go too broad the competition is likely to be very high and conversions low. If I go too long-tail I risk not getting enough organic traffic. Using the Google Adwords keyword tool will help me identify better keywords later.

SEOmoz and SEO Book both have keyword research tools, but perhaps the most valuable is the SEOmoz How to Do Keyword Research tutorial. In a post linked from the tutorial, Rand Fishkin of SEOmoz breaks down some key metrics of keywords, such as the Search Demand Curve below.

I definitely recommend reading the guides from SEOmoz in full, but I’m going to paraphrase a little. Before I start researching keywords I need to think about my goals. What is it that I want this site to do?

Choose keywords by identifying your objective.

The ultimate goal of my website is to drive new consulting and training customers.

How?

I need to increase my visibility on the web, especially where it matters (more on this later).

How?

Build strong relationships with other online marketers.

How?!

I need to build up my authority on the subject matter that interests them and I need to be real (more on this later).

How?!!

My website needs to answer questions that are being asked by the community and by potential customers.

OMG How?!!

Begin research into what people are really asking about and develop my content around those ideas.

What are people searching for?

Maybe a better question is “how are people searching for what I offer?” Google auto suggest is an excellent starting point to identify search phrases related to your topic that you may not have thought of.

This is a good start, but I want to see more data. Ubersuggest takes this idea of using auto suggest to the next level. Using the query * seo consultant, Ubersuggest returns the following phrases that people are searching for on Google:

See the keywords: Round 1 Keyword List

Identify Searcher Intent

Next I take the bold words and identify who may be searching for these terms. If you’re like me, the wheels are turning as I go through this list, which I’ve narrowed down below:

See the keywords: Round 2 Keyword List

Identify Potential Resources as Content

I believe I can create resources for most if not all of these terms, but I want to write for my primary audience first. I also want to think about the intent of each query, as Google might. Beyond just the words, what is it this searcher is looking to accomplish?

The phrases in bold above may be more relevant to someone looking to start or advance their career in SEO as opposed to someone looking for a consultant. Training is part of my ultimate conversion, but I want to be hired as a consultant first and foremost.

Here is my new list of potential key phrases.

seo consultant wordpress
seo consultant for hire
seo consultant wordpress
seo consultant for hire
seo consultant articles
ethical seo consultant
real estate seo consultant
ecommerce seo consultant
seo consultant for hire
seo consultant for small business
qualities good seo consultant
google authorized seo consultant
white hat seo consultant
reasons hire seo consultant
joomla seo consultant
seo consultant list
local seo consultant
law firm seo consultant
seo consultant nj
seo consultant nonprofit
seo consultant new jersey
cost of seo consultant
responsibilities of seo consultant
questions ask seo consultant
qualities good seo consultant
seo consultant interview questions
seo consultant responsibilities
seo consultant real estate
freelance seo consultant rates
seo consultant skills
seo consultant small business
seo consultant wordpress
why hire seo consultant

I really feel like this phrase “seo consultant responsibilities” is something that is both important and something I can create a resource around. Back to Ubersuggest I go to see what else I can uncover about search behavior for this phrase.

Next time I’ll go through my content creation process for this term and measure my return. Thanks for reading!

Improve Page Speed with CloudFlare

12 Replies

What is page speed?

Page speed is one of my favorite topics to explore when optimizing a website. Simply put it’s the measure of how fast a website goes from being requested to being fully rendered in a browser. File size, distance from server, the amount of traffic, server environment and other factors influence page speed. In this part of my series on starting a new campaign, I’ll show you how to tap into the power that is “the cloud”.

It’s challenging, Google loves it and it just makes sense from a user experience standpoint. Improving page speed may involve the following:

Avoid CSS @import
Avoid a character set in the meta tag
Avoid bad requests
Avoid landing page redirects
Enable Keep-Alive
Enable compression
Inline Small CSS
Prefer asynchronous resources
Put CSS in the document head
Serve resources from a consistent URL
Serve scaled images
Specify a cache validator
Specify a character set

Minimize redirects
Combine images into CSS sprites
Defer parsing of JavaScript
Leverage browser caching
Optimize the order of styles
Inline Small JavaScript

Minify CSS
Minify JavaScript
Minimize request size
Optimize images
Minify HTML
Specify image dimensions
Remove query strings from static resources

Some of these items are more important than others. Visit the Google Developers section on Page Speed and download the Chrome or Firefox plugin (requires the Firebug plugin, which I also recommend) to familiarize yourself with the important or high priority speed issues.

Check out this awesome post on page speed by Ian Lurie: How we made Portent.com really freaking fast.

Seriously check it out, I’ll wait. OK, back?

A year ago I would have recommended a plugin, W3 Total Cache, as the solution to improve page speed for my WordPress site. But more recently I’ve started using a reliable content delivery network (CDN), CloudFlare.

How do I set up CloudFlare?

If there is a plugin that can handle page speed, why bother using an external service? First, let’s look at what a CDN is:

A content delivery network (CDN) is a large distributed system of servers deployed in multiple data centers in the Internet. The goal of a CDN is to serve content to end-users with high availability and high performance. CDNs serve a large fraction of the Internet content today, including web objects (text, graphics, URLs and scripts), downloadable objects (media files, software, documents), applications (e-commerce, portals), live streaming media, on-demand streaming media, and social networks.

Source: http://en.wikipedia.org/wiki/Content_delivery_network

CloudFlare is a free CDN that is well-worth taking the time to set up for your website.

In addition to being a CDN, the graphs below show traffic data, potential threats to your website and how much bandwidth is saved by using their service.

What do I need to set up CloudFlare?

Access to your DNS (domain name servers)
A CloudFlare account

Create a CloudFlare account and add your website. Below you can see what nameservers I’ll need to use. It seems very technical, but remember that the internet is just a large network of connected computers, each with a unique IP address. The nameserver points an alphanumeric web address to a numeric IP address. It’s easier to remember words than it is to remember a series of numbers and dots.

In most cases your website files are stored on one computer, a server, in a static location. The farther away someone is from that server, the longer it takes to send your files over the network. A CDN combats this by distributing your files to hubs all around the world with the closet hub to the end user delivering the data. Browser caching helps to eliminate the number of files that need to be send while both text and image compression reduce the sizes of files that need to be transferred.

I need to change the DNS for my website from Network Solutions servers to the CloudFlare servers I’ve been given. Like all websites, mine has a unique IP address. You can get some basic hosting information about a website from services like whoishostingthis.com.

As you can see in the image above, the IP for my website is listed and my CloudFlare settings have already taken effect. But how did I make that change?

How to Change Your DNS

Network Solutions is my website host, but most hosts should have a similar way of modifying DNS settings. Navigate to the DNS settings page and replace the nameservers with what CloudFlare gives you. In my case, I replaced the existing DNS with ray.ns.cloudflare.com and vera.ns.cloudflare.com.

Once propagated CloudFlare will “interrupt” requests to my website as shown in the graphic below.

Now that CloudFlare is integrated, we can apply some additional page speed optimizations.

What CloudFlare settings should I use?

There is no one size fits all approach to optimizing CloudFlare or other performance plugins. Factors such as how your website is programmed, limitations of your host and plugin conflicts may determine which settings you can use. Some performance settings, such as JavaScript minification may “break” your website’s design or functionality.

Such was the case with this site. The Website Preloader caused CSS conflicts which caused the background image to not display. I recommend that you test each setting independently and purge the cache after each test to ensure that your website loads normally.

In the case of this website JavaScript and CSS minification didn’t cause a problem, but HTML minification did, which is not something I see often. Be sure to test your settings thoroughly to make sure you’re not sacrificing user experience for speed.

How fast is my website?

Hopefully this page loaded quickly for you, but I’m not one to assume anything. I want to see some data that supports that what I’ve done actually had a positive impact in load time.

WebPageTest.org is a good resource for checking the performance of your website.

Overall it looks like my website is doing pretty well in the speed department after setting up CloudFlare. It’s important to remember that Web Page Test is a simulation and should be used a reference. Actual speed for users may not be exact.

Document Complete			Fully Loaded
	Load Time	First Byte	Start Render	Time	Requests	Bytes In	Time	Requests	Bytes In
First View	4.323s	2.569s	3.130s	4.323s	65	413 KB	8.247s	72	760 KB
Repeat View	4.758s	1.740s	2.124s	4.758s	13	28 KB	4.770s	13	28 KB

Another test I should run is Google Page Speed. The page SEO and Social Media Consulting by Ch… got an overall PageSpeed Score of 86 (out of 100).

My goal is to make the site as lightweight (less code the better) and as fast as possible without spending too much time manually optimizing components, so for being on a shared server, I think this is overall a success. The biggest issue with the speed of this site is the “first time to byte” metric, which I don’t have much control over because of the hosting environment. CloudFlare, however has reduced my DNS lookup to almost nil.

If you are running your own Apache HTTP server, consider installing mod_pagespeed from Google. It’s an open source Apache server module that applies some of the same things you’ll see when using CloudFlare, such as JS libraries and image optimization.

Please feel to ask any questions in the comments or on Twitter @chriscountey.

Starting a Technical SEO Audit

Leave a reply

This is already the 8th post in the series of my new marketing campaign and we’re still just in the setup phase. You may be wondering when I’ll get to keyword research or when I’ll talk about link building (which I’ll call relationship building when I get to it). Fear not! I most certainly will address those critical pieces of the SEO puzzle, but right now we’ve still got on-page SEO work to do.

So far we’ve purchased a domain name, set up hosting, Google Analytics and Google Webmaster tools. In my opinion, there are 2 distinct overarching themes in SEO: the stuff you can control (on-site) and the stuff you can’t (off-site).

On-site or technical SEO is the process of making your website as user-friendly and as spider-friendly as possible by making it fast, easy to navigate and accessible to all visitors. It’s also important to use the proper code when necessary so that search engines don’t have to guess about the meaning of specific elements, such as images and videos.

Is my site indexed by Google?

I like to start a technical SEO audit by making sure the website can be discovered and indexed in its entirety by Google. It’s important to remember that behind the juggernaut that is Google is a collection of web pages. If your web pages aren’t in Google’s database (index) then any further efforts on optimizing your pages will be moot.

Read the Stanford research paper to see the vision that was to become Google.

Image Source

Google can crawl and index the web at remarkable speed, so chances are your pages are already indexed. To see what is indexed you can do a site search.

Google returned 18 results (pages) indexed, but I want to compare this against my Google Webmaster Tools data, particularly my XML sitemaps.

Google Webmaster Tools Lag Time

Google Webmaster Tools (GWT) is showing that I’ve submitted a single XML sitemap that contains only 8 submitted and 7 indexed pages. I don’t have hard data to support this, but GWT always seems to lag behind the actual index in terms of displayed data. For this reason, I recommend that you use GWT as a way to start your investigation, but don’t rely on its data alone.

Even the data between the sitemaps and the index status sections for the same website don’t match up, so be careful about taking it at face value.

The data I trust the most is what I see in the site search, which is showing that all of the pages I want indexed are and none of the pages I don’t, aren’t. This means Googlebot crawled the site successfully and respected my meta robots tags. Thanks, Googlebot!

How else can we give Googlebot and other spiders directions?

The robots.txt (I say it as robots dot text) is something spiders look for first for directions about which files and directories you do not want the spider to visit. Using the disallow function is a request, not a command, to essentially ignore the parameter behind it.

Without this file or if a directory is not included, spiders will assume any directories it finds are fair game and will attempt to crawl them. What you may not know is that there are thousands of spiders crawling the web, not just from search engines. So I’m going to take a look a the Wikipedia website’s robots.txt file to see if there are some additional parameters I want to add. (I chose Wikipedia because it probably gets a lot of spider traffic.)

I can see already that Wikipedia keeps tabs on spiders. And just because I don’t have as much content as Wikipedia doesn’t mean that I shouldn’t optimize user experience by eliminating unnecessary spider traffic.

To update my robots.txt file, I simply download the file, open it with a text editor and paste my new commands.

Here is what I intend to change my file to or you can download rename-robots (rename it to robots.txt before upload, but back up your existing file first!)

# advertising-related bots:
User-agent: Mediapartners-Google*
Disallow: /

# Wikipedia work bots:
User-agent: IsraBot
Disallow:

User-agent: Orthogaffe
Disallow:

# Crawlers that are kind enough to obey, but which we'd rather not have
# unless they're feeding search engines.
User-agent: UbiCrawler
Disallow: /

User-agent: DOC
Disallow: /

User-agent: Zao
Disallow: /

# Some bots are known to be trouble, particularly those designed to copy
# entire sites. Please obey robots.txt.
User-agent: sitecheck.internetseer.com
Disallow: /

User-agent: Zealbot
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: linko
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: Microsoft.URL.Control
Disallow: /

User-agent: Xenu
Disallow: /

User-agent: larbin
Disallow: /

User-agent: libwww
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: Download Ninja
Disallow: /

#
# Sorry, wget in its recursive mode is a frequent problem.
# Please read the man page and use it properly; there is a
# --wait option you can use to set the delay between hits,
# for instance.
#
User-agent: wget
Disallow: /

#
# The 'grub' distributed client has been *very* poorly behaved.
#
User-agent: grub-client
Disallow: /

#
# Doesn't follow robots.txt anyway, but...
#
User-agent: k2spider
Disallow: /

#
# Hits many times per second, not acceptable
# http://www.nameprotect.com/botinfo.html
User-agent: NPBot
Disallow: /

# A capture bot, downloads gazillions of pages with no public benefit
# http://www.webreaper.net/
User-agent: WebReaper
Disallow: /

SEO with Screaming Frog

Leave a reply

If you’ve been following the series you’ll remember that we created and submitted XML sitemaps in the last post. Before I move into the spider simulation, let’s check on the status of the sitemap. Navigate to your Google Webmaster tools page and make note of any messages or warnings you see there. In my case, there is nothing new to report. No news is good news!

The Google Webmaster Tools dashboard is a great tool to quickly potential issues such as crawl errors or problems with your sitemap. The middle chart depicts the number of times my site is showing up in Google search. It’s low, but that’s to be expected with a new campaign. My focus is on the right graph, which shows the status of my sitemap.

It looks like things are running smoothly in the sitemap department, but my next step is to simulate a crawl of my website using a tool called Screaming Frog.

Screaming Frog SEO Spider

The free version of Screaming Frog will up to 500 URLs (pages), but since my website is new, that’s plenty. You can download Screaming Frog here: http://www.screamingfrog.co.uk/seo-spider/. To get an idea of what Screaming Frog can do, watch this video:

Starting from my home page, as directed, the spider will hop through hyperlinks much like Googlebot or another search engine spider would. Sitemaps and crawling a website are 2 ways that spiders can discover and report on content. For this exercise I’m going to filter my results to show only HTML. In other words, my pages. Screaming Frog found about 20 pages versus the 7 pages Google Webmaster Tools is showing in my sitemap.

What happened?

Robot Instructions

Earlier in the series I used Yoast’s WordPress SEO plugin to give the search engine spiders directions. I asked them (not ordered them) not to index certain types of pages: tags and categories. I also instructed my WordPress installation to exclude these post types from the sitemap. The image below shows that even though these pages were found, the directive “noindex” is present.

Looking good! Let’s do one final check to make sure this piece of the puzzle really is falling into place. I mentioned earlier that directives like “noindex” are requests, not really commands. It’s up to the search engine to follow these requests, so I’m going to do a site search of this domain to see what Google has in its index. (Google is partly a big, really big, database.)

You can see what Google has indexed for a website by typing site:http://yourwebsite as the search string. In my case, it’s site:http://marketingchris.com.

I see that there are 14 pages indexed versus 7 pages in my sitemap. Now what happened? Google must have crawled my site prior to me updating the settings asking it to not include categories in its index. Not a big deal! I know that these instructions are in place now, so I just need to be patient and wait for the next crawl.

Or not!

Fetch as Google

There is a tool in Google Webmaster Tools for the impatient SEO, like me, called Fetch as Googlebot.

I want Googlebot to re-spider my content to see that some of my requests of what to index have changed. Please let me know if you have any questions and I’ll see you next time!

Website Index and Crawl Health

3 Replies

Creating and Submitting XML Sitemaps

Since I’m building this site on WordPress with Yoast’s WordPress SEO plugin, setting up my XML sitemap has essentially been done for me. Check the box in the SEO settings of the XML sitemaps page and boom. A sitemap! I also checked off categories, tags and format as page types I didn’t want to include in my sitemap. I’ll get into that later on in this post, but my objective is clear: allow Google to index every possible piece of content on my website, but index it only once.

Now that I’ve got my sitemap file location from Yoast, http://marketingchris.com/sitemap_index.xml, it’s time to submit it to Google via Webmaster Tools, which I set up earlier in the campaign. From your GWT dashboard, click Optimization > Sitemaps > Add/Test Sitemap

Excluding Categories and Tags

It might seem strange that I didn’t want to include my category and tag pages in my sitemap, but I actually don’t want those indexed by Google. I mentioned earlier that I only one want my content indexed once and unless I make modifications to my category and tag pages, they will have the same exact content as the posts themselves. I want my pages to work together, not compete for link juice or traffic.

Look at how similar the category and tag pages are below.

The content I really want indexed are the articles themselves, not a list of articles. Thinking of it from a user experience perspective as well as respecting the resources needed to discover and store this information may be that which separates your website from others.

Create a Sitemap for a Non-WordPress Site

If you’re using another CMS (content management system), chances are it has a way for you to create XML sitemaps easily. However, if you (or your niece) built your website in a completely custom fashion, you may need to rely on another program to do the heavy lifting.

I recommend Xenu Link Sleuth as your starting point and keep it handy – you’ll probably use it later to detect potential issues with your website. Fire it up and launch your first crawl (yep, it’s a spider!) by clicking File > Check URL.

Once it’s finished crawling your website, click File > Create Google Sitemap File and then follow the directions above to submit your new XML sitemap to Google.

During my crawl I noticed some 404, Page Not Found errors. I’ll address those in my next post. If you want to skip ahead, please feel free to contact me! Thanks for reading.

Google Analytics Query Data

On Page Factors

Keyword in URL

Keyword in Page Title

Keywords in Headings

Elements

Tag Cloud

Meta Keyword Data

Social Media’s Role

Link Profile

Analyze Ranked Pages Fast

Analyze Backlinks Quickly

Uncover Real People, Not Links

Find Local People

What is search intent?

Connect a New WordPress Site to Old WordPress Data

WordPress Important Tool

Capturing Lost Link Flow

How to Set Up Google Adwords [VIDEO]

Using Adwords Competition to Determine Content Hierarchy

Creating the Page

Campaign Update

Choose keywords by identifying your objective.

What are people searching for?

Identify Searcher Intent

Identify Potential Resources as Content

What is page speed?

How do I set up CloudFlare?

What do I need to set up CloudFlare?

How to Change Your DNS

What CloudFlare settings should I use?

How fast is my website?

Is my site indexed by Google?

Google Webmaster Tools Lag Time

How else can we give Googlebot and other spiders directions?

Screaming Frog SEO Spider

Robot Instructions

Fetch as Google

Creating and Submitting XML Sitemaps

Excluding Categories and Tags

Create a Sitemap for a Non-WordPress Site