Written by Neil Patel on September 1, 2015, Shares his deep insights to concentrated research
Competition can be scary, especially when it comes to SEO.
When trying to rank for a keyword, you might be going up against a junior intern, or you might be going up against an experienced SEO company.
Typically you will be competing with other SEOs around your own skill level, although that’s not always the case.
The good news is that, if you take the time to do comprehensive keyword competition analysis, you’ll be able to identify keywords that haven’t been targeted by the most skilled SEOs in your niche.
While it will take you a lot of time up front, it will save you much more in the long run.
Assuming you’re producing high quality content, you’ll have a much easier time getting it seen and getting it ranking highly on Google and other search engines.
You will need exponentially fewer backlinks, which will save you either a ton of time or money.
In truth, there are only a small amount of low competition keywords in a niche at any given time (new keywords pop-up as others disappear), but that’s all you need.
Ranking for just a handful of low volume, but low competition keywords will get your organic search traffic started. Traffic has a tendency to grow exponentially.
As you build your authority and trust with visitors and search engines, you’ll be able to start ranking for more competitive terms down the line without any major new investment.
I’m going to show you a comprehensive method of analyzing keywords to target. None of it is too complicated, but it is a lot of work. Be prepared to put it in now and it will save you effort in the long-term.
Why no competition analysis is perfect
Before we get going, there’s something important that you need to learn.
Keyword competition analysis is an estimate of the competition for a keyword, but it’s not a science.
Just because a keyword appears to be easy, or easier than another keyword to rank for, doesn’t mean it will be easy to rank for in all cases.
Yes, competition analysis is very useful. It will give you a good idea of what you’re up against and where your opportunities are.
However, remember that your results won’t always be accurate. There are 2 main reasons for this.
First, no one knows what Google is thinking: While we understand which ranking factors are most important for the most part, we can’t exactly quantify them.
Your competition analysis is a reflection of how you think Google ranks sites. But even teams of very smart analysts have been unable to recreate the Google ranking algorithm.
This means that our competition analysis methodology isn’t 100% accurate.
In addition, Google constantly changes their algorithm (about 500 times per year).
So even if you’re able to perfectly predict the competition level for a keyword today, it might be a bit off in a few weeks or month.
Secondly, there’s a tradeoff: When doing keyword analysis, you will always face a tradeoff between efficiency and accuracy.
The more ranking factors that you try to take into account, the more resources (time and money) you will need to do your competition analysis.
If you only take into 1 or 2 ranking factors into account, you can do competition analysis quickly, but it won’t be too accurate. As you add more ranking factors into the equation, you start to get more accurate results, but it takes more time.
It’s up to you to decide on a good balance between efficiency and accuracy for your situation.
The most important SEO factors to consider
The main strategy behind keyword competition analysis is to look at how the top rankings fare when it comes to the most important SEO factors.
For example, one of the factors we will look at is the number of backlinks. If a page has 0 backlinks, it’s likely to be easier to beat than a page with 100 or 1,000 links.
When we do this for several factors, we are able to see if a keyword is “low competition” or “high competition.” Then, you can decide whether or not to target that keyword.
Our first step is to decide on which factors to consider in our competition analysis.
To do that, let’s turn to SearchMetrics 100 page report of the most important ranking factors.
This report’s data consists of over 100,000 different search engine result pages (SERPs). The team looked at different potential ranking factors to see if top ranked sites tended to score higher (in regards to these factors) than low ranking sites.
Below is an image excerpt of the factors with the highest correlations, which means that top sites tended to have the most of these factors:
One key thing to remember is that correlation does not equal causation.
What this means is that just because top ranked sites had a lot of Facebook likes and engagement, doesn’t mean that Facebook likes and engagement causes you to rank highly.
In fact, in that specific case, it’s more likely that it doesn’t. Sites that rank highly get more traffic, which likely leads to the boost in social sharing.
In addition, sites that have large social followings are usually large sites that already have a lot of domain authority, so they naturally rank higher.
Are any of those correlations useful?
Yes, because some are legitimately caused by the factor helping sites to rank better.
It’s our job, as SEOs, to test each factor individually and figure out which ones do and don’t help.
For years, we’ve known that backlinks help. However, Google has come out and stated that social signals do not affect rank in the past. Some case studies have seen temporary ranking increases from social signals, but it’s not a factor I would suggest focusing your attention on at this time.
For the purposes of this post, I’m going to look at the most important factors that are known to help pages rank. If you’d like to add more on top later in your own analysis, then you’re welcome to.
Factor #1 – Backlinks: When a site links to another site, it counts as a “vote” for the site being linked to. That’s nothing new. We know that backlinks are a key ranking factor, and they need to be part of any analysis.
However, we need to look at them on a few different levels, which is where it can get a bit tricky for beginners.
First, we need to consider that backlinks are important on both a page level (links to the exact page), and a domain level (total amount of links to all pages on the domain).
Second, we need to consider that not all links are created equally. Links can have different value based on which page they are located on.
To analyze backlinks, you need a backlink database tool. For a serious analysis, you’re going to need a paid plan to one of the best tools. I recommend Ahrefs or Majestic. They are by far the 2 most comprehensive backlink database tools.
When you want to analyze a specific page (from a SERP), you’ll simply plug it into the textbox on one of these tools.
As an example, let’s say that you saw my beginner’s guide to online marketing in a search result, and decided you wanted to see how hard it would be to outrank it.
Typing the URL into Majestic reveals that there are 232 domains that link to that specific page, and over 4,600 total backlinks from those domains.
That’s quite a bit for any single page.
In addition, you also need to check how strong the domain is in general. By switching the dropdown beside the URL, or changing the URL to the root domain, you can see all the links to Quick Sprout:
Just about 16,000 referring domains, and well over 1 million backlinks.
So that’s how you look at quantity, but how do you look at quality?
One high quality link is worth hundreds or thousands of low quality links, so it’s important to not just go by numbers.
You could examine the quality of each link individually. A high quality link is:
- on a page that has a lot of links to it itself
- relatively high up on the page
- found naturally in the page’s text (surrounded by appropriate description)
- is on a relevant page
- is on a page without too many links (link power is divided by number of links)
In other words, quite a bit goes into it.
It would be impossible to evaluate this for every single link.
Luckily, link database tools have a pretty good solution for us. They algorithmically try to determine the quality of each link. It’s not perfect, but it’s pretty good.
On Ahrefs, every page and domain get a score:
- URL rating: A score that represents the overall quality and quantity of links pointing to that specific URL, on a scale from 0-100.
- Domain rating: A score that represents the overall quality and quantity of links pointing to anywhere on the domain, also on a scale from 0-100.
Majestic is slightly different. It uses 2 metrics:
- Trust flow: A score purely based on the quality of links to the page you enter.
- Citation flow: A score based on the quantity of links to the page you enter.
In general, citation flow will be a bit higher than trust flow, but if it’s more than about 1.5 times higher, it’s likely that the page has a lot of low quality links.
Instead of manually checking the quality of every link to a page, we’ll be using these metrics (or similar) for a quick check.
Factor #2 – Relevance: When it comes to search results, relevance is the most important factor. When someone searches for “yellow tables”, they will be disappointed unless they find a page about yellow tables.
A long time ago, relevance was mainly determined by having the exact keyword in the domain, title, and body of the page.
However, Google is now great at picking up synonyms, along with user intent.
So now, if a searcher types in “yellow tables”, Google knows that they probably want results where they can buy a table, not a fluff article about what a yellow table is. That’s user intent.
In addition, Google would also include synonyms for either yellow or table in the results. So you might see results for “golden tables” or “yellow stands.”
This is a factor that you will have to assess manually. I have not come across a reliable wide-source automated tool for this.
We can look at basic keyword density, but it’s very difficult to understand user intent and include the right synonyms without manually looking at a page.
Let’s walk through a quick example.
Pretend you were searching for:
“content promotional tactics”
Google will bold synonyms in meta descriptions and URLs. In this case, we see that Google knows that “content promotion strategies” means essentially the same thing as what we searched.
If we’re assessing the top few results for relevance, our main question is: “how well does this satisfy someone who searches for the keyword?”
If the answer is, “not very much”, you can probably outrank it.
So here’s the second result from that search:
This page has “50 promotional tactics”.
And while that’s fairly comprehensive, I feel that the quality could be improved. The picture above is of the first 2 tactics, which aren’t really promotional tactics at all.
They are also not very data-driven and there’s no clear cut examples or walkthroughs, which the visitor would probably appreciate.
Overall, it is relevant, but I think it could be improved on significantly.
If you come across a keyword where the first 3 results are basically perfect answers for the query, don’t bother trying to outrank them, it’s going to be very difficult. However, that’s also a rare situation.
Factor #3 – User satisfaction: This is related to relevance, but there are a few differences in how we will evaluate it.
It also needs to be done manually, but we can look at a few different factors to determine how much users typically like the page.
Since we can’t see things like bounce rate and time on page, we need to rely on public information.
First, we can start with how many social shares it has. A page that everyone loves will have a decent amount of social shares. So if we see a page with few shares (in niches where people aren’t embarrassed to share), we know that it’s probably not fully satisfying searchers.
On most sites, you can see the share count displayed somewhere prominent:
But if you can’t find the share count, use a share count tally tool.
Just type in the URL when prompted and submit it:
It will pull up the number of shares from the most popular networks:
The second place that we’re going to look is the comment section (if there is one).
If people are saying things like:
- “Amazing post!”
- “This changed my life”
- “This is the best post on (topic) I’ve ever read”
Then they’re likely satisfied. On the other hand, if there are a lot of complaints or suggestions, most visitors probably left the page unsatisfied and went back to the search results.
Look for both the number of comments, and what’s in the comments themselves.
Factor #4 – Do you consider all your visitors?: Google has made it clear lately that it wants you to optimize your website for your visitors.
With the recent mobile-friendly update, and preference given to fast loading sites in search results, it’s clear that Google wants mobile friendly and fast pages in their results.
If the top results are not mobile-friendly and load slowly, it’s an indicator that Google has to rank a page that it doesn’t really want to. Unfortunately, there are no other pages of the same content quality, but that are faster and responsive. You could fix that.
This is another manual check, so you won’t need to do it for every search result. But it can be used as a final check before you make a decision to target or not target a keyword.
First, check if the page is mobile friendly by using Google’s own mobile-friendly test:
Put in the URL of the page and click analyze. It’s a simple pass and fail test.
Next, check site speed using a site speed tool like Gtmetrix. Paste in the URL and click “analyze.”
After a quick scan, you’ll get a performance report for the site. Pay special attention to the “Page Details” box:
In this case, the page isn’t slow, but it’s not fast either.
In general, a fast loading page loads in under 2 seconds and only has a few dozen requests. If you see a page that takes 4+ seconds to load, there’s a real opportunity to beat it.
Tools to help you do this faster
By now you might realize that it would be next to impossible to do a large scale keyword competition analysis without the help of some automation, which is where tools come in.
There are hundreds of tools out there to do this specific job, some better than others.
Most of them work similarly, so I’ll walk through a few so that you can understand how to use them, and how they work.
Tool #1 – Term Explorer: This tool can be used to find keywords, but also to analyze their competition, which is what I’ll focus on.
Your first option, once you create an account, is to run a bulk keyword job:
You enter one or more keywords, and it will give you a list of results anywhere from 1,000 to 90,000, depending on your account type and choice.
Once you run the job, you will get keyword results along with search volume data. You can easily filter the results according to keywords or search volume.
If you find a few keywords that you like (or many), you can check the box beside them, and then click the blue button at the top to send them to the keyword analyzer:
Alternatively, you can enter in keywords from other sources into the keyword analyzer directly.
You will get a similar report this time, but you will see an overall “difficulty” score from 1 to 10. This is based off the 3 categories to the right: relevancy, link strength, and trust.
A higher difficulty means that it’s more difficult to rank for the term. With this particular tool, a score under 3.5 or so is “easy”, while 3.5-5.0 is “normal”, and above that is difficult.
Tool #2 – KWfinder: This is another tool similar to Term Explorer. To generate keyword ideas, enter a seed keyword into the only box on the page:
It will quickly generate a list of keywords (I believe based on Google’s Keyword Planner).
You’ll also note that each keyword will have an “SEO” score beside it:
The scores range from 0-100 and are color coded (green is easy, red is hard). If a score isn’t showed by default, you will need to click the magnifying glass.
In addition, you can get a more detailed look at the competition behind any of the keywords.
If you click a keyword, the search results for that keyword will pop-up on the right. It shows data from Majestic (trust flow, citation flow), and shows you an SEO score for each search result.
The average score is the one you initially saw.
This is important because average scores can be skewed by one or two results. If there are some very low scored pages at the bottom of the first page (maybe they’re temporary), they could make the keyword look much easier than it really is.
Tool #3 – Moz Keyword Difficulty tool: A third tool for keyword competition analysis that you can use comes from Moz.
It’s one of the many tools that comes with a Moz Pro subscription.
Type in any keywords you’d like to analyze into the tool’s main text box.
It will bring up a quick report that shows you the keyword difficulty as a percent (maximum 100).
To get more detail, you can click the “view” button under the basic SERP report column:
The bottom section shows the same information as the top section, just in a bit more detail.
Note that the page authority and domain authority here come from Moz’s Open Site Explorer, which isn’t as reliable or complete as the other link databases we’ve looked at.
Tool #4 – Ahrefs Toolbar: This tool isn’t an automated competition checker. However, it can be used to quickly assess the strength of an actual SERP.
When you search for a term in Google, the toolbar will load a small bar under each result:
It shows link information from the Ahrefs database on a page and domain level. In particular, you can look at the URL rank (“UR”) and the domain rank (“DR”).
Step #1: Gather your keywords
Now that you have a good idea of what competition analysis consists of, and some of the tools that you can use to simplify the process, I want to walk you through, step-by-step, of how to actually do it.
Before you analyze the competition of keywords, you’ll need to gather a rather large list of them. It’s more efficient to do competition analysis in bulk.
Keyword research is an important skill by itself, and it’s something that you should spend some time learning before doing competition analysis. Here are some guides that will help fill in any gaps in your knowledge:
Try not to take the easy way out when you’re doing keyword research.
If you just plug in a popular keyword like “content marketing,” you’re going to get the exact same keyword list of thousands of SEOs before you.
Using some of the creative methods in those guides above, you can find “hidden” keywords that fewer people are targeting.
When less people are targeting a set of keywords, you’re more likely to find some low competition keywords ripe for the picking.
Step #2: Start filtering out keywords
Once you have a large list of keywords, it’s time to start assessing the competition of them.
Unless you have an experience eye (and even then it’s difficult), you’re not going to be able to reliably pick out low competition keywords without a thorough assessment.
This means you have 2 options:
- a manual assessment: You could manually review all of your keywords. However, unless you have a few weeks of free time, this isn’t really feasible.
- an automated/manual hybrid assessment (recommended): You can use tools to get started and find the highest competition keywords. Take these out of the results and then manually review the more promising keywords.
I would hope you’d pick the second option.
Run your keywords through a tool I showed you above (or similar tool) so that you can get an estimate of the competition:
It’s important to remember that this is just a quick estimate. There are many ways that the results can be skewed.
If you’re creating a ton of content, you might be able to just target all the lowest marked keywords, and then see which pages are ranking the easiest. However, most people need to be more selective.
You can typically check the box beside a keyword and save the selected ones to a private list. If not, save the most promising ones manually to a spreadsheet.
The idea here is to take out any keywords that are obviously going to be too difficult to target.
Step #3: Dig in deeper
By the end of that process, you should have a much smaller list of keywords. I recommend aiming for between 10 and 20% of what you initially started with.
Give the list a quick look over and take out any keywords that don’t make sense for your site.
Once you have a final list, the hard work begins.
Your tools have told you that every query you are left with is relatively easy to rank for based on the most common metrics, which is usually a combination of domain and trust authority.
Now, you get to see how easy they are to rank for from a user’s perspective.
For each keyword, search it in Google, but you need to make sure it isn’t personalized.
Go to “http://Google.com/ncr”, which is the global Google search (the “ncr” prevents redirection to your local Google). Make sure you’re in a private (or incognito) browser and you aren’t logged in to any Google account.
You can choose how many results to look at. I recommend starting with the top 3, and if you’re not sure if the keyword is low enough competition, continue with the next 5 to 7.
For each of the results for each keyword, you want to evaluate the following things:
- how many (and what quality) backlinks point to the page
- how many (and what quality) backlinks point to the domain
- if I was a user who searched this keyword, would I be fully satisfied with this result?
- does it load quickly?
- is it mobile friendly?
You can also add in any other Google ranking factors to your analysis as you’d like, but it will take you more time.
Step #4: Make a decision
After you analyze each keyword, you have to decide if it’s worth going after or not.
Unfortunately, there’s no magic metric or tool that will help you this.
To make this decision, you need to first consider many factors:
- How strong is your domain? (How many links point to it, what’s its domain authority?)
- How easily can you get backlinks? (Do you have experience in SEO? Do you have a network of contacts to ask for links from?)
- What level of content can you produce? (Do you have the skills or financial situation to create the best content?)
At some point, the authority, trust, and relevance of the page you create for a particular keyword, needs to exceed all other results that you have examined.
The stronger your domain, the easier it is to rank.
The easier you can get quality backlinks, the easier it is to rank.
The bigger the budget you have for content, the easier it is to produce something that no one else can match. Not only does this make it easier to rank short term, it also makes it easier to stay there.
Based on these factors, you have to determine if a keyword is worth going after during your manual review.
Keyword competition analysis is not a science.
While you can get some information from a tool, you’re also going to have to apply some SEO expertise of your own to find low competition keywords.
This is going to take some practice and time. The good news is that you’ll save that time, and so much more, if you pick your keywords carefully.
The easier the keywords that you target are to rank for, the more consistent your results will be.