Keyword Difficulty: How to Analyze SEO Competition?

One of the most challenging tasks in blogging or SEO is doing keyword research. For a fresh blog, you cannot simply target highly competitive keywords. You should focus on keywords that are easy to rank or should I say keywords that have a low keyword difficulty.

Well, yes, in this article I will talk about one of the most controversial and confusing terms in SEO.

It is “Keyword Difficulty” or “Keyword Competition”.

I will try to help you understand this simple yet complicated term in the easiest way possible. To explain this, I have prepared a lot of examples and snapshots that should really open up your mind.

Also, note that this is not a keyword research guide, I won’t be showing different ways to find keywords (that’s for some other post). I will be simply explaining What is Keyword Difficulty and how you can easily analyze the SEO difficulty of any keyword.

This article is specially written for absolute starters who often fall in the trap of SEO tools. 

What is “Keyword Difficulty” in SEO?

Keyword difficulty is a metric or a measure that tells you how difficult it is to rank a particular keyword on the first position of Google Search Result Page (SERP).

If I say that a keyword has a high Keyword Difficulty, it will simply mean that the keyword is really competitive and not easy to rank.

If you are planning to start a blog, your ideal move will be to hunt for keywords with low keyword difficulty.

Simple, right? Let me help you understand the factors contributing towards this term.

Factors that Contribute to Keyword Difficulty?

There are many factors that can determine the difficulty of keywords, but the most common ones (in my opinion) are;

1. Keyword Intent

The “Keyword Intent” means what the keyword/search term actually asks for? For example, If I search “My IP Address”, I will see Google itself showing my IP address on Search Page.

This adds difficulty as such keywords are not only difficult but not really worth targeting. You will see a lot of sites targeting this exact keyword but with Google showing the result, those sites are essentially competing against Google on Google Search Page.

Irony, huh!!!

2. Backlinks

“Backlinks” is the core factor when it comes to SEO. For a keyword, if you see that sites are ranked with the help of  “quality backlinks”, it is a good hint that the keyword is difficult to rank.

3. Competitors

Competitors or other search results are also a factor to consider while determining keyword difficulty. A first page filled with big authority sites tells a lot about the difficulty of the keyword.

4. Content

Content also plays a vital role in keyword difficulty. If you see the first page filled with quality content, then you can assume that a lot of work is already done by others (or your competitors). This makes it more challenging for you and eventually, the keyword becomes more difficult.

5. SERP Volatility or Movements

SERP Movements also tell a lot about keyword difficulty. If there is no movement it also indicates that the content is satisfying what the reader needs. So if you find the same first 10-20 results or minimum movements, it can be an indication that the keyword is difficult to rank.

For example, the keyword “on page SEO” has no SERP Movements which indicates that already very strong articles/pages are ranking for the term.

6. Current Condition of your Blog

Last but not the least, if you are doing keyword research for an existing blog of yours then the current condition/authority of your website also determines the difficulty of a keyword.

For example, The keyword “Coronavirus” is extremely difficult to rank (tools would also say 70+) but if Wikipedia wants to rank for this term, the competition will be less for them.

I believe that these 6 factors are the key to understand the concept of keyword difficulty.

You now know what is keyword difficulty and the factors behind it. Hurray!!!

All of this sounds so simple, right?

Well, not really!!

Here comes your frenemy!!

SEO Tools!!

Keyword Difficulty (KD) Score on SEO Tools

Every SEO Tool in the market has its own “Keyword Difficulty” Metric. Things get confusing when you start seeing different or wrong results on every other tool you try.

For a keyword, Ahrefs might be showing 10 KD, Moz might show 50, while Semrush may show 80%. Every tool claims that they show the most accurate difficulty scores. So, Which tool to trust?

In Simple words, STOP LOOKING AT KEYWORD DIFFICULTY ON SEO TOOLS. Or Don’t Rely on them entirely.

These tools are still evolving but they can never hit the sweet spot. If you trust these tools, you might end up making a list of keywords that are not easy to rank.

1. Ahrefs Keyword Difficulty

Ahrefs says that it focuses on the number of referring domains. The more referring domains for top results, the more will be the difficulty on Ahrefs Scale.

Here is the scale of Ahrefs;

To be honest, Ahrefs KD score is the worst among all SEO tools.

Can You Rely on Ahrefs for Difficulty?

NO!!

2. Semrush Keyword Difficulty

Here’s what Semrush says about its difficulty score;

To calculate keyword difficulty, SEMrush takes into consideration the authority of the domains that are showing up on the results page and then estimates how hard it would be for a new website to outrank its current competitors on the SERP.

They do not mention the metrics clearly, but their scores are comparatively better than Ahrefs.

Should You Trust Semrush Keyword Difficulty Percentage?

Not entirely, but you can get an idea and match with your own research.

3. Moz SEO Keyword Difficulty Tool

Moz takes into account the Domain Authority (DA) and Page Authority (PA) of top search results to come up with a difficulty score.

In the SEO world of Fancy Metric, DA and PA are still considered very reliable.

Here is what Moz says about their tool;

Should You Trust it? 

It is better than Ahrefs and can be used to get an Idea about the SEO competition. 

4. KWFinder Keyword Difficulty Tool

KWFinder by Mangools claims to have the most accurate SEO keyword difficulty tool.

KWFinder evaluates each URL on the 1^st SERP by using the Link Profile Strength (LPS). The overall Keyword Difficulty is based on the LPS scores of all the URLs.

The LPS of KWFinder is based on the link profile, Moz, and Majestic metrics.

According to me, KWFinder does a decent job of calculating KD. 

5. UberSuggest SEO Difficulty Tool

It was launched as a Free Tool but was made paid recently. If you are planning to buy an SEO tool, this is not worth buying (at least for now).

UberSuggest normally checks the first 20 search results for the keyword to come up with a keyword difficulty score.

According to UberSuggest, the keyword “On-Page SEO” has difficulty of just 18. Don’t rely on this tool.

There are many more tools like Majestic, LongTailPro, SEOPowerSuite, Alexa that have their own keyword difficulty scores. You will always find different scores by each tool, thus making it more confusing for a newbie to decide whether a keyword is difficult or not.

Let me show you some examples that shall convince you to stop relying on keyword difficulty scores of various SEO Tools.

Comparing KD Scores from Various SEO Tools

For research, I tested and compared more than 50 keywords and found that these tools show common scores for the majority of the high volume keywords. This is a good indication but I still noticed a huge difference KD scores for some long-tail keywords or low/medium volume keywords.

Keyword: Cool Office Gadgets 

Here are the results from 4 big SEO tools;

1. Semrush shows 74%

2. KWFinder shows “Still Easy” with a score of 23/100

3. Moz Shows a difficulty of 40

4. Ahrefs gives the keyword difficulty of just 8 to this keyword. 

So, you can clearly see the KD score ranging from 8 to 74 on 4 popular keyword research tools. 

I checked KD scores for some more keywords, here are the results;

Most of these keywords are actually difficult to rank. However, some tools suggest otherwise.

The tool that you should not use for checking KD is Ahrefs. I cannot reveal some of my keywords, but I can assure you that Ahrefs has the most inaccurate scores among all.

If you ask me to pick the best tool, I would pick SEMRUSH. Most of the Scores from Semrush falls close to my manual research.

Manual Research!! Yes, that’s what you will learn in the next section.

How to Analyze SEO Keyword Difficulty Manually?

The best way to understand the SEO difficulty of a keyword is to analyze the search page manually.

There can be many ways to analyze the competition. I will be mentioning a few common methods along with the method that I generally follow while doing keyword research.

1. Solid Method to Analyze SEO Competition for a Keyword

You must already be knowing about this method. This is all about analyzing the first 10 or 20 search results.

In simple words, analyze different SEO factors of ranked pages and come to a conclusion. This entire process takes time but gives you a very clear picture of the SEO competition of the keyword in hand.

While following this method, you should consider the current state of your website. After all, a keyword that is easy to rank for my website can be really difficult for your website.

So, If your website is brand new, you need to pick keywords with very low difficulty. While if your site has some age and performs decently, go for some medium difficulty keywords or according to the authority of your website.

There’s no fixed number or metric that you should be targeting. Just analyze the factors on the first page manually and decide yourself.

Remember one fact that SEO is all about Experiments and Experiences. 

Here’s what to do;

Search the keyword and analyze the following factors on the first 10-20 positions of Google results;

1. Backlinks

What is the average number of backlinks on the top 10 ranked pages? How many domains are pointing to the root domain of the first 10 ranked results? Are they quality links or are they forum, comment, or spam links? Can you replicate those links? Can you build similar or better backlinks?

If you feel that the backlinks of ranking pages are less and of low quality, you can consider working on the keyword.

You will come across many keywords where there might be 0 backlinks pointing to the ranked page. It doesn’t indicate that the competition is low. That page may be ranking due to the overall authority of the website. You can see a clear picture when you analyze the backlinks of other 9 results.

If you see all low-quality sites on the first page of Google, then you can come to a conclusion that the keyword is easy to rank. (However, you must check the other factors)

2. Content of the Ranked Pages

What is the average word count of the ranked articles? Can you write a more detailed/in-depth article? Are the ranked pages according to the intent of the reader? Are they actually providing what the user needs? Or are they confusing the reader?

Can you write a better piece of content?

If you feel that most of the first 10-20 results are of low quality and you can do better then go for the keyword. It might be easy to rank.

You must also note that content alone cannot help you rank. In many cases, you can see low-quality content ranking on Google for many months just with the help of backlinks. 

3. Overall State of the Ranked websites

What kind of websites are ranking on the first page? Are they big giants or are they niche sites or are they starter sites? How old are those websites? Does your fresh site stand a chance among them? Does your 1-year-old site stand a chance against them?

If you see big giants dominating the keyword, it might be a tough keyword to rank. Always look for keywords where you can see at least 1-2 positions occupied by small sites. After all, your first target will be to outrank those small sites and then compete with the big sites.

4. Check the SERP Volatility for the Keyword

Search the keyword on Ahrefs or any SERP History Checker and see how often do the rankings change for that keyword. More changes might mean that the ranked pages are not properly satisfying the intent of the users.

Here is the Search Position History of a keyword that has a huge volume (Over 30000) but heavy volatility on top 5 positions. I managed to rank #1 for the keyword on Feb 2020 and it still sits strong at #1.

So, yes, SERP Volatility shows a good indication of keyword difficulty.

These were some common factors that you must consider while determining the difficulty of a keyword manually. Tools can never answer the above questions and that is why it is recommended to do a manual check instead of blindly picking a keyword that your favorite tool recommends.

Oh btw, don’t just check one factor and leave the rest, if you want to master the art of analyzing Keyword difficulty, you must make sure you follow and analyze every contributing factor.

You can use various chrome extensions to speed up this process.

If you are an Ahrefs user, you can use Ahrefs SEO Toolbar to can see the backlink status of all ranking pages on the Google SERP itself.

If you want a free tool, you can use SEOQuake Extension that fetches backlink status from Semrush (you will have to connect a free Semrush account). It also shows age and some other important metrics.

I have not mentioned any specific or exact number to follow since there’s no fixed number as such. The faster you understand this, the better will be your SEO journey.

2. iftiSEO Method or My Method (Fast Process – Best for Bulk Research)

This method is very fast and requires only a few seconds or a minute to get a clear idea about the keyword competition.

For this, I take into consideration the following metrics;

1. Age of the Websites that rank on the first 20 positions of Google
2. Alexa Rank (Don’t judge me, I will explain :p )
3. Number of Indexed URLs of Ranking Sites (For an Idea, not a serious metric)
4. Referring Domains/Backlinks
5. SERP Movement/Volatility

The basic idea behind my method is;

The more number of new sites (by domain age) ranked on the first page, the easier will be the keyword difficulty. 

There’s more to it but most of it revolves around this idea.

I believe that, if new sites are able to come on the first page of Google, then I can also come up there. A keyword becomes easier if more than 1 new site is ranked on the first page.

Lastly, if there are heavy movements on the first page of Google, it is a good sign for me. But to be honest, I do not check volatility for every keyword as it is a time-consuming process.

All of this can be easily checked on SERP itself using SEOQuake Extension. It shows the number of indexed pages, Alexa rank, Backlinks and Age (Wayback age with Whois button) of all the search results.

So, A keyword is relatively easy if the first page has;

1. More than 3 Websites that are aged less than 1 or 2 years.
2. More than 3 sites with an Alexa ranking higher than 500,000 (Such sites are generally not so good in terms of authority)
3. More than 3 websites with less than 200 indexed pages. (I consider this mainly when I look for keywords that are targeted by long term blogs with 100s of posts)
4. More than 2 new pages entering the first page over the past 3-6 months. (You can use Ahrefs for this purpose)

I refine these numbers accordingly, but the whole idea is to pick keywords that are dominated by new sites or small sites or not so good/well-performing sites. 

Following the exact strategy, I have taken many of my fresh domains from 0 to 1000 page views per day without building any contextual or quality backlinks.

In fact, for some keywords, I ended up at #1 without building a single link to the entire website or the post.

So, for me, the keyword difficulty is EASY when I see new or fresh sites dominating the first page. The more new sites on the first page, the better will be your chance to outrank them. How to find those fresh sites? Well, for now, you can read the Sweet Steal Technique, later I will come up with a detailed guide on how to find keywords for SEO.

3. Double Quotes Method + KGR Technique

Many SEO experts promote this technique as the best way to find and figure out low competition keywords. I don’t want you to solely depend on this method.

So, how does this work?

In simple words, check the number of search results when you search the keyword with an allintitle operator.

Like this;  allintitle:keyword

The less the results, the better is your chance to rank for that keyword. In my opinion, I would say “keyword competition is easy only if the search results are very less (less than 30)”.

You can also search the keyword within double quotes to get a broader idea about the SEO competition.

In both cases, the SEO keyword difficulty is easy if the search results are less. However, I do not recommend you to rely only on double quote results or allintitle results to figure out the competition.

KGR which stands for Keyword Golden Ratio is the next step to evaluate whether a keyword is easy/worth targeting or not.

The formula to calculate KGR is in the picture below.

If the KGR is less than 0.25, the keyword can be easily ranked on your website or at least hit top 100 as soon as they get indexed. A KGR of more than 1 means that the keyword is highly competitive.

The basic idea behind KGR method/double quotes method is that the keyword difficulty will be easy if fewer sites are targeting the exact keyword (in the title). This makes sense but it still doesn’t speak much about the first page results. What if the KGR is less than 0.25 and even Double quote results are also less than 100 but the first page is filled with big authority websites? That’s where you need to be extra careful.

Do you want me to write a simple guide on the KGR technique with examples? If I get 2 comments asking for it, I will start working on it. 😉 

Key Takeaways

Most of the things I mentioned in the article are known by everyone but SEOs are sometimes very lazy and do not spend time doing proper research.

Even the newbies end up relying on SEO tools or they simply watch random YouTube videos that promote KD scores given by the tools. This is really bad for them as they fail to understand the basics.

Anyways, here are some takeaways from this article;

1. Never Rely on Keyword Difficulty (KD) shown by various tools like Ahrefs, Semrush, KWfinder, Moz, etc. The tools are improving slowly but not yet perfect.
2. Always analyze the SERP (Search Engine Result Page) manually.
3. Your Wife is not My Wife. Similarly, an Easy keyword for me might not be an easy keyword for you.
4. It is good to match your findings with the metrics shown by SEO Tools. Tools can sometimes give you an idea. 😉
5. There is no solid metric or number to follow while analyzing keyword competition. Even the SEO tools follow different algorithms for keyword difficulty.

I hope I successfully decoded Keyword Difficulty for you.

Now go ahead, do some manual research, and share your experiences.