Google penalties can leave website owners feeling stranded and losing a lot of money. One website owner came to me for help after learning that their site wasn’t even ranking for its own brand name. The site, an industry leader with a 10-year old domain, was under an algorithmic penalty.
The site was
ranking well until 2015 when traffic started to decline.
You know the story: they hired multiple SEO agencies over the years, these agencies exported SEMRush reports, and the site’s traffic remained low. When the they hired an SEO Agency or so-called “Google Penalty Removal Agency”, they hoped to get the penalty removed. These guys worked on the site for six months with no success.
A company this big relies on a healthy stream of traffic and leads to their site. Six months is a lot of lost:
So, they came to me for help. It took eight weeks of work, but luckily I was able to remove the penalty and get the site’s traffic back to previous levels.
The steps I
took are steps that every site owner can take to recover from a Google penalty.
automate some of the tasks and keep me organized during the recovery process.
Since the site had over 50,000 pages at the start, I definitely needed a few
tools to help me make sense of the site, its backlink portfolio and to provide
a good foundation for a penalty recovery.
Week 1: Learning the Site’s History and Tracking Changes
has a past, and with a ten-year-old site, it’s easy for common issues to go
overlooked. A broken link or redirect issue may be noted by one team member and
never be corrected. I decided to build a foundation for the site and make sure
that the site’s main issues were corrected first.
I took seven
main steps during the first week.
1. Crawl the Site with Screaming Frog
Screaming Frog takes a lot of the manual work out of the equation. I decided to run the program to:
Conduct a link audit
Find broken links
Find redirect issues
Find site errors
Review page titles
Review meta data
Find duplicate pages
version works fine for sites with less than 500 URLs, but there is a paid
version with a lot of extras, including crawl customization, AMP crawling and
validation, and other options that are not available on the free version.
You can find most of the main issues that I’ll be talking about with the free version, if your site is small.
2. Added Search Engine Updates as Annotations in Google Analytics
Google pushes a lot of updates and algorithmic changes to hold their position as the top search engine. What I wanted to do was find a way to add all of these updates into Google Analytics so that I had a clearer picture of what happened to the site’s traffic and when.
Search Engine Watch has a great article on how to add annotations and all of the search engine updates to Google Analytics.
I did this was:
Annotations allow me to quickly see what major updates correlate with the site’s drop in traffic.
To determine which of the site’s issues were caused during major updates.
able to visualize the site’s traffic, it was a lot easier to make update-specific
changes to the site in an effort to remove the site’s penalty.
3. Track My Own Actions in Annotations
It’s easy to forget every step that you take to correct a penalty. I wanted to have a clear way to know which actions I took and when. I decided to track all of my own actions through annotations to have a strong history of the steps were taken to remove the Google Penalty.
4. Identified Broken Site Code
When looking through Screaming Frog, I noticed that there were a lot of pagination issues for all of the site’s URLs, especially in the blog section. I discovered that each “new” page had issues with:
Duplicate content of the main page
For example, /page/ and /page/12 would have the same content. There was broken code that was producing this issue, so I tracked it down.
5. Corrected Broken Site Code
I was able
to correct the issue by fixing the broken coding, but you may need to hire
someone to do this part for you. Even after I fixed the broken code, I found
that there were thousands of these pages still indexed in Google search.
These pages should drop off eventually.
If these URLs have a common prefix, you can use the Google Search Console’s “Remove all URLs with this prefix” feature:
6. Updated All Broken Links
With the pagination issue corrected, I decided to open up Screaming Frog again and update all of the broken links that were found. Some links were removed, and some were updated.
Every time I ran a crawl, I used Excel to load in all the data and compare to the previous version of the site. My report page logged and showed me a number of issues, including but not limited to:
Number of pages crawled
Number of 301s
Number of 404s
Number of other errors
Number of pages for each section of the site (blog, main pages, pillar content, etc.)
Number of pages without meta details
Number of index/noindex pages
7. Updated All 301 Redirects
step that I took was to update all of the 301 redirects on the site to make sure that they
worked properly and went to the appropriate pages. You’ll have to look through
your .htaccess files or use a plugin (depending on your CMS) to make all of
performed all of these tasks, I decided to run Screaming Frog again to make
sure that there were no issues that I missed. If you find any issues that you
missed, we’ll go through an additional stage in week two to correct these.
Week 2: Major Site Cleanup, External Link Removal
Week two, I
sat down and went through the site even further. Each step in this week’s
removal process is rather straight forward.
1. Removed Unused Development Subdomains
often create subdomains to test a site and any updates to the site or coding
before pushing the changes to the live server. I found a few subdomains that
were left behind by the developer.
The subdomains were all indexed by Google, potentially causing duplicated content issues.
I decided to remove all of these subdomains and started the deindexation process on search engines.
2. Cleaned Up More Broken Links and 301 Redirects
The unnatural links and redirects that needed to be corrected after the first week’s work were corrected this week. When you have a major site with over 50,000 pages, you may not be able to fix all broken links or redirects in a week.
3. Action Tracking and Screaming Frog Progress
Screaming Frog every day. I would also:
a summary page
data from export
I wanted to
be able to track my progress, so I made sure to run Screaming Frog to make sure
I fixed any major errors along the way. I also used Excel to track each action
I was taking so that I had a clear way to identify everything I did.
My initial crawl of the site had 50,000 pages, and during week two, we were down to 2,000 pages (remember the pagination issue). I still had quite a few pages to remove at this point, so I continued correcting any duplicate or unnecessary pages on the site.
4. Cleaned Up the Site’s External Links
The site had
a lot of external links, and a lot of links needed to be removed. I went
through each page on the site to remove any of the following external links
that didn’t make sense:
homepage links (many overused)
I left all
of the external links on the site that made sense, but I made sure to remove
any links that no longer made sense to have on the site.
5. Conducted a Full Content Audit
a full content audit that I would use in subsequent weeks to further clean up
the site. Screaming Frog already provides you with a list of all of your
content, so you can use this list for your audit.
Export the results of the list to CSV or XLS, and then start to analyze your pages.
You’ll want to make note of pages that have very thin content or pages that no longer make sense for the domain to publish. You can easily do this in Excel and use it in week three.
6. Site Speed and Lighthouse Reports
Google wants sites to be snappy, and if they’re not, it will impact rankings and user experience. I ran the site through PageSpeed Insights and created LightHouse reports so that I could better understand the site’s overall speed.
You’ll find that both of these tools will provide you with insights and suggestions to improve your site’s speed. You can also integrate Google’s PageSpeed Insights API with Screaming Frog.
Work on your
site’s speed until your reports are sufficient.
7. Removed Firewall Blocks Blocking Google Bot IP’s
through the site, I noticed that WordPress and Cloudflare both manually blocked
some of Googlebot IP addresses. I removed all of these firewall blocks to
ensure that all of Google’s crawlers could properly crawl the site.
You can run a reverse DNS on the IPs, or you can follow a general Googlebot IP address list to find any IPs you may be blocking that belong to Google.
Week 3: Corrected Issues Found in Week 2 and Started to Tackle Backlinks
I did a lot
of preliminary work in the first two weeks and started to get a feel for the
site I was working on. A lot of key issues were corrected, and I did a lot of
the groundwork for the weeks to come.
week is where I started to correct a lot of the issues I already knew existed
on the site from last week’s research.
1. Removed Low-Quality Pages and Pages with Little Traffic
audit from last week allowed me to remove all of the thin, low-quality pages on
the site. I removed all of these pages and also removed any pages that:
Were created more than two months ago
Had less than 10 visitors in the past year
through each page on the site and removed all of these low-quality pages one by
But I also
looked at all of the pages I was going to remove to find any pages that had
valuable backlinks. I used Ahrefs for this so that I could leverage
some of this link value in the next step.
2. Removed Additional Broken Links
pages on the site caused a few broken links to pop up. I ran Screaming Frog to
correct all of these broken links that were linking to the pages in the first
step of week three.
3. 301 Redirected Removed Pages That Had Valuable Links
I removed a lot of pages from the site, but a lot of these pages still had considerable backlinks to them. I decided to 301 redirect these pages to maintain some of these valuable backlinks.
Remember, don’t just 301 everything to the homepage, always do the redirect to related content or related pages.
4. CMS Cleanup
The site was using WordPress, but the site you may be working on could use any other CMS. What I did was look at the theme and found that it was:
I noted all of these generation issues and also removed any plugins that:
massive amounts of code
little purpose to the site
slow down a site drastically, and removing them helped clean up a lot of code
and speed issues.
5. Correct Site Speed Issues
provides in-depth information that I passed along to the development team to
correct. The team followed all of the recommendations from Lighthouse to speed
up the site considerably.
6. Time to Tackle Backlinks
always going to be a major focal point when trying to recover from a penalty. I
decided that it was time to dig into the backlinks that the site had because I
was going to be working on them extensively in the coming weeks.
I used two main
tools to help me:
both of these tools, it will be quick and easy to find backlinks that are toxic
to your site.
the disavowed links file, which had over 2,000 links listed. I started to look
into all of the URLs individually to learn more about the links that were
disavowed and potential links that I would add to the list in the coming weeks.
7. Deleted Pages That May Get the Site Considered as YMYL
and YMYL really allows us
to better understand how Google distinguishes between high-quality and
low-quality content. YMYL stands for: Your Money or Your Life.
YMYL was a
major issue when I decided to look through historical pages on the site.
that many of these pages mentioned specific drugs, and I was very worried that
these pages may be considered YMYL.
The backlinks to the site also had a lot of links from controlled substances pages, so I asked the company if they still deal with these substances. Since they no longer dealt with the substances, it was safe to delete all of the pages that may fall in the scope of YMYL.
some of these pages and the links to them may have been seen as:
I knew that
the coming weeks would include a lot less groundwork, and a lot of steps to
really try and lift the penalty off of this client’s site.
Week 4-6: Backlink Cleanup, Structure Data Issues and Citation Work
next few weeks, I decided it was time to work on backlink cleanup. I had been
adding URLs to the disavow, but I noted that there wasn’t any recovery as a
result. I was trying to recover anything at this point, but the site didn’t
even budge for branded keywords, such as “company name.”
Glassdoor and Yelp, along with a few other popular sites, started to dominate
the search results, pushing the website’s rankings down even further.
A lot of the
work done during these few weeks included:
1. Tedious Backlink Cleanup and Disavow File Updates
removed the disavow file, but I found that the site’s rankings dropped a few
positions. When the site’s rankings started to fall, I uploaded the disavow
file again and started the time-intensive backlink cleanup.
through all of the sites, added any sites that were toxic and updated the
days, I would upload an updated disavow file. Combing through all of the
backlinks took up a lot of time, but it was an important part of the entire
process. A lot of sites that were once authority sites were now toxic.
2. Structured Data Issues Fixes
Structured data helps Google display your website with search appearance elements. Google has a really good introduction to structured data if you’ve never heard of it before.
When you use
structured data properly, it helps organize and optimize your website. Search
engines will “read” this data and display it accordingly. For example, a lot of
recipe sites will use this structured data to have richer snippets on Google
that include ingredients, pictures, ratings and other useful information.
I found that
the site had a lot of issues with structure data.
through the site to correct these issues. With the structured data cleaned up,
I was confident that the site would have “richer” displays in the search
3. Citation and Social Profile Updates
The site had a lot of citation links and social profiles that linked to their site. Some of these links pointed to the non-www version of the site and some linked to the “http” version rather than the “https” version.
I wanted to
eliminate the unnecessary load on the server from the 301 redirects that these
What I did was
make sure that all of these citations and social profile links were linking to
the site’s “https://www” version of the site.
Week 7-8: Penalty Recovery and Rankings Restored
Google has a
lot more resources and processing power than I have at my disposal. When I was
going through my tedious backlink checking tasks, I realized that Google
already knows that these sites are toxic.
I decided to
delete all of the disavow files and trust that Google knew what
backlinks were toxic already.
The site had
four main properties:
I removed the
disavow file from all four of these properties and let Google take care of the
When I first
started to clean up the website, it had over 50,000 pages. A lot of these pages
were created with erroneous pagination code. Some pages had thin content or
content that simply didn’t make sense for the site to have any longer.
The page total
after the entire site cleanup was 400-500 pages, with:
on-site issues corrected
speeds increased by at least 50%
data cleaned up
LinkDetox is a tool I relied on heavily to find good and bad links in my disavow files and in the site’s backlink portfolio. When I removed the disavow files, I needed Google to check all of these backlinks again and really allow Google to evaluate the backlinks on its own.
Boost® really helped in this case because it helps sites recover from Google
penalties faster. The disavow only works when Googlebot recrawls the links to a
site. The Boost tool nudges Googlebot to check out these links again and value
links that actually make sense.
After a few
days, Google re-indexed pages and started its own evaluation of the backlinks.
about a week before the site finally reappeared in the search results for its
own brand name. The site was in position two in the search results, and
a day later, was in position one for its own brand name.
The penalty was finally lifted.
results took a few additional days to catch up to the regular search results,
but they eventually did catch up.
happy. I was happy. The client was happy, and started to recover to traffic
levels they hadn’t experienced in years.
What to Check When Doing a Penalty Audit
performing my penalty audit, I had a long list of items that needed to be
checked. If you’re performing a penalty audit, I recommend following my penalty
audit checklist below:
Broken site code
Social profile links
If you know you website received a manual penalty then you should check the reason in Google Search Console, fix the issues and communicate the updates back to Google Search Quality Team for further review. Submit a reconsideration request when you have fixed the issues.