Segmenting Websites For Large Scale SEO

gray wooden maze

SEO Projects for Large Websites Can be Daunting

When it comes to SEO at scale – websites with thousands, tens of thousands of pages or more, approaching a search engine optimization project can be intimidating. However, this is the situation that government, colleges, and other sizable organizations frequently find themselves in.

I have run into this scenario often when working with college websites and government agencies. Organic search traffic is dropping and the organization wants to bring it back up, but they’re lost at where to start an SEO project.

It can be hard to grasp the whole website structure for some sites. A recent federal agency website I consulted for has millions of pages!

Fortunately, there is a simple way to go forward.

The Key to Success With Large Scale SEO is Segmenting the Page Scope into Reasonable Pieces

As the saying goes, you eat an elephant (or the world’s largest pizza) “one bite at a time” and a massive search engine optimization project is no different. The way to approach these projects is to divide them up into reasonably sized, related pieces. Along with those “bites” of the larger website come distinct ways to locate, group together, and measure success for those pages.

So, how do you go forward on this kind of project? There are six steps below that outline the process and what to keep in mind along the way.

Six Simple Steps to SEO At Scale Projects

1. Use Your Existing Website Analytics to Find Groups of Pages for Search Optimization

If your search project is starting from scratch, there are often pages that already perform either very well or somewhat well at attracting organic visitors on your website today. Along with the good performers will also be other tiers of pages: the “pretty ok,” “decent,” and “not so good” pages.

One method I use to find out which sections of the site fall in each bucket is to look at the number of landing page sessions from organic search in Google Analytics.

This metric is a good indicator of how strong pages are at attracting visitors from the search engine results page (SERP) right to the page, without a visit to the Home page or another page in between. Therefore, it can be a good approximation of the relative search strength of pages over a period of time.

SEO at scale is difficult with extra large websites
Photo by mali maeder on

Sometimes it makes sense to focus on pages that are in one of the groups above. A couple of ways to start would be to find pages that are good, but not great, at bringing in organic visits and put them together in a group for optimization.

Another strategy is to focus in on a specific website section that is highly important to the organization. For the government agency mentioned earlier, we started with the online exhibitions section of the site.

How big should your SEO page groups be? I would aim for no more than 200 pages in each set to start. This is large enough to have a major impact but not so large that it’s difficult to keep track of.

2. Use an SEO Audit to Assess Your Current Search Engine Optimization State

After you decide on groups of pages for optimization, a good next step for SEO at scale is to perform an automated audit crawl of your website. Most of the major SEO tools out there, like SEMrush, BrightEdge, and others, can do this for you.

SEO audit tools usually have a finite number of pages they can crawl at once. For a larger website, it usually makes sense to adjust the settings for your crawl to make sure the pages covered are ones that:

  • You can actually make changes to, and
  • Have enough content that it makes sense to crawl them

Make sure to also include your focus page group. These audit crawls will take some time, so you will likely need to return for the results several hours or a day later.

3. Fix Any Technical SEO Issues That Arise from the Audit

One of the great things about running an SEO audit crawl for large scale SEO is that you will normally find several “low hanging fruit” items that are easy fixes with big impact.

One example is a university I worked with that had slow page load times across their website. It turned out that three images in the footer on almost all pages were completely non optimized and several hundred kilobytes each! Without any development time, they were able to change the graphics and drastically improve their page speed.

Other frequent items that may come up in your audit include pages with missing meta descriptions (which affect how your site looks on search results pages and how likely a user might click your link over your competitors), and other, similar, easy to fix items. Address any of these that you can do quickly.

4. Research Keywords for Each Page, And the Page Group as A Whole

SEO at scale requires careful keyword research
Photo by Suzy Hazelwood on

Next, it’s time for keyword research. Many, many options exist for getting this done. Your SEO tool should have several ways to get this done, and then there are supplemental services such as the Google Ads Keyword Tool and Answer The Public that can help generate more ideas.

I recommend creating a document with focus keywords for each page, primary and secondary, and then for the page group as a whole where there are commonalities. The common attributes could include your geographic location, the type of organization, the particular section of your site, or many other factors.

5. Evaluate Draft SEO Pages for Their Keyword Density Against Your Research

Your writer (you do have a writer, don’t you?) should be able to take your keyword recommendations and content direction to produce a new draft of each page in your optimization group.

Once you have drafts in hand, look them over closely for:

  • Keyword density: are the keywords you want to focus on mentioned enough times and spaced out enough throughout the document? A rule of thumb would be to have your primary keywords appear 5 times on average, while your secondary keywords are inserted at least twice. This is assuming that the length of your new page is at least 600-700 words.
  • Relevant cross-links: the new pages should link to your existing content several times where it makes sense to do so. This could include linking to news, your services, or anything else that’s relevant. Also make sure those links have keywordsin them and not generic words like “click here.”
  • Readability: do they still make sense and are easy to read for users, not just search engines?

6. Launch Your Updated Search Optimized Pages, And Assess Every 90 Days

Once you’re happy with your page drafts, it’s time for lift off! Update your pages with the new content. Give them time to “age in” and don’t do a performance check too early. It can often take new content 90 days or more to truly shine in the SERP.

Your SEO tools and Google Analytics can help show whether you’re seeing improvement with the new page content over time. A good pace for evaluation is to report on page performance and look for both wins and areas for improvement every quarter or so.

Look for overall organic search visits, specific keyword performance, and gaps or possibilities for further gains down the road. SEO is never static, and the rules are always in flux.

%d bloggers like this: