What We Learnt from Scraping & Analyzing 1000+ Local SERPs
Local search is here to stay.
According to local marketing statistics, 46% of all searches on Google are local searches and 75% of users never scroll past the first page of search results.
All marketers know the importance of having a solid local SEO strategy backed up by real data in place.
Unfortunately, SEO and data don’t always go hand in hand. They should, but it isn’t always the case.
We decided to fix that.
Disclaimer: We’re not data scientists, but we have some technical chops and our curiosity got the better of us.
Over the past few weeks we’ve run a series of SERP checks with a focus on local terms, collecting any and all data Google would give us.
It looked a little something like this:
Here’s a summary of what we learned running this experiment:
- Higher rankings in local/maps results are linked to strong organic rankings for those same SERPs.
- Local Businesses need to double down on content & links.
- High amounts of directory/lead gen sites in a particular SERP leave less room for local businesses in page 1 organic results
- You should be out there getting even more reviews on your GMB listings right now.
Keep reading to learn more details about the findings:
About This Study
We selected 32 different queries from a variety of professions and did searches across 30 different cities of varying sizes across the US & Canada. Just shy of 1000 unique queries run.
Our primary goal for these tests was to determine whether or not there was any link between local (maps/3 pack) ranking positions and organic ranking positions.
While collecting this data we were also able to collect a few other metrics including review counts and ratings for local listings.
With the keywords and cities selected, we ran multiple rounds of searches for these across the US and Canada. Our system scraped the top 100 organic search results for each query, the top 3 local listings, as well as the top 20 or so listings from the ‘more places’ button below the local pack.
Our tech was nothing fancy, the tech stack isn’t actually too interesting:
- Beefy Server (40 CPU cores)
- LAMP Stack (Linux, Apache, MySQL, PHP)
- A lot of proxies
Some custom code glues all of the pieces together – mostly php, some bash scripting, and curl to pull the results, stored in a database (MariaDB), and then crunched with some additional custom PHP.
Having in-house developers means we get to do some fun things with scraping. Our devs used a modified version of our Citation Audit tool to collect these results using a variety of proxies and scraping techniques.
Limitations and Challenges
Our biggest limitation with this test was localization.
We opted for keyword + city queries (i.e. “maid service seattle”) as these gave us some consistency to test against across so many locations. We didn’t want to completely rely on the accuracy of the IP geolocation data for the proxies we used, nor Google’s own localization URL operators.
These tests don’t account for deep localization/personalization of SERPs that you’d get running these on a signed-in mobile device at different points within a city.
We focused only on keyword + city queries with search volume (as reported by keyword research tools like Ahrefs Keyword Explorer) to help ensure these are local terms that actually get searched.
Over approximately 24 hours per run, we scraped the local and organic listings for all of our chosen queries.
Our system parsed the results and tagged each domain found in the local listings and organic listings with a position number. Along with position information we also collected the total review count and rating of each local listing.
Matching up domains and comparing the position data of their local versus organic listings, we were able to average the results across locations, keywords/professions, as well as an overall average. We did the same with our GMB review count & ratings data.
Comparing the output of multiple runs and doing manual spot checks to ensure the accuracy of the data, we’re confident that we got an accurate picture of the SERPs detailed.
Going a step beyond the position data, we also decided to use our new list of domains to determine what types of sites ranked in the organic results for these local terms.
Our primary point of interest here was a comparison of actual businesses versus directory and aggregator site results, i.e. how many sites like Yelp, AngiesList, and other similar directories took up space in the organic results compared to actual businesses.
Okay, so now you know how we got the data, here are the actual findings:
Local & Organic Ranking Correlation
We looked at the relationship between local and organic rankings.
From our results, averaged across all keywords/professions and cities searched, we found a direct correlation between position in local results and a higher ranking page in organic results.
Take a look at our top 5 breakdown:
|GMB Rank||Org Rank|
Which leads nicely into average organic position vs GMB position:
We also went further and established rankings for additional map placings, utilizing the ‘more places’ results listings to give positions to these hidden results.
We found the organic/local correlation held true through most of the top 20 local positions.
However, in some cases, position 4 and 5 were outliers, like in this example of optometrist searches:
We believe has something to do with those positions being “borderline” – living just on the edge of showing in a top 3 results local pack, sometimes teetering in and out as Google’s algorithms test different results.
So, what does this mean for marketers?
Higher rankings in local/maps results are linked to strong organic rankings for those same SERPs.
With on-page SEO coming in as the #1 local seo ranking factor, this isn’t exactly new or groundbreaking but the ranking correlation across so many industries did surprise us a bit.
Local Businesses need to double down on content & links.
If you’re focused on local rankings then you should go all-in on your organic strategy once the local foundations are squared away (GMB optimization, citations, generating consistent customer reviews, etc)
Business Domain vs. Local Directories
While reviewing results we also looked at the discrepancies within organic search results, looking at presumed local business domains vs directory listing sites like Yelp, AngiesList, etc.
We found on average 4 out of 10 results in a local SERP were for a business directory, leaving 6 out of the top 10 results for an actual business to rank.
This varied between SERPs, based on how many larger directories and lead generation sites were competing for the traffic.
As you’d expect, high dollar services like general contractors had a higher percentage of directory sites taking up organic search positions.
With general contractors we saw only 22% actual business domains across all locations, meaning a whopping 78% of the SERP was taken up by directories.
On the flip side, the lowest amount of directories could be found in SERPs like auto glass repair with only 23% of results owned by directories.
As one might expect, this also had an impact on the average rankings for those sets of SERPs.
A higher percentage of lead generation domains taking up the rankings pushed the average for actual businesses down – with the top 3 local/map ranked businesses sitting at: 13.3, 26.9, and 28.7 in organic results.
Auto glass repair terms, with the lowest directory/lead gen competition, saw the top 3 ranked in local/map results sitting at: 11, 11.5, and 10.8 in organic across all locations.
High amounts of directory/lead gen sites in a particular SERP leave less room for local businesses in page 1 organic results
This can easily push average organic rankings for those in the local pack down to page 2, 3 or further back due to the high authority competition in front and/or differing SERP intent based on the profession.
Local Rankings vs. Ratings & Reviews
As part of our data collection we also gathered a lot of information on ratings and review counts for the top 20 ranked business in local/map results.
We weren’t able to dig into details like the content of the reviews or keywords mentioned, just taking a look at the overall correlation in local rankings to the star ratings and review counts across all ~1000 queries.
What we found is that while review counts can vary, the top rankers on average almost always had both the largest volume of reviews as well as overall star rating.
Across all queries we found an average rating of 4.25 and an average review count of 137.43 for the top ranking position in local. This breaks down for the lower positioned GMB listings as follows:
|GMB Rank||Review Rating||Review Count|
You should be out there getting even more reviews on your GMB listings right now.
Both quantity and quality of reviews are important, never stop pushing for more.
We set out to test some ideas we’ve been thinking about for awhile and were just curious to know if the raw data confirmed our personal experiences in SEO and some of our hunches.
With Google’s secret, ever-evolving algorithm the results from tests like ours will never be 100% conclusive.
What we found was an interesting correlation that, more often than not, matches local/map ranking positions with organic result positions (as compared to others in the same local SERP).
What we took away from this is that standard organic SEO tasks are maybe ever-so-slightly more important than some of us may have previously thought.
What should we do next?
There were many lessons learned in the pursuit of the data you see above. We had quite a bit of fun gathering and crunching the numbers in this experiment, so this is far from our last study on local and organic SERPs.
We obviously have many areas where we can improve and a bunch of ideas for other interesting data points we could gather and compute.
Some standouts include:
- running a much larger experiment – collecting many many more data points.
- bringing link volume/authority data from Ahrefs/Moz/etc into the equation.
- perhaps testing some more of Google’s advanced parameters to do searches from multiple locations within a specific city.
Our question to you is: what would you like to know?
If you had the ability to look at a wide swath of search results and compile it together to test a hunch, what hypothesis would you put to the test first?