Large-scale platform · Technical SEO
Designing search systems for a directory with millions of pages
Template-level search strategy for a national directory where a small implementation decision could affect an enormous population of pages.
The engagement in context
An in-house SEO leadership role at a national directory. The challenge was to make technical decisions at template and platform level rather than optimise pages one at a time.
At directory scale, every local fix is really a product decision
As SEO Lead for one of the UK’s largest online directories, I worked across more than 20 million pages. A canonical rule, internal-link module or template change could improve an entire section or create millions of weak URLs.
The job was therefore to identify patterns, understand how crawlers moved through the platform and decide which changes belonged in the underlying system.
I treated crawlability and discovery as an information architecture problem
The work covered crawl-budget management, directory and location structures, internal linking, redirects, structured data and Core Web Vitals. Analysis was segmented by page and template type so aggregate numbers did not hide the real behaviour.
Content workflows were designed around reusable rules and meaningful local information rather than attempting to optimise pages manually. That connected search performance with a more useful directory experience.
Aggregate reporting was not enough to diagnose the platform
A directory can look healthy overall while entire page families create crawl waste or fail to earn useful visibility. I segmented evidence by template, location depth, category, indexation state and organic contribution so the behaviour of one large population did not hide another.
Log and crawl evidence helped show where search engines spent time. Search and analytics data showed which structures created value. Looking at both prevented the team from treating crawl volume as success or low traffic as proof that a page type had no purpose.
The strongest recommendations changed rules, not individual pages
Work focused on the system points with the greatest reach: which combinations should create URLs, how location and category paths should connect, when pages should consolidate, and how canonical, redirect and internal-link logic should behave.
That also meant resisting manual optimisation as the default answer. At this scale, a sustainable improvement needed a template, data or product rule that could make millions of pages more consistent without pretending every location had the same evidence.
Scale changed how evidence had to be communicated
Recommendations needed to explain the size of the affected population, the likely upside or risk, and the implementation point where a team could intervene. That made prioritisation possible for product and engineering teams managing a very large estate.
The result was a more useful way to prioritise platform SEO
The team could judge work by the population affected, the rule that needed to change and the evidence that would show whether it helped. That made technical SEO easier to plan alongside product and engineering work than a backlog of disconnected page-level fixes.
The experience changed the unit of analysis I use today
It taught me to distinguish a broken page from a broken rule. A single example is useful for demonstrating a problem, but the real question is which population shares the same cause and which system creates it.
That perspective still informs my work with marketplaces, publishers, international estates and any organisation where the right template-level decision matters more than a long list of individual fixes.
