What search teams learn when a large product aggregator barely gets indexed
A large product aggregator can give Google more to crawl and still end up with very few indexed pages, and a new public thread lays out why. The site in question was crawled 1.17 million times across a few weeks and still showed only four indexed URLs in Google Search Console, with the homepage among the unindexed pages. The diagnosis points to two concrete problems: a cluster of indexed URLs that redirect straight to the homepage, and the general quality concerns that come with building an aggregator on programmatic, auto-generated content at scale.
What the crawl and index numbers looked like
The owner shared their Google Search Console data publicly, and the contrast is sharp. Across the last few weeks before the post, Googlebot had crawled the site roughly 1.17 million times. The page indexing report, against a catalog of about 180,000 pages, showed four URLs in the index. The homepage itself was not indexed. Crawling at that volume, paired with an index of essentially zero, is the kind of mismatch that signals to a search engine that something is structurally wrong rather than that the content is being held back by chance.
Why the redirects matter
The first issue raised was technical: the small set of URLs that did make it into Google’s index all redirected to the homepage. A search engine that follows a chain of indexed URLs and lands on the same destination page starts to lose trust in the structure of the site. If every indexed entry point collapses onto one page, the engine has no reason to keep a wide and varied index surface for the domain. Fixing that means resolving the underlying redirects so that product or category URLs resolve to themselves and offer their own distinct content, instead of folding everything back to the root.
Why programmatic SEO is the harder fight for aggregators
The second issue goes deeper than redirects. Building a large product aggregator is not trivial to do or maintain well. At scale, the catalog has to be populated from feeds and templates, and a large share of the URLs end up being programmatically generated. The challenge then becomes twofold: convincing a search engine that the auto-generated pages are worth indexing, and convincing a real user who lands on one that it is worth their time. Google’s stated position, repeated in the thread, is that the company is generally not a fan of programmatic SEO as a practice. Aggregators that rely on it have to clear a higher bar on content quality, differentiation, and trust signals than a site of a comparable size built by hand.
What this means for a team running a large catalog site
Three practical takeaways follow from the thread. First, audit which URLs are actually resolving to themselves and which ones collapse to a parent page, since indexed redirects are a fast way to flatten an index surface. Second, treat programmatic page generation as a quality system rather than a publishing shortcut: templates, data sources, and unique-content layers need oversight. Third, expect the bar to be higher for aggregators than for smaller sites, and plan crawl budget and indexing work around the assumption that scale alone will not earn trust.
How to tell whether your aggregator has a redirect problem
Open the page indexing report in Search Console and look at the sample of indexed URLs. Pull each one and check the HTTP response chain. If a high share of indexed entries resolve to the same destination, redirect cleanup is the first order of business before any content work begins.
What separates a programmatic page that earns indexing from one that does not
The pages that get indexed and stay indexed tend to offer something a user could not get by searching the underlying product directly: structured comparison, aggregated reviews, pricing history, or a category view no single merchant provides. Pages that simply republish the merchant’s own description tend to be treated as duplicative and earn less visibility over time.
FAQ
Why is a large product aggregator only getting four pages indexed after millions of crawls?
Because the indexed URLs that exist all redirect to the homepage, and the rest of the catalog consists of programmatically generated pages that Google is not yet convinced are worth keeping in the index. Both problems need to be addressed before indexing can scale.
Is Google against programmatic SEO for product aggregator sites?
Google’s stated position is that it is generally not a fan of programmatic SEO, and aggregators built that way have to clear a higher bar on quality and distinctiveness to earn indexing.
What should a team do first when an aggregator has very low indexing?
Start by checking which indexed URLs redirect to the homepage and fixing the redirect chain, then review the programmatic templates and data sources that generate the rest of the catalog for thin or duplicate content.
Related coverage
- Google March 2026 Core Update: What Your Business Rankings Need to Survive
- E-E-A-T: Why Google Now Rewards Real People Over Corporate Websites
SEOScanPRO
SEOScanPRO has the rank tracker that runs a full technical audit of a site and shows the measured result behind every check. Open the rank tracker.
Want help with this in Myrtle Beach or Long Island? Talk to us.
NewSunSEO
Grow in Google and AI Search
AI-SEO, local search, paid ads and websites from one team, with offices on Long Island and in Myrtle Beach.
- AI-SEO
- Local AI-SEO
- AI Consulting
- PPC
- Web Design
- Web Development
- Long Island Office(631) 320-1700
- Myrtle Beach Office(843) 492-7131
Request a Free Proposal
Tell us about your business and where you want to grow.
We will reply by email to talk through your goals and what a proposal would include.
We use your details only to reply to you and send what you asked for. See our Privacy Policy.
