When would Google show multiple results from the same site?

Matt's answer:

MATT CUTTS: OK. Today’s question comes from Vancouver. Jzbecker wants to ask, under which circumstances will Google decide to display multiple results from the same website? It’s a really fun question, because the answer has changed over the years. But the high level answer is, when we think it’s useful and it doesn’t hurt diversity too much. So let’s look at the interplay between those two factors. Let me talk about a system that we used for years and years and years, like since I joined Google, this long. There was something called host crowding. And what host crowding did is, suppose you had one result that showed up from example.com in the search results– these are search results– and another example that also showed up from example.com. So what host crowding basically said is, OK, to make things a little bit more easy for the user to parse and read through, let’s take anything that’s on that first page, and we will indent it under. So you’d have two results from example.com, and the second one would just be indented a little bit underneath the first one. That makes it easier for the user to visually group those results and know, oh, these are both from example.com. It’s safe to say that we’ve done a bunch of different experiments. Host crowding lasted for a long, long time. And it was really, really nice because you’d often have a see more results from example.com. So you could click that, and you can do an even deeper dive. But it was also very nice because it ensured diversity. That is, if you had seven or eight results from GeoCities, sometimes that can crowd out other good results. And so you could see a couple of the best ones from GeoCities, but then you’d also get diversity. You’d get the rest of the results showing all sorts of nice things along those lines. Now what’s the limitation of that? Well, host crowding, if you only allow two results, that’s kind of a hard limit. Sometimes it might be more appropriate to show more results. Sometimes it might be appropriate to only show one result. And host crowding was a little bit limited in that respect. So that was one thing to consider. The other thing is it also messes with the UI just a little bit. Because once you get used to the indentation, it’s really nice, but it limits your ability to do other sorts of things. It limits your flexibility a little bit. So it was very nice for the time. It was a radical improvement over many search engines that might return 60 results from one site by grouping them all together. But we’ve sort of evolved in our thinking over time. So the sorts of things that we have done after host crowding include deciding to show multiple results. So maybe you’re willing to show a few or even several results. But then over time, it gets harder and harder to see results from that particular host name. And that can be kind of useful, because if you search for antique green glass, you might get eight results from one site. And if that’s the best site in the world to get results from, then that’s fine. You might want to see that. But if it’s not really a great site, as far as you don’t want eight out of the top 10 to belong to that site, then if it gets a little bit harder for each subsequent result to show up, then you’ll get you tune it, so that you get as many as you think is useful. And then you start to get diversity after that. And you start to get good stuff. Just to jump back to host crowding just for a second. There was also the fact that host crowding, by definition, is on hosts. And so at least for a while, if you had a different subdomain, then you had a different host name. And so a few people would try to make multiple subdomains on the same domain to try to get around host crowding and try to get as many results as they could. So for a while, people used to think, oh, I’ll just make a whole bunch of subdomains, and I’ll get around host crowding. And that’s the sort of thing that is still not a good user experience, because it hurts the diversity for users. So let me talk a little bit about the way that we think about diversity and the number of results you can see from one website right now. You want to show as many results as you think is useful. And that’s the tricky bit. What the user is looking for can vary depending on what they’re searching for. For example, if they type in something like HP or IBM, probably a lot of pages or a lot of results from HP is a good answer. So several people have noted that it’s possible to get more than two, more than four, lots of results from Hewlett Packard if you search for HP. But that’s OK. The user has indicated that’s their interest by doing that query. But in general, what we try to balance is this trade-off between a good diversity of results, because you don’t know exactly what the user was looking for. So you want to give them a little bit of a sampling to say, OK, here’s a bunch of different possible interpretations. Here’s what you might be looking for. And then we also want to absolutely give the results that we think match the query well. And sometimes that can be from multiple pages within the same site. So there’s always a tension. There’s always a trade-off in trying to figure out what is the best set of search results to return. There’s no objectively true or perfect way to do it. We’ve varied our scoring. We’ve varied our user interfaces. And if there’s one thing you can count on, it will be that Google will continue to test out ideas. Google will continue to evolve how often we think it’s appropriate to show how many results from how many sites in the search results. But that gives you a little bit of the insight about what we’re thinking about, why it’s sometimes a good experience, and why sometimes it’s not as helpful for users to see a lot of results from the same site. Thanks very much.

by Matt Cutts - Google's Head of Search Quality Team

Original video: