Wrong email or password! Try one more time.

Forgot password?

An account with this email already exists.

An email with a confirmation link has been sent to you.

Did you forget your password? Don't panic. Enter your email address,
and we will email you a link where you may create a new password.

If this address exists, we will send you an email with further instructions.

Back to authentication

Why aren’t geographic target and query parameters specified in robots.txt?

Why aren’t geographic target and query parameters specified in robots.txt? - answered by Matt Cutts

Matt's answer:

Today’s question comes from Andy in New York. And Andy asks, “Settings and webmaster tools, like geographic target and query parameters to ignore, are great. But it means other search engines won’t have access to this data. Why not propose a new robots.txt directive for these settings?” I understand the question, but you have to be a little careful about– you don’t want to just throw everything in the kitchen sink into the robots.txt file. Robots.txt predates even Google, and it’s relatively well-established, and there’s [UNINTELLIGIBLE] that .txt code that a lot of people rely on. So typically, it’s better to rely on, you know, not necessarily going in and monkeying with something that is really well understood, and is relatively simple. Google has introduced Sitemaps, which is a standard that all the major search engines support and know how to read. So new Sitemaps, video Sitemaps, and stuff like that. And we have looked at, when there’s new data to be introduced, introducing it into those Sitemaps. So for example, there was stuff about, what if you have boilerplate in one language versus content in a different language? And I think that that setting might have been within Sitemaps, for example, or at least they were talking about doing it there. So I do take the point. But especially if you can make your query parameters, your URL parameters, such that you never have to tell them to be ignored in the first place, then all the search engines benefit. The stuff that’s in Webmaster Tools is often things where we really need to know that it’s you as the webmaster. It’s stuff that’s often a little too complex, where you wouldn’t want to shoehorn it into the robots.txt file. And it’s always a good discussion to have about, where is the best place to put this. But for example, Yahoo introduced the URL parameters to ignore on their webmaster console even before Google. So you know, it’s definitely the case that if search engines want to offer this functionality, they can offer it within their site as well.

by Matt Cutts - Google's Head of Search Quality Team


Original video: