Why aren’t geographic target and query parameters specified in robots.txt?

Matt's answer:

Today’s question comes from Andy in New York. And Andy asks, “Settings and webmaster tools, like geographic target and query parameters to ignore, are great. But it means other search engines won’t have access to this data. Why not propose a new robots.txt directive for these settings?” I understand the question, but you have to be a little careful about– you don’t want to just throw everything in the kitchen sink into the robots.txt file. Robots.txt predates even Google, and it’s relatively well-established, and there’s [UNINTELLIGIBLE] that .txt code that a lot of people rely on. So typically, it’s better to rely on, you know, not necessarily going in and monkeying with something that is really well understood, and is relatively simple. Google has introduced Sitemaps, which is a standard that all the major search engines support and know how to read. So new Sitemaps, video Sitemaps, and stuff like that. And we have looked at, when there’s new data to be introduced, introducing it into those Sitemaps. So for example, there was stuff about, what if you have boilerplate in one language versus content in a different language? And I think that that setting might have been within Sitemaps, for example, or at least they were talking about doing it there. So I do take the point. But especially if you can make your query parameters, your URL parameters, such that you never have to tell them to be ignored in the first place, then all the search engines benefit. The stuff that’s in Webmaster Tools is often things where we really need to know that it’s you as the webmaster. It’s stuff that’s often a little too complex, where you wouldn’t want to shoehorn it into the robots.txt file. And it’s always a good discussion to have about, where is the best place to put this. But for example, Yahoo introduced the URL parameters to ignore on their webmaster console even before Google. So you know, it’s definitely the case that if search engines want to offer this functionality, they can offer it within their site as well.

by Matt Cutts - Google's Head of Search Quality Team

Original video: