What is caffeine indexing?

Matt's answer:

MATT CUTTS: Today’s question comes from Manvesh in India. Manvesh asks, hi Matt, is Caffeine indexing really helpful for SEO? What is Caffeine indexing exactly? So Caffeine was essentially this switch from a batch method of indexing to an incremental method of indexing. So the way to think about it is if we had to crawl a billion documents, and maybe we did every day before we would start to index the very first document that we crawled that day, that can cause a lot of latency. It takes a lot longer for that document to be indexable or indexed so that someone can search over it. So the switch to Caffeine was essentially saying the first time we crawl that document, let’s go ahead and index it and put it into our website results so that people can search over it within a few seconds. So the latency goes way, way down. Instead of waiting for like a day for your average document to be able to be searched, now you’re talking about really just a few seconds after it’s been crawled and indexed. So it’s a really nice thing in terms of freshness. The documents became on average something like 50% fresher. So a lot of people might not really notice like the individual documents being fresher. But as a whole the indexing gets a lot better and the search quality becomes a lot better. Because you can search for breaking news and find it without having to wait quite as long. So that’s roughly what Caffeine was we. Were very excited about it. When you pair that with Google Instant which allows you to sort of see the results as you’re typing, and then preview so you can see the results before you even click on them. So you don’t have to click and go all the way to find out if this is the right side for what I need. All of that is designed to try to make sure that you can find the results you want much faster. And if you don’t have to spend as much time looking, you can use that time for something else.

by Matt Cutts - Google's Head of Search Quality Team

Original video: