A federated search engine for frequently updated Webpages. Although Google is great for large and small static sites that change infrequently, its method sucks for frequently updated small sites. Why? Google doesn't know when a site has been updated. The result: out of date results and added expenses for needlessly crawled sites. This was very apparent during 9-11, when… Continue reading A federated search engine for frequently updated Webpages. Although Google is great for large and small static sites that change infrequently, its method sucks for frequently updated small sites. Why? Google doesn't know when a site has been updated. The result: out of date results and added expenses for needlessly crawled sites. This was very apparent during 9-11, when the need for up to the minute news was vital.
A better system is one where a site informs the search engine when it is ready to be crawled (even 10 times a day if it is active). Dave already announced