Full Speed

[ February 24, 2006 ]

IceRocket is Dumb

I was watching some logs today, just to see how much traffic was being forwarded from this site’s former domain, and I found this interesting bit:206.196.125.91 - - [24/Feb/2006:13:20:53 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:20:53 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:20:53 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:20:54 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:21:07 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:21:07 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:22:32 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:22:32 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"

And then, a few minutes later, there was this:
206.196.125.91 - - [24/Feb/2006:13:25:51 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:25:51 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.101 - - [24/Feb/2006:13:26:04 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.101 - - [24/Feb/2006:13:26:04 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:26:08 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:26:08 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:26:09 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:26:09 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:26:09 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.91 - - [24/Feb/2006:13:26:09 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.101 - - [24/Feb/2006:13:26:09 -0600] "GET /index.xml HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"
206.196.125.101 - - [24/Feb/2006:13:26:09 -0600] "GET /index.rdf HTTP/1.1" 301 247 "-" "BlogSearch/1.1 +http://www.icerocket.com/"

It’s bad enough that IceRocket’s blog search is still pulling these URLs, which have been 301’ed for a long time now, but it’s really annoying when the crawler sends back to back requests, ignoring the 301 each time. The 301 status code has been a part of HTTP for quite some time. Why can’t a search engine get this simple detail right?

Possibly Related:

© 2014 Scott Johnson
• •