Comments on: The death of robots.txt? http://technoccult.net/archives/2009/11/09/the-death-of-robotstxt/ A General Interest Blog for Weirdos Tue, 16 Mar 2010 22:52:09 +0000 http://wordpress.org/?v=2.9.1 hourly 1 By: Klint Finley http://technoccult.net/archives/2009/11/09/the-death-of-robotstxt/comment-page-1/#comment-329157 Klint Finley Tue, 10 Nov 2009 05:05:57 +0000 http://mutateweb.com/?p=8482#comment-329157 I turn to other sources for certain searches - like Amazon, Yelp, and Wikipedia. Google has also been employing human editors to review SERPs. So yes, it seems we may be at the limit of non-human search results. Before Google got big I had taken to searching DMOZ before search engines, much the way I often check Wikipedia on a subject and look through its references and external link sections. All that said, I still use Google a LOT. I turn to other sources for certain searches – like Amazon, Yelp, and Wikipedia. Google has also been employing human editors to review SERPs. So yes, it seems we may be at the limit of non-human search results.

Before Google got big I had taken to searching DMOZ before search engines, much the way I often check Wikipedia on a subject and look through its references and external link sections.

All that said, I still use Google a LOT.

]]> By: Justin Hager http://technoccult.net/archives/2009/11/09/the-death-of-robotstxt/comment-page-1/#comment-329128 Justin Hager Tue, 10 Nov 2009 02:32:51 +0000 http://mutateweb.com/?p=8482#comment-329128 In general, I find that search engines are not very adept at finding things that I am most interested in. If I'm looking for a product review, being able to Google the make + review is helpful, but if I'm trying to find more literary/philosophic information Google increasingly just comes up with a bunch of academic journal articles for sale from about 12 different sources (same article). Could we end up bouncing back into the era where the portal and the human edited link stream is a lot more useful than a search engine could hope to be as a result of a flooding of the market with repetitions of the same data and purchasable content? In general, I find that search engines are not very adept at finding things that I am most interested in. If I’m looking for a product review, being able to Google the make + review is helpful, but if I’m trying to find more literary/philosophic information Google increasingly just comes up with a bunch of academic journal articles for sale from about 12 different sources (same article).

Could we end up bouncing back into the era where the portal and the human edited link stream is a lot more useful than a search engine could hope to be as a result of a flooding of the market with repetitions of the same data and purchasable content?

]]>