preGadget

Web site policy makers who use robots. txt files as gatekeepers to specify what is open and what is off limits to Web crawlers have a bias that favors Google over other search engines, say Penn State scientists whose study of more than 7,500 Web sites revealed Googles advantage. That finding was surprising, said C. Lee Giles, the David Reese Professor of Information Sciences and Technology who led the research team which developed a new search engineBotSeerfor the study.

Original Source!, Click here for detail story!