Page 1 of 1

Last.fm's robots.txt, Asimov style

Posted: 29 Jul 2010, 21:53
by Mesarthim
<!-- m -->http://www.last.fm/robots.txt<!-- m -->



User-Agent: *

Disallow: /music?

Disallow: /widgets/radio?

Disallow: /show_ads.php



Disallow: /affiliate/

Disallow: /affiliate_redirect.php

Disallow: /affiliate_sendto.php

Disallow: /affiliatelink.php

Disallow: /campaignlink.php

Disallow: /delivery.php



Disallow: /music/+noredirect/



Disallow: /harming/humans

Disallow: /ignoring/human/orders

Disallow: /harm/to/self



Allow: /

Re: Last.fm's robots.txt, Asimov style

Posted: 30 Jul 2010, 00:44
by Devy
Confused about the movie, Internal Affairs et cetera