# contact: webmaster @ linux.kiev.ua User-agent: * Disallow: /index.php?id=280 Disallow: /ru/account/register/ Disallow: /ua/account/register/ Disallow: /ru/news/archive/select/ Disallow: /ua/news/archive/select/ Disallow: /ru/forum/profile/ Disallow: /ua/forum/profile/ # per-site... #Sitemap: http://www.linux.kiev.ua/?id=234&L=1 # not that much original/unique images here User-agent: Googlebot-Image Disallow: / User-agent: Yahoo-MMCrawler Disallow: / # for having stupid business model -- go elsewhere User-agent: Webalta Disallow: / # what's that? User-agent: TurtleScanner Disallow: / # per robot (due to extensions) User-agent: Slurp Crawl-Delay: 60 User-agent: Yahoo! Slurp Crawl-Delay: 60 User-agent: Teoma Crawl-Delay: 120 # should be IP-banned anyways User-agent: msnbot Crawl-delay: 1800 Disallow: /ru/ Disallow: /ua/ User-agent: UWO-InktomiSearch Disallow: /ru/ Disallow: /ua/