NetToolkit-SEO

NetToolkit-SEO/1.0 is the reader behind our free SEO Sitemap Analyzer. It only runs when someone asks us to analyze a website, reads that site's sitemap and up to 500 pages, and stores nothing.

Respects robots.txt rules for NetToolkit-SEO and *.
Makes at most 4 requests at a time and reads only the first 256 KB of each page.
Never submits forms, logs in, or follows links outside the site being analyzed.

Blocking it

User-agent: NetToolkit-SEO
Disallow: /