Below are details we hope will help to gain some feedback on the issue we're experiencing.
The error is reported by the IIS SEO Crawler using the System.Diagnostics.Trace infrastructure:
…
Warning: 0 : Error parsing URL:http:///support.cfm Invalid URI: The hostname could not be parsed.
Error: 0 : Error in crawler:Collection was modified; enumeration operation may not execute.
Stopping Workers
Information: 0 : Signaled all threads, all-done:True
...
After further inspection we found what happens is that the exception is caught in method Microsoft.Web.Management.SEO.Crawler.WebCrawler.CrawlerThreadProc, by a catch block that reads like this:
catch (Exception exc)
{
Trace.TraceError(string.Concat("Error in crawler:", exc.Message));
_workerQueue.Stop(true);
}
IIS SEO Toolkit is running in a VMWARE virtual environment hosted by a server with an AMD Opteron 6180 SE processor. The virtual machine has assigned 4 cores and 16GB of RAM. It is running Windows Server 2008 R2 Standard Edition (x64).
The issue is happening on more than one instance of development sites. Websites are served by a web farm running Internet Information Services 7, and are developed using .NET framework 4.
Thanks in advance for any help given!