Site Monitoring

Monitoring all pages on a web site

Scheduler - Spider Monitor

If you have a web site you will want to know that all the links are functioning, including links to pages within the same site and also links to external sites.

Site Vigil lets you scan a whole site, following all the links it finds until it has built a complete map of a web site. As it scans is keeps track of the time to access the pages and where all the pages are referenced from.

You can spot if a page is missing from a web site and also if pages or graphics are no longer used (not referenced by any other pages). To configure automated scans of a site you need to enter the start page which will be used as the initial point to keep track of the pages it finds. You can limit the number of pages to be scanned and the time interval between each scan. If you want to check not only the internal pages on a site you can also choose to check that the referenced external pages are accessible by selecting the External linked resources option. Normally the spider will not download large files such as executable files, if you want to check these files and monitor how long they take to download select the Check download speed option. If you want to look at the site scan data after it has been made or compare with previous scans you must choose the appropriate storage option.

If you choose to save the scan information in a map file you can view it at any time by clicking on the Show Map button. The map is shown in the Site Vigil Analyser utility. To compare the results of the two most recent scans press the Compare Maps button, this will show the differences between the last two site scans.

The list contains a summary of recent spider scans of the web site that have been made, to look at the detailed results of any scan double click on the entry in the summary list.

An alert will be raised for each problem it finds with a web site, these will be also listed in the Summary report.

Any automatic spider scans will take account of the directives in the Robots.txt file or the META tags at the head of HTML files.


© Copyright Silurian Software 2001 thru 2012