Indexing sources that require login
Indexing of sources that require a login works in Sitevision Crawler exactly the same way the same way as for a standard version of Nutch with one exception.
As of version 1.1 of Sitevision Crawler, it is now possible to authenticate against systems that use form-based login where the login form lacks an ID tag.
Example of httclient-auth.xml using the above function:
Did you find the content on this page useful?