Need flexible, region-aware, and reliable web crawling? Norconex + Oculus Proxies has you covered.
Set up Java
Configure Environment Settings
Set JAVA_HOME
JAVA_HOME
. For the Variable value, provide the full path to your installed JDK directory
.
Confirm by clicking OK.Set up Norconex
Configuration
Norconex → Examples → collector-http-config-reference.xml
.
Open this file using a text editor such as Notepad for example.Within the file find the <httpFetchers>
and </httpFetchers>
tags, and place the necessary configuration code between them.Verify Configuration