Open Source Crawlers in Java

Ex-Crawler is divided into three subprojects. Ex-Crawler server daemon is a highly configurable, flexible (Web-) Crawler, including distributed grid / volunteer computing features written in Java. Crawled informations are stored in MySQL, MSSQL or PostgreSQL database. It supports plugins through multiple Plugin Interfaces. It comes with it's own socket server, where you can configure it, add urls and much more. Including user accounts and user levels, which are shared with the webfrontend search engine. With the Ex-Crawler distributed crawling graphical client, other people / computers can crawl and analyse websites, images and more for the crawler. The third part of the project is the webfrontend search engine.