Download
Easy Spider is a distributed Perl Web Crawler Project from 2006. It features code from crawling webpages, distributing it to a server and generating xml files from it. The client site can be any computer (Windows or Linux) and the Server stores all data.
Websites that use EasySpider Crawling for Article Writing Software:
https://www.artikelschreiber.com/en/
https://www.unaique.net/en/
https://www.unaique.com/
https://www.artikelschreiben.com/
https://www.buzzerstar.com/
https://easyperlspider.sourceforge.io/
https://www.sebastianenger.com/
https://www.artikelschreiber.com/opensource/
It is fun to look at some code that is few years ago and to see how one has improved himself. If you want to write text automatically try https://www.artikelschreiber.com/en/ or https://www.unaique.net/en/!
Features
- Client/Server Distributed Crawling
- Perl Programming Language
- Config File Support
- PDF,DOC,XLS,PPT Extraction Support
- Article Writing with https://www.artikelschreiber.com/en/
- SEO Article Generator with https://www.artikelschreiber.com/
- SEO Text Generator API with https://www.artikelschreiben.com/
- Our Web Interface: https://easyperlspider.sourceforge.io/
Project Samples
License
GNU Library or Lesser General Public License version 2.0 (LGPLv2)User Reviews
-
Easyspider is a perl client/Server architecture to crawl the web for interessting webpages. The Server can be any box that has internet access and allows perl programms to run. The client connects to the server, gets its working task, fullfills it and give the resuts as xml stream back to the server. the server then can install that xml file into a oracle/mysql/mariadb etc database or can be parsed by the sphinxsearch.com fulltext indexer to generate searchable content for your webpage. Happy Hacking ;-)
