Adding Task

After authorization in the control panel of the search system, you may find yourself on the the Search tab, where it is possible to test the search results and configure the ranking algorithm.
In the Task tab the possibility to add tasks is present:
  • Octopus - site scanning;
  • Worder - processing previously scanned sites;
  • Kavunka - update task (reload);
  • Runer - automatically add tasks for crawling.
Tasks can be added one by one, or you can add several site crawl tasks at once. Only three sites can be crawled at a time.
You can see the scan results in the Scan statistics table. Below you can see three windows Oct1, Oct2, Oct 3, that display the site scanning speed.
  • nit - the number of Octopus requests to the website;
  • good pages - the number of pages that were processed and found to be good by Octopus (in the future, these pages will participate in the search);
  • bad pages - pages that do not have good content or the content is non-unique or the page language does not match the selected license language.
After all sites will be scanned, you need to add a task for the Worder. Also you should add a task for Kavunka to update search results (reload).
After that, all sites that have been scanned will participate in the search. In order for Octopus to use a proxy, you need to edit the file Proxy.txt in the TFiles folder. It is necessary to add IP addresses and ports of proxy servers in this file.
Each IP address must be from new line. When adding tasks, you need to specify so that Octopus uses a proxy.
Personal Search Engine
Powerful Crawlers
Fast WEB Scraping
Try for Free