Crawling & Indexing

Setting up the crawler

Getting started with the Searchine crawler.

Read more

Add & update content

Read more

Multiple domains support

Read more

Crawling frequency

Read more

Exclude specific URLs

Read more

Searchine Page Snippet

The page snippet is used to pass extra information to Searchine regarding the crawled page

Read more

Robots meta tag

Searchine supports the robots meta tag.

Read more

Add allowed URL parameters

Read more

How to configure synonyms & stopwords

Read more