How does Searchine index my site ?
Searchine uses self made spider technology to crawl your site and analyse the documents structure and text.
We will take notice of hyperlinks to other pages / documents and follow those as well.
The crawler will take notice of noindex and nofollow requests as it will process robots / XML sitemaps and behave just like any other crawler.
How can I improve the index results ?
It all starts with providing each content page with relevant and unique content and adding a page title and meta descriptions with the most important keywords.
Also make sure that duplicate pages have a canonical link tag so we can eliminate posslbe duplicate search results.
Can I prevent some pages from being indexed ?
The most effective way to prevent a page from being indexed is to have a noindex meta tag on the requested page.
<meta name="robots" contents="noindex, follow">
By using "noindex, follow" any link on the page will be followed.
Another way is to have the page excluded in robots.txt
How often do you crawl my website ?
That will depend on the subscription.
We will take notice of the changefrequency property in sitemaps to determine content pages that change frequently and index those more often.
Do you support robots and sitemap.xml ?
Yes we will take notice of the robots.txt file and crawl any sitemap mentioned in the file.
Do you support canonical URL’s ?
Yes, we suggest using canonical URL's to prevent duplicate search results.