Need a site search engine in asp.net with a spider/crawler that would index all static and dynamic pages and output to SQL. Also search should provide full text search (bollean And/or/phrase) with keyword highlighting in search results. The spider/crawler must follow robot rules and skip code marked between "no index" tags.
Implementation of [url removed, login to view] is preffered. More specs available through PMB.