rennea
Gold Member
- 3,728
Follow along with the video below to see how to install our site as a web app on your home screen.
Note: This feature may not be available in some browsers.
janetupnorth said:Yahoo! Slurp is Yahoo!'s web-indexing robot. The Yahoo! Slurp crawler collects documents from the Web to build a searchable index for search services using the Yahoo! search engine. These documents are discovered and crawled because other web pages contain links directing to these documents.
As part of the crawling effort, the Yahoo! Slurp crawler will take robots.txt standards into account to ensure we do not crawl and index content from those pages whose content you do not want included in Yahoo! Search Technology. If a page is disallowed to be crawled by robots.txt standards, Yahoo! will not read or use the contents of that page. The URL of a protected page may be included in Yahoo! Search Technology as a "thin" document with no text content. Links and reference text from other public web pages provide identifiable information about a URL and may be indexed as part of web search coverage.
(From Yahoo)
The_Kitchen_Guy said:I think I liked my answer better.
One more piece of software snooping around your stuff.dannyzmom said:Ok - and can you translate that to English??? :
A Yahoo Slurp Spider is a web crawler or bot used by Yahoo to gather information from web pages for indexing and search engine results.
The purpose of a Yahoo Slurp Spider is to crawl and index web pages for Yahoo's search engine. This allows for more accurate and relevant search results for users.
A Yahoo Slurp Spider works by following links from one web page to another, gathering information and indexing it for search results. It also takes into account factors such as keywords and meta tags to determine the relevance of a page.
No, a Yahoo Slurp Spider is not harmful to your website. It simply gathers information for indexing and does not make any changes to your website's content or structure.
Yes, you can block a Yahoo Slurp Spider from crawling your website by using a robots.txt file or by setting specific rules in your website's server configuration. However, this may affect your website's visibility in Yahoo's search results.