Web search engines work by sending out a spider to fetch as many documents as possible. Spider also called "crawler" or a "bot" which can goes to every pages or can representative pages on every web site that wants us to be searchable and reads it. By using this spider which will using hypertext links on each page to discover and read a site's other pages.
Another program, called an indexer, which can reads these documents and creates an index based on the words contained in each document. Each search engine uses a proprietary algorithm to create its indices such that, ideally, only meaningful results are returned for each query.
Different Search Engine Approaches
- Major search engines such as Google, Yahoo (which uses Google), AltaVista, and Lycos index the content of a large portion of the Web and provide results that can run for pages - and consequently overwhelm the user.
- Specialized content search engines are selective about what part of the Web is crawled and indexed. For example, TechTarget sites for products such as the AS/400 (http://www.search400.com) and CRM applications (http://www.searchCRM.com) selectively index only the best sites about these products and provide a shorter but more focused list of results.
- Ask Jeeves (http://www.ask.com) provides a general search of the Web but allows you to enter a search request in natural language, such as "What's the weather in Seattle today?"
- Special tools and some major Web sites such as Yahoo let you use a number of search engines at the same time and compile results for you in a single list.
- Individual Web sites, especially larger corporate sites, may use a search engine to index and retrieve the content of just their own site. Some of the major search engine companies license or sell their search engines for use on individual sites.
- Yahoo (http://www.yahoo.com)
- search.com (http://search.com)
- EasySearcher (http://www.easysearcher.com)
A significant advantage of a Yahoo search is that if you locate an entry in Yahoo, it's likely to lead you to a Web site or entire categories of sites related to your search argument.
A search.com search primarily searches the Infoseek index first but also lets you search the other major search engines as well.
EasySearcher lets you choose from either the popular search engines or a very comprehensive list of specialized search engine/databases in a number of fields.
Yahoo, search.com, and EasySearcher all provide help with entering your search phrase. Most Web portal sites offer a quickly-located search entry box that connects you to the major search engines.
- If you know of a specialized search engine such as SearchNetworking that matches your subject (for example, Networking), you'll save time by using that search engine. You'll find some specialized databases accessible from Easy Searcher 2.
- If there isn't a specialized search engine, try Yahoo. Sometimes you'll find a matching subject category or two and that's all you'll need.
- If Yahoo doesn't turn up anything, try AltaVista, Google, Hotbot, Lycos, and perhaps other search engines for their results. Depending on how important the search is, you usually don't need to go below the first 20 entries on each.
- For efficiency, consider using a ferret that will use a number of search engines simultaneously for you.
- At this point, if you haven't found what you need, consider using the subject directory approach to searching. Look at Yahoo or someone else's structured organization of subject categories and see if you can narrow down a category your term or phrase is likely to be in. If nothing else, this may give you ideas for new search phrases.
- If you feel it's necessary, also search the usenet newsgroups as well as the Web.
- As you continue to search, keep rethinking your search arguments. What new approaches could you use? What are some related subjects to search for that might lead you to the one you really want?
- Finally, consider whether your subject is so new that not much is available on it yet. If so, you may want to go out and check the very latest computer and Internet magazines or locate companies that you think may be involved in research or development related to the subject.
From Low Lik Yuen (Eric)