 Logo since 2018 | |
Type of site | Search engine |
|---|---|
| Available in | English |
| Owner | System1 |
| Created by | Brian Pinkerton |
| URL | webcrawler |
| Commercial | No |
| Registration | none |
| Launched | April 20, 1994 |
| Current status | Active |
WebCrawler is a search engine, and one of the oldest surviving search engines on the web today. For many years, it operated as a metasearch engine. WebCrawler was the first web search engine to provide full text search. [1]
Brian Pinkerton first started working on WebCrawler, which was originally a desktop application, on January 27, 1994 at the University of Washington. [2] On March 15, 1994, he generated a list of the top 25 websites. [1]
WebCrawler launched on April 21, 1994, with more than 4,000 different websites in its database [2] and on November 14, 1994, WebCrawler served its 1 millionth search query [2] for "nuclear weapons design and research". [3]
On December 1, 1994, WebCrawler acquired two sponsors, DealerNet and Starwave, which provided money to keep WebCrawler operating. [2] Starting on October 3, 1995, WebCrawler was fully supported by advertising, but separated the adverts from search results. [2]
On June 1, 1995, America Online (AOL) acquired WebCrawler. [2] After being acquired by AOL, the website introduced its mascot "Spidey" on September 1, 1995. [2]
Starting in April 1996, [2] WebCrawler also included the human-edited internet guide GNN Select, which was also under AOL ownership. [4] [5]
On April 1, 1997, Excite acquired WebCrawler from AOL for $12.3 million. [2] [6]
WebCrawler received a redesign on June 16, 1997, adding WebCrawler Shortcuts, which suggested alternative links to material related to a search topic. [7]
WebCrawler was maintained by Excite as a separate search engine with its own database until 2001, when it started using Excite's own database, effectively putting an end to WebCrawler as an independent search engine. [8] Later that year, Excite (then called Excite@Home) went bankrupt and WebCrawler was bought by InfoSpace in 2001. [2]
Pinkerton, WebCrawler's creator, led the Amazon A9.com search division as of 2012. [9] [10]
In July 2016, InfoSpace was sold by parent company Blucora to OpenMail for $45 million, putting WebCrawler under the ownership of OpenMail. [11] OpenMail was later renamed System1. [12]
In 2018, WebCrawler was redesigned from scratch and its logo was changed. [13] [14]
WebCrawler was highly successful early on. [15] At one point, it was unusable during peak times due to server overload. [16] It was the second most visited website on the internet in February 1996, but it quickly dropped below rival search engines and directories such as Yahoo!, Infoseek, Lycos, and Excite in 1997. [17]
Lycos, Inc., is a web search engine and web portal established in 1994, spun out of Carnegie Mellon University. Lycos also encompasses a network of email, web hosting, social networking, and entertainment websites. The company is based in Waltham, Massachusetts, and is a subsidiary of Ybrant Digital.
A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing.
robots.txt is the filename used for implementing the Robots Exclusion Protocol, a standard used by websites to indicate to visiting web crawlers and other web robots which portions of the website they are allowed to visit.
Excite is an American website operated by IAC that provides outsourced internet content such as a metasearch engine, with outsourced weather and news content on the main page. As of 2024, all of Excite's operations are controlled by services outside of the business.
Dogpile is a metasearch engine for information on the World Wide Web that fetches results from Google, Yahoo!, Yandex, Bing, and other popular search engines, including those from audio and video content providers such as Yahoo!.
A9.com was a former subsidiary of Amazon that developed search engine and search advertising technology. A9 was based in Palo Alto, California, with teams in Seattle, Bangalore, Beijing, Dublin, Iași, Munich and Tokyo. A9 has development efforts in areas of product search, cloud search, visual search, augmented reality, advertising technology and community question answering.
AlltheWeb was an Internet search engine that made its debut in mid-1999 and was closed in 2011. It grew out of FTP Search, Tor Egge's doctorate thesis at the Norwegian University of Science and Technology, which he started in 1994, which in turn resulted in the formation of Fast Search & Transfer (FAST), established on July 16, 1997.
Yahoo! Japan is a Japanese web portal. Its search engine was the most-visited website in Japan, nearing monopolistic status.
Infoseek was an American internet search engine founded in 1994 by Steve Kirsch.
Yahoo! Search is a search engine owned and operated by Yahoo!, using Microsoft Bing to power results.
Infospace, Inc. was an American company that offered private label search engine, online directory, and provider of metadata feeds. The company's flagship metasearch site was Dogpile and its other notable consumer brands were WebCrawler and MetaCrawler. After a 2012 rename to Blucora, the InfoSpace business unit was sold to data management company OpenMail.
A search engine is a software system that finds web pages that match a web search. It searches the World Wide Web in a systematic way for particular information specified in a textual web search query. The search results are generally presented in a line of results, often referred to as search engine results pages (SERPs). The information may be a mix of hyperlinks to web pages, images, videos, infographics, articles, and other types of files. As of January 2022, Google is by far the world's most used search engine, with a market share of 90.6%, and the world's other most used search engines were Bing, Yahoo!, Baidu, Yandex, and DuckDuckGo.
Google was officially launched in 1998 by Larry Page and Sergey Brin to market Google Search, which has become the most used web-based search engine. Larry Page and Sergey Brin, students at Stanford University in California, developed a search algorithm at first known as "BackRub" in 1996, with the help of Scott Hassan and Alan Steremberg. The search engine soon proved successful and the expanding company moved several times, finally settling at Mountain View in 2003. This marked a phase of rapid growth, with the company making its initial public offering in 2004 and quickly becoming one of the world's largest media companies. The company launched Google News in 2002, Gmail in 2004, Google Maps in 2005, Google Chrome in 2008, and the social network known as Google+ in 2011, in addition to many other products. In 2015, Google became the main subsidiary of the holding company Alphabet Inc.
Info is a metasearch engine, which as of 2013, provided results from search engines Google, Yahoo!, Ask, Bing, Yandex, and Open Directory. As of 2004, news search was powered by Topix.net, Info.com's web search engine information was powered by Shopping.com and Info.com had White Page and Yellow Page search. As of 2013, Info.com also had search plugins for Google Chrome, Internet Explorer and Firefox.
Truveo is a search engine for Web video, based in San Francisco and operated by Oath Inc. It was founded in 2004 by Timothy Tuttle and Adam Beguelin. Truveo launched its first commercial video search service in September 2005. Truveo was acquired by AOL in January 2006. The name Truveo is a combination of the modern French verb trouver and the Latin term video.
MetaCrawler is a search engine. It is a registered trademark of InfoSpace and was created by Erik Selberg.
Waterfox is a free and open-source web browser and fork of Firefox. It claims to be ethical and user-centric, emphasizing performance and privacy. There are official Waterfox releases for Windows, macOS, and Linux. It was initially created to provide official 64-bit support, back when Firefox was only available for 32-bit systems.
Zoo.com is a metasearch engine, which as of 2006, provided results from search engines and other sources, including Google, Yahoo! and Wikipedia. Also as of 2006, Zoo.com provided news content from ABC News, Fox News and Yahoo! News.
