 Logo since 2018 | |
Type of site | Search engine |
|---|---|
| Available in | English |
| Owner | System1 |
| Created by | Brian Pinkerton |
| URL | webcrawler |
| Commercial | No |
| Registration | none |
| Launched | April 20, 1994 |
| Current status | Active |
WebCrawler is a search engine, and one of the oldest surviving search engines on the web today. For many years, it operated as a metasearch engine. WebCrawler was the first web search engine to provide full text search. [1]
Brian Pinkerton first started working on WebCrawler, which was originally a desktop application, on January 27, 1994 at the University of Washington. [2] On March 15, 1994, he generated a list of the top 25 websites. [1]
WebCrawler launched on April 21, 1994, with more than 4,000 different websites in its database [2] and on November 14, 1994, WebCrawler served its 1 millionth search query [2] for "nuclear weapons design and research". [3]
On December 1, 1994, WebCrawler acquired two sponsors, DealerNet and Starwave, which provided money to keep WebCrawler operating. [2] Starting on October 3, 1995, WebCrawler was fully supported by advertising, but separated the adverts from search results. [2]
On June 1, 1995, America Online (AOL) acquired WebCrawler. [2] After being acquired by AOL, the website introduced its mascot "Spidey" on September 1, 1995. [2]
Starting in April 1996, [2] WebCrawler also included the human-edited internet guide GNN Select, which was also under AOL ownership. [4] [5]
On April 1, 1997, Excite acquired WebCrawler from AOL for $12.3 million. [2] [6]
WebCrawler received a redesign on June 16, 1997, adding WebCrawler Shortcuts, which suggested alternative links to material related to a search topic. [7]
WebCrawler was maintained by Excite as a separate search engine with its own database until 2001, when it started using Excite's own database, effectively putting an end to WebCrawler as an independent search engine. [8] Later that year, Excite (then called Excite@Home) went bankrupt and WebCrawler was bought by InfoSpace in 2001. [2]
Pinkerton, WebCrawler's creator, led the Amazon A9.com search division as of 2012. [9] [10]
In July 2016, InfoSpace was sold by parent company Blucora to OpenMail for $45 million, putting WebCrawler under the ownership of OpenMail. [11] OpenMail was later renamed System1. [12]
In 2018, WebCrawler was redesigned from scratch and its logo was changed. [13] [14]
WebCrawler was highly successful early on. [15] At one point, it was unusable during peak times due to server overload. [16] It was the second most visited website on the internet in February 1996, but it quickly dropped below rival search engines and directories such as Yahoo!, Infoseek, Lycos, and Excite in 1997. [17]
Lycos, Inc., is a web search engine and web portal established in 1994, spun out of Carnegie Mellon University. Lycos also encompasses a network of email, web hosting, social networking, and entertainment websites. The company is based in Waltham, Massachusetts, and is a subsidiary of Ybrant Digital.
robots.txt is the filename used for implementing the Robots Exclusion Protocol, a standard used by websites to indicate to visiting web crawlers and other web robots which portions of the website they are allowed to visit.
MapQuest is an American free online web mapping service. It was launched in 1996 as the first commercial web mapping service. MapQuest vies for market share with competitors such as Apple Maps, Here and Google Maps.
AltaVista was a Web search engine established in 1995. It became one of the most-used early search engines, but lost ground to Google and was purchased by Yahoo! in 2003, which retained the brand, but based all AltaVista searches on its own search engine. On July 8, 2013, the service was shut down by Yahoo!, and since then the domain has redirected to Yahoo!'s own search site.
HowStuffWorks is an American commercial infotainment website founded by professor and author Marshall Brain, to provide its target audience an insight into the way many things work. The site uses various media to explain complex concepts, terminology, and mechanisms—including photographs, diagrams, videos, animations, and articles.
Excite is an American website operated by IAC that provides outsourced internet content such as a metasearch engine, with outsourced weather and news content on the main page. As of 2024, all of Excite's operations are controlled by services outside of the business.
Dogpile is a metasearch engine for information on the World Wide Web that fetches results from Google, Yahoo!, Yandex, Bing, and other popular search engines, including those from audio and video content providers such as Yahoo!.
A9.com was a former subsidiary of Amazon that developed search engine and search advertising technology. A9 was based in Palo Alto, California, with teams in Seattle, Bangalore, Beijing, Dublin, Iași, Munich and Tokyo. A9 has development efforts in areas of product search, cloud search, visual search, augmented reality, advertising technology and community question answering.
AlltheWeb was an Internet search engine that made its debut in mid-1999 and was closed in 2011. It grew out of FTP Search, Tor Egge's doctorate thesis at the Norwegian University of Science and Technology, which he started in 1994, which in turn resulted in the formation of Fast Search & Transfer (FAST), established on July 16, 1997.
Yahoo! Japan is a Japanese web portal. Its search engine was the most-visited website in Japan, nearing monopolistic status.
Infoseek was an American internet search engine founded in 1994 by Steve Kirsch.
Yahoo! Search is a search engine owned and operated by Yahoo!, using Microsoft Bing to power results.
Infospace, Inc. was an American company that offered private label search engine, online directory, and provider of metadata feeds. The company's flagship metasearch site was Dogpile and its other notable consumer brands were WebCrawler and MetaCrawler. After a 2012 rename to Blucora, the InfoSpace business unit was sold to data management company OpenMail.
A search engine is a software system that provides hyperlinks to web pages and other relevant information on the Web in response to a user's query. The user inputs a query within a web browser or a mobile app, and the search results are often a list of hyperlinks, accompanied by textual summaries and images. Users also have the option of limiting the search to a specific type of results, such as images, videos, or news.
MetaCrawler is a search engine. It is a registered trademark of InfoSpace and was created by Erik Selberg.
Waterfox is a free and open-source web browser and fork of Firefox. It claims to be ethical and user-centric, emphasizing performance and privacy. There are official Waterfox releases for Windows, macOS, Linux and Android. It was initially created to provide official 64-bit support, back when Firefox was only available for 32-bit systems.
Zoo.com is a metasearch engine, which as of 2006, provided results from search engines and other sources, including Google, Yahoo! and Wikipedia. Also as of 2006, Zoo.com provided news content from ABC News, Fox News and Yahoo! News.
System1 is an American Internet advertising company. Formerly known as OpenMail, it was founded in 2013. It describes itself as operating a "Responsive Acquisition Marketing Platform", and cites privacy as one of its principal foci, although it has been criticized for its influence on privacy-focussed properties, including search engine Startpage.com. It is headquartered in Marina del Rey, California.
