The Shocking Evolution of Search Engines: From Dial-Up to TikTok
The Evolution of Search Engines: A Brief History
Search engines have become an indispensable part of our lives, providing us with instant access to information on almost any topic imaginable. It’s hard to imagine a world without them, especially for younger generations who have grown up in the digital age. Gone are the days of spending hours in the library poring over index cards and encyclopedias. Thanks to modern search engines, we can find what we need with just a few clicks.
But where did it all begin? In this article, we’ll take a brief look at the evolution of search engines, from the early days of Yahoo directories to the complex algorithms of Google and beyond. Join us as we explore the fascinating history of search engines and how they have revolutionized the way we discover and learn.
Search Engine Updates for 2023
Search engines have come a long way since their inception in the early days of the internet. With the rise of social media platforms such as TikTok and Instagram, these platforms have also become search engines in themselves, with users using hashtags and keywords to find relevant content. This has led to a shift in the way businesses approach search engine optimization (SEO) strategies, with many now incorporating social media optimization (SMO) as well.
In addition to the rise of social media platforms as search engines, there have been significant changes and updates to search engine algorithms in recent years. In 2019, Google introduced its BERT algorithm update, which focused on understanding natural language queries and delivering more relevant results. This update has had a significant impact on the way businesses approach keyword research and content creation, as it prioritizes high-quality, informative content that provides value to users.
Another significant change in the world of search engines is the increasing use of voice search. With the rise of virtual assistants such as Siri and Alexa, users are now using natural language queries to find information online. This has led to a shift in the way businesses approach SEO strategies, with a focus on long-tail keywords and conversational content.
In conclusion, the evolution of search engines has brought about significant changes and updates, with social media platforms becoming search engines in themselves, updates to search engine algorithms, and the rise of voice search. Businesses must stay up-to-date with these changes to remain competitive and effectively reach their target audience.
History of Search Engines Developments Timeline 1945 to 1996
Vannevar Bush (1945) first mentioned the need for categorization/indexing of information and presented his theories in an article published in The Atlantic Monthly. Early concept of Search Engines.
Archie (1990) was the first tool created by Alan Emtage and L. Peter Deutsch for indexing and is considered the first basic search engine. What began as a school project at McGill University in Montreal, was an index that predated the world wide web. Gopher, released in 1991 by students from the University of Minnesota, was a protocol used to index and search for documents online as a form of anonymous FTP. Archie, Gopher, and similar counterparts lost traction in the late ’90s.
Tim Berners Lee (1991) set up a Virtual Library | The first Web site built was at https://info.cern.ch/ and was first put online on August 6, 1991, | CERN webserver hosted a list of webservers in the early age of the internet.
Veronica (1992) Searched file names and titles stored in Gopher Index Systems
Jughead (1993) also searched file names and titles in Gopher index systems but only searched a single server at a time.
Lycos (1993) was created as a university project but was the first to attain commercial search engine success. In 1999 Lycos was the most visited search engine in the world and was available in 40 countries. Now it is currently comprised of a social network with email, web hosting, and media entertainment pages.
Primitive Web Search (1993) – JumpStation: Info about a page’s title and header using simple linear search | World Wide Web Worm: Indexed titles and URLs (these two listed results in the order they were found without ranking) | RBSE Spider (had a rating system) – Unless exact title was a match, it was extremely hard to find anything
World Wide Web Wanderer (1993) Created by Matthew Gray; a bot counted active web servers and “measured the growth” of the Internet. The bot was soon upgraded to capture actual URLs | Database was called Wandex | Bot accessed the same page hundreds of times a day and caused lag
ALIWEB (1994) – Created by Martijn Koster, it crawled meta info and allowed the user to submit the pages they wanted to be indexed along with a description | No bot & no using excessive bandwidth | People didn’t know how to submit their sites
EINet Galaxy (1994) – Efficient in its utilization of different web search features | Unnecessary library due to the small size of the Internet
Infoseek (1994) was a search engine begun by Steve Kirsch and was bought by The Walt Disney Company in 1998, merging with Starwave to become go.com. Eventually, it was replaced by Yahoo, and no longer exists.
W3C (1994) Berners-Lee founded the World Wide Web Consortium (W3C) at the Massachusetts Institute of Technology.
WebCrawler (1994) created by Brian Pinkerton WebCrawler was the first crawler that indexed complete pages online. AOL purchased WebCrawler, using the technology for their network, and when Excite purchased WebCrawler, AOL used Excite to run their program NetFind. WebCrawler was one of the foundational search engines.
Yahoo! (1994) was started at Stanford University by Jerry Yang and David Filo (both electrical engineering grad students) that became a web portal and search engine.
AltaVista (1995) an industry leader, was once the most popular search engine of its time. It differed from its contemporaries because of two factors: Alta Vista used a multi-threaded crawler (Scooter) that covered more webpages than people knew existed at the time. It also had well-organized search-running back-end advanced hardware. By 1996 AltaVista had become the sole search results provider for Yahoo. In 2003 Alta Vista was bought by Overture Services, Inc.; which was acquired later by Yahoo (months later).
Angies List (1995) William S. Oesterle and Angie Hicks founded Angie’s List. The idea resulted from Hicks’s search for a reliable construction contractor in suburban Columbus, Ohio, on behalf of Oesterle, a venture capitalist who was Hicks’s boss.
AOL (1995) bought Web Crawler
Excite (1995) Founded originally as “Architext” by Stanford University students, Excite was launched officially having purchased two search engines (Magellan and WebCrawler) and signed exclusive agreements with Microsoft and Apple. Fellow Stanford students Larry Page and Sergey Brin once offered to sell their startup “Google” to the Excite team for $1 million, which was refused. Google became a $180 billion company, and Excite is now used as a personal portal called My Excite.
Looksmart (1995) competed with Yahoo! Directory and had an initial goal to create a substantial directory of websites. When it went public in 1999, it lost a fair amount of customers. By 2002 Looksmart became a pay-per-click provider, and after being dropped by Microsoft, bought a search engine called WiseNut. Sadly it never gained serious traction, and Looksmart lost its momentum.
Netscape (1995) – begins using Infoseek as their default search.
History of Search Engines Developments Timeline 1996 to 2000
ASK (1996) was originally titled “AskJeeves.com” and was designed by Garret Gruener and David Warthen in Berkeley, CA. The goal was to provide users with answers to queries typed with normal everyday language and colloquialisms. It was acquired in 2005 by IAC and continues to grow with over 100 million users.
Dogpile (1996) was a search engine developed by Aaron Flin and shortly thereafter sold to Go2net. Now Dogpile fetches results from Google, Yahoo, and Yandex.
Google (1996) – BackRub – Started as a research project by Stanford students Larry Page and Sergey Brin. They created a search engine that would rank websites based on the number of other websites that linked to that page. Prior to this, other engines ranked sites based on the number of times the search term appeared on the webpage. This strategy developed the world’s most successful search engine today.
Hotbot (1996) – Inktomi – a search engine also popular in the ’90s was launched by Wired Magazine. Inktomi pioneered the paid inclusion model but was not as efficient as pay-per-click by Overture.
LYCOS (1996) – Lycos identifies 60 million documents (more than any other search engine)
Excite (1997) buys out WebCrawler
Yandex (1997), originally standing for “yet another indexer,” is the largest search engine in Russia, and ranked as the 4th largest search engine in the world, serving over 150 million searches per day.
Ixquick.com (1998) is a MetaSearch engine that offers a proxy service for Ixquick and an email service that offers privacy protection, called StartMail. It was re-launched in 2005 and included a re-engineered MetaSearch algorithm. Devoted to privacy, Ixquick entirely ended recording IP addresses and only used one cookie, which is set to remember the user’s search preferences for future searches and is removed once a user does not return to the search engine homepage after 3 months. Ixquick was recertified with Startpage in 2013.
MSN Search (1998) was the engine used by Microsoft, sourcing search results from Inktomi, and later Looksmart. By 2006 Microsoft started performing its own image searches, and MSN became branded as Windows Live Search, then Live Search, and finally to Bing (2009) which was set to replace the Yahoo search engine.
Overture (1998) was originally named “GoTo,” where top listings were sold on a cost-per-click or pay-per-click basis. In 2000, it began driving traffic by placing its paid listings on other search engines and was eventually bought by Yahoo in 2003.
Alltheweb (1999) began in 1994 out of FTP Search, from the Norwegian University of Science and Technology, when then turned into Fast Search & Transfer, or FAST. Alltheweb (1999) was said to have once rivaled Google, but the number of users declined when Overture bought the company in 2003.
AOL Search (1999) bought Web Crawler (one of the major crawler-based engines of its time) in 1995, and after a number of deals, purchases, and exchanges, AOL re-launched its search engine, calling it AOL Search. Teaming with Google, the search engine re-launched in 2006 with newer features including video, search marketplace, etc.
Excite (1999) – bought by @Home for $6.5 billion
Google (1999) – gets funding from Sequoia Capital as well as from a few other investors
Naver (1999) was the first South Korean portal website with a self-developed search engine. SE incorporates content from users, social platforms, and paid searches. Brands that gain the most visibility are the ones that have a presence across as many areas of the search results as possible.
Baidu (2000) is one of the main search engines in China, based on a special identification technology that classifies and groups articles. Baidu locates information, products, and services through Chinese language search terms (via phonetic Chinese), advanced searches, snapshots, spell checkers, stock quotes, news, images, video, space information, weather, train and flight schedules, and other local information. Baidu’s greatest competitors are Google Hong Kong and Yahoo! China.
LookSmart (2000) – buys non-commercial directory Zeal for $20 million
Teoma (2000) meaning “expert” in Gaelic, was a search engine created by Professor Apostolos Gerasoulis and Tao Yang at Rutgers University. Teoma’s subject-specific technology centered on a link popularity algorithm that allowed pages to rank higher if other pages with a similar content and subject matter linked back to the page.
History of Search Engines Developments Timeline 2001 to 2009
AskJeeves (2001) buys Teoma to replace DirectHit. Rebranded as Ask.com.
Excite (2001) – bankruptcy leads to Infospace purchasing it for $10 million
Inktomi (2001) is exposed for accidentally allowing the public to access a database of spam sites (over one million)
WiseNut (2001) was a crawler-based search engine that was introduced as a beta and was owned by Looksmart. Initially, the site was well-reputed with an unsullied database, and automatic clustering of search results by using a technology called WiseGuide. Looksmart bought WiseNut in 2002 and was eventually closed in 2007.
LookSmart (2002) transitions into a pay-per-click provider, destroying reliability. LookSmart buys WiseNut
Yahoo! (2002)starts working on their search engine again, beginning to acquire other search directories. Until this time, they had outsourced their search services
AllTheWeb (2003) was bought by Overture for $70 million
Google (2003) releases their first officially named update, “Boston,” announced at Northeastern’s SES Boston
Inktomi (2003) is bought out by Yahoo! for $235 million
LinkedIn Company Directory (2003) started out in the living room of co-founder Reid Hoffman. Reid has previously been on the board of Google, eBay, and PayPal, so he had a proven track record before taking the first round of funding and eventually floating on the New York stock exchange.
LookSmart (2003) feels the sting of rejection when they were dumped by Microsoft and lost more than 65% of their annual revenue
Overture (2003) plans to purchase AltaVista for $80 million in stock, $60 million in cash
Yahoo! (2003) buys Overture for $1.63 billion
Facebook (2004) – Mark Zuckerberg launched the site as a Harvard sophomore, called “TheFacebook.com”
Lycos (2004) is sold to Daum Communications, the second-largest Internet portal in Korea
MSN (2004) launches a preview of the new search engine
Yelp (2004), named for the concept of “Yellow Pages” began as an email service exchange recommending the local business. Yelp now is connected to social networking sites and functions as a search engine, where users can access reviews for companies/restaurants/businesses under a specific search/product. Yelp recently announced that it is now powering the Microsoft Bing local search engine results. Yelp became a public company in March 2012 and became profitable for the first time two years later.
IAC (2005) – Owner of ticketmaster.com & match.com – buys Ask Jeeves for $1.85 billion, changes name to Ask.com and drops Teoma
Merchant Circle (2005) was founded in 2005 and announced in 2006. MC is an online business directory, a social business network that combines traditional yellow pages with social media such as Facebook. In 2007, it was named “Newcomer of the Year” by AlwaysOn Media in 2007.
MSN (2005) drops Yahoo! organic results for own in-house technology in January 2005
Snap (2005) – Overture owner Bill Gross launches Snap search engine which shows search volumes, revenues, and advertisers | Proved to be too complicated, not simplistic enough for the average web surfer
The advent of “nofollow”: Google, Microsoft, and Yahoo! clean up spammy blogs everywhere with the introduction of “no-follow” links
DuckDuckGo (2006) is a search engine that does not store or share any information about the user and is unique to other search engines by providing all users the same results for a given search term, as well as providing search results from what they describe as the “best sources” rather than from the most sources. DuckDuckGo’s results are sourced from places like Yahoo, Search Boss, Wikipedia, Wolfram Alpha, and their own web crawler the DuckDuckBot, etc.
HubSpot (2006) was founded by Brian Halligan and Dharmesh Shah at the Massachusetts Institute of Technology (MIT). The company grew from $255,000 in revenues in 2007 to $15.6 million in 2010. Later that year HubSpot acquired Oneforty, the Twitter app store founded by Laura Fitton.
LookSmart (2006) shuts Zeal down
Microsoft (2006) announces the launch of Live Search Product
MSN (2006) drops Yahoo!’s search ad program
Google (2006) changes the SERP for good with “Universal Search”—integrating plain, 10-listing SERPS with features like News, Video, Images, Local, and other verticals
Cuil (2008) was a search engine that arranged pages by content, showing large entries with pictures, and thumbnails for results, etc. The search engine claimed to have over 120 billion web pages indexed, and would not store users’ search activity or their IP number. By 2010, Cuil suddenly shut down after an acquisition agreement failed to go through, and their patents were sold to Google when the company dissolved.
DuckDuckGo (2008) was founded by Gabriel Weinberg, an entrepreneur who previously launched Names Database, a now-defunct social network. Search engine because it does not store any user information. It shows the same results to all users regardless of search history or location
Google (2008) releases “Google Suggest”—users receive drop-downs of suggested topics related to their queries
Thumbtack (2008) – Contract hiring site / Local Professional
Bing (2009) – Rebranding of MSN/Live Search | Inline search suggestions for related searches directly in the result set
Foursquare (2009) is a location-based social networking search engine for mobile devices, utilizing a GPS hardware system where users can search for restaurants/entertainment, etc. in their immediate locale and connect with others in the area.
Foursquare (2009) local search-and-discovery service mobile app which provides search results for its users. The app provides personalized recommendations of places to go to near a user’s current location based on users’ “previous browsing history, purchases, or check-in history”.
StartPage (2009) is a secure search engine, meaning it pulls all the same results as Google but uses the privacy protection of Ixquick, which allows users to search with privacy.
Wolfram Alpha (2009) is a “computational knowledge engine” that answers factual queries by computing the answer from externally sourced “curated data” instead of listing relevant websites that could lead to the answer.
History of Search Engines Developments Timeline 2010 to 2018
Google (2010) – releases ‘Caffeine’ a web indexing system, that delivers 50% fresher search results | Users receive real-time search results as they type their queries
Google (2011) – release “Panda” marks a considerable change in Google’s ranking algorithm. Content farms and scraper sites, among others, are negatively impacted. 12% of Google search results in the US are affected
Google | Yahoo | Microsoft (2011) – releases ‘Schema’ with a goal of maintaining and promoting schemas for more structured internet data, Google, Yahoo!, and Microsoft collaborate to create Schema.org
Apple Maps (2012) is a web mapping service developed by Apple Inc.
Bing (2012) launches Social Sidebar. Users see search results through the lens of their social networks
Google (2012) – “Penguin” update continues to keep SEOs on their toes. Sites primarily penalized: those buying links, or obtaining them through link networks designed to boost search rankings
So.com | Qihoo 360 (2012) a very popular search engine in China – controls 1/3 of the market.
Google (2013) – Hummingbird – the first search algorithm with the ability to parse the intent behind a query, rather than just the language itself
Google (2014) “Pigeon” – A Google update to deliver more useful, relevant, and accurate local search results; Google prioritizes website security: calls for “HTTPS Everywhere,” and begins using HTTPS as a ranking signal
Google My Business (2014) – is any local business’s best friend on the Internet. It is a huge change from the sponsored listings and directory listings of the past on Google. This allows any business owner to get the visibility they need on Google in an easy way that is also easy to manage.
Yahoo! (2014) signs a deal to be Mozilla Firefox’s default search provider in the United States
Bing (2015) – releases its own mobile-friendly algorithm update
Google (2015) – Mobilegeddon – releases an update designed to benefit mobile-friendly pages in mobile search results | RankBrain – reveals that machine learning has played a role in its ranking algorithm for months
Google (2016) releases the (unofficially named) “Possum,” — local search results are diversified; spammy sites are penalized; “Penguin” goes real-time, and becomes a part of Google’s core ranking algorithm
Google (2017) “Google begins punishing sites with aggressive interstitials and pop-ups that damage the mobile user experience; Google search algorithm update unofficially dubbed “Fred” punishes sites with low-quality backlinks, and sites that prioritize monetization over user experience; Maccabees Updates
Google (2018) confirms via Twitter that a broad core algorithm update had been rolled out.
Yahoo! (1994) was started at Stanford University by Jerry Yang and David Filo (both electrical engineering grad students) that became a web portal and search engine.
MSN Search (1998) was the engine used by Microsoft, sourcing search results from Inktomi, and later Looksmart. By 2006 Microsoft started performing its own image searches, and MSN became branded as Windows Live Search, then Live Search, and finally to Bing (2009) which was set to replace the Yahoo search engine.
DuckDuckGo (2006) is a search engine that does not store or share any information about the user and is unique to other search engines by providing all users the same results for a given search term, as well as providing search results from what they describe as the “best sources” rather than from the most sources. DuckDuckGo’s results are sourced from places like Yahoo, Search Boss, Wikipedia, Wolfram Alpha, and their own web crawler the DuckDuckBot, etc.
