Search engines as we know them today are not nearly what they once were. What we have come to know as “Googling something” actually began as a much simpler, data querying solution in the early 1990s. Now, more than 20 years later, we take to search engines to discover seamlessly anything and everything about the world in multiple media formats. Our search engines are ridden with AdWords and SEO-rankings; things that were not a twinkle in the Internet’s eye at the time of the first search engine.
How it Began
The first search engine was called “archives” and later shortened to “archie”; referring to the solution for a data scatter problem which would match a user query to a database of web-based filenames. To put things in perspective, at the time of use, Archie was able to access roughly 2.6 million files on the web and 150 GB of information. Compared to the incredible oversaturation of the Internet today, this may seem like nothing. But looking back, it was pretty impressive for the time.
At this time, there was no such thing as the World Wide Web. Instead, to share information, users would establish a File Transfer Protocol (FTP), which could be located by others using an FTP client. In 1991, the World Wide Web was invented and the opportunity to build websites – not just host faceless files on servers – became a reality.
Something to Search For
Search engines developed steadily from the mid-1990s on. As more content became available on the WWW (W3), search engines were used to crawl the web for information from websites, similar to how we use them today. Search engines of the 90s – and even early 2000s – were less complex and slower than today’s top engines: Google, Bing, and Yahoo.
What is a Search Engine?
A search engine is comprised of three parts that allow users to enter a query and see results from around the W3: spiders, indexes, and search interface and relevancy software. The first aspect – the spider – crawls the web by scanning pages that have been indexed or recently updated. The second part – the search engine index – is actually a catalog of content, which almost accurately represents the current content on the web. And the third ingredient to a working search engine is the software, which acquires a user query, quickly checks the query for accuracy and relevancy to other queries, and then gathers a list of relevant results. The results are ranked based on the content of the pages in relation to the query.
This is relevant because now, as the world focuses a tremendous amount of attention on the Internet, everyone is eager to create content that lands high on searches, no matter of its true relevance. Even when a website is relevant to a query, it is not guaranteed to show in the first page of search results because of how much content exists.
Search Engines of the 1990s
There were a number of search engines conceived in the 90s, all with various searching concepts that are quite foreign to you and me. Here are a few that didn’t make it past the early 2000s:
The original search engine of Yahoo. The company did not fully disintegrate, however, it switched gears to focus on enterprise search solutions, instead.
Infoseek planned to charge people for their searches. Interesting how search engines do make money today, but in a very different way.
Had potential when first launched, utilizing a groundbreaking technology, but was neglected after the company was purchased and formally closed.
These, along with many others, strove to deliver information from across the web using what little was really known at the time about how people were going to use search engines. Some of our major technological advancements have come from understanding user behavior and developing solutions to fit them.
As early as the late 90s, we saw companies trying to monetize search engines. Some – like Infoseek – wanted to have users pay to use their service. This is kind of like paying to read a book you could easily get for free at the library. Others saw the future and potential of search engines as a necessity and leveraged their funds from companies who wanted to appear in search results. Search Engines like Overture – previously, GoTo – provided paid listings for other sites and those who paid the most would rank the highest in the search results. Interestingly, companies like MSN and Yahoo still use these paid listings, resourced from Overture.
By the early 2000s, most search engines were using a variation of the crawler technology. Some, however, like LookSmart favored human-powered results. While LookSmart doesn’t act as a traditional search engine anymore, it is utilized by some larger companies like MSN to provide search results based on the combination WiseNut crawler technology and human-powered results.
What We Know Today
The most popular search engine in use today is of course Google. Google was conceived in 1998 as a research project and continued to develop the ability to analyze web links and produce an entirely new way of generating extremely relevant search results, contrived from crawler technology.
The Internet as we know it is currently made up of 60 trillion individual pages. Today’s search engines have the power to crawl those 60 trillion pages within the index in just the click of your Enter key and sort the results by relevancy and quality of content using extensive algorithms and formulas to deliver the most accurate results in the least amount of time possible. Get this: as you are typing in your query – before you even press Enter – the search engine begins to analyze what you may be looking for as to speed up your search results and deliver to your browser the best results.
A Long Way
While it seems like search engine history is just that – history – what we have today would never have been if it weren’t for the very first database querying software, Archie. The endless resources we have at the tip of our fingers are thanks to twenty-five years of patient growth, trial and error, and incredible development in the world of technology and the Internet.
And hey, if it weren’t for people like you reading this blog and using the Internet – formerly referred to as the World Wide Web – there would be no need for the incredible advancements surrounding search engines.