Search Engine Archeology

I read a novel not long ago, Rainbow’s End by Vernor Vinge, that suggested that in the future one of the most popular technology positions would be that of software archeologist, with development and programming skills capable of digging through many lines of code to decipher where they originated and how they might work with other kludge within a program to interact in meaningful ways. It made me wonder how important it would be to have a sense of the history of the growth and development of the Web.

A trip to the US Library of Congress Photographs website showed me a little of the local history of my region that I didn’t know much about, including the existence of a resort I hadn’t heard of before about five miles from where I live that could house more than a thousand people, and which had been the vacation spot of Presidents, Senators, Supreme Court Justices, and more.

A postcard showing the Fauquier Virginia White Sulpher Springs resort in its heyday.

I was excited to learn about the Fauquier Virginia White Sulpher Springs Spa partially because it was part of something that happened in my region that wasn’t directly tied to the revolution or the Civil War. Most of the local history I hear about involves one or the other of those events as if the time between them, and the time after almost never happened.

In the 1830s, a couple of large hotels and a number of cabins were constructed around springs at a location roughly 50 miles west of Washington, DC, to allow people to take advantage of the healthy waters of those springs. During a cholera epidemic in Richmond in 1849, the Virginia Legislature picked everything up and moved to the resort to continue the business of government in healthier surroundings.

The resort was located within a mile of a bridge that played a strategic transportation role during the Civil War, and a number of the larger buildings were set on fire during a battle for possession of the bridge. No one knows whether the North or the South was responsible for the destruction. The image below from Harper’s Weekly shows Union troops camped in front of the ruins, and notes that they were still drinking from the springs on the property.

An image from Harper's weekly showing a Union army camping in front of the ruins of the resort.

After the War, the resort regained popularity as a resort, with people being transported by stage coach from a train station in Warrenton. In the 1930s, Walter P. Chrysler purchased the land, and supposedly a number of the buildings on the site were restored to their former glory.

A visit this past weekend showed that the spa had been replaced by a country club with a fairly modern main building and golf cart paths and greens dotting the country side. One grass overgrown patch near the entrance to the country club showed some landscaping features that included a few stone steps down to a place were water could be collected, and a very weathered birdbath.

A picture from today showing some stairs leading to a water cache near the entrance of the country club.

I asked a grounds keeper if he knew of any old buildings or structures on the property, and he said that he was new to the job, only having been there a year, and didn’t know about anything like that. The people inside the main building did know about a fire that caused the present building to replace the old one. But that fire was one that happened maybe 30 years ago. A look around the clubhouse showed some old prints from the early 1800s, but it seemed like the people who worked there were oblivious to much of the history of the land they worked upon.

There were many resorts in Virginia, and in the rest of the Country that centered around healthy spring waters. In more north eastern states like those in New England, many blossomed into urban centers, but in Virginia these resorts were often places found in the countryside to visit during the warm summer months. The healing powers of the waters and the country air were a prescription of health for the times, and seem to have been replaced by a morning or mid day getaway for a few holes on the greens.

I can’t help but think of how the landscape of the Web is changing and transforming as well, from the days when Altavista and Excite and Lycos were some of the most popular search engines on the face of the World Wide Web, to be surplanted by Google and Bing these days.

The Yahoo Directory was one of the most important places to have your website listed in the 90s, and well as the Open Directory Project, and now it seems clear that a human edited directory can’t keep up with the growth of the Web.

With a phrase like “More wood behind fewer arrows“, Google recently announced that they would be ending many of the projects they were testing at Google Labs.

Once extremely popular social site Myspace keeps on losing ground to newer social sites such as Facebook, and even Google’s new social site Google Plus.

The search engine optimization and web promotion I’ve been doing for the last 15 years has changed over that time as well, as the web and search changes.

Thinking back, I remember producing ranking reports years ago that included 10 or 11 search engines. Many site owners that we did SEO for only had rudimentary log file statistics programs on their sites that they barely looked at, and now many of the sites you look at have Google Analytics account code installed on their pages.

The help guidelines from the search engines have transformed over time as well. I remember when I first read the AltaVista FAQ that suggested adding up to 1024 characters worth of keywords to your meta keywords tag. Google’s Webmaster Guidelines used to suggest that you submit your web address to high quality directories to make it easier for your site to be found online. That suggestion appears to have been replaced by a statement to “Make sure all the sites that should know about your pages are aware your site is online.”

Many sites you visit these days have buttons and widgets that allow people to share your pages with others through Twitter and Facebook and Google Plus and other services. Years ago, sites sometimes provided ways for you to share a link to a page via email.

I’ve been writing about patents and whitepapers from the search engines here, and sometimes feel as much like a historian as a blogger, or someone in search of ideas and assumptions and concepts explained in those primary documents from the search engines.

Some of the patents I’ve looked at describe changes that likely happened at Google or Yahoo or Microsoft, though possibly not quite in the same manner as described within the patents themselves. Some of them describe things that might still happen, as soon as the technology or the web itself matures enough to make them possible. Some of those patents describe alternative pathways that the search engines could or might have taken, but didn’t because of changes in technology or because they might not have made sense from a business standpoint.

Many of them have described things the search engines may have implemented that were impossible to see on the surface, that required digging down below what we see of the interfaces of Google or Yahoo or Bing. I’m staggered sometimes in seeing some of the hints at technology below the surface that many patents hold.

Just like I’m staggered to imagine the resort that existed five miles down a sleepy little country road from me, that housed thousands, and was a place of healing and recreation for many people for a good number of years, is now a sleepy little country club for people to casually hit around a few golf balls.

Share

20 thoughts on “Search Engine Archeology”

  1. I remember the first time I have used Netscape Navigator to browse Yahoo Directory, then Altavista in the early days that was a great search engine with the biggest index (and shortly lots of spammy webpages unfortunately). Thanks for your article.

  2. Hi Nicholas,

    Thank you.

    I remember downloading Navigator and upgrades to IE back in the day of slow dialup access and it taking forever to do so, especially when new versions of them were first released, and everyone else was trying to as well. Faster internet speeds have really made a difference there as well.

    I used to use the Yahoo directory a lot, and remember setting it as the default homepage on people’s computers when I was setting up new computers for people while doing tech support. Before Google came along, I used AltaVista as a first search engine for many of the searches I performed, but I always used the advanced search so that I could use the special search operator to remove results that had certain terms in them. (I’d perform a search, and look at the results, and find terms in results I didn’t want to see, and remove those with the “-” operator in front of those keywords).

  3. Bill,

    As someone who has struggled to find his “niche” or place in this very broad industry, I’ve always looked at you as an example of someone who stands out because of his focus.

    We know we can come to SEO by the Sea to see a straight shooter talking about inner-workings, things that are extremely unique to this site.

    It’s amazing to think how quickly things evolve in retrospect. However, your writing style and focus stay true. You go to the big sites and see short resources on the latest & greatest that barely provide just more insight than what you already possessed. So you, good sir, are more like the region itself than a building or structure that changes so often. Full of volumes of knowledge, able to provide valuable insight not often recognized for the value it truly has.

  4. What an interesting article Bill. It made me dig up some memories of myself using Lycos and Altavista as well as Yahoo! Geocities to build my very first website at the age of 14…

  5. install excel => three 3.5 discs
    install coral draw => eleven 3.5 discs

    and every update I had to do this => hours of looking on the bars ;)

    Netscape was a big revolution ;)

    15years before or today: SEO is a matter of strategy. Maybe there are more tools, but this doesn’t mean that there are more people who are knowing how to read the tools.

    “Full of volumes of knowledge, able to provide valuable insight not often recognized for the value it truly has.”

    simple:yes :-)

  6. It’s interesting to read your article and to note how fast the web has indeed changed in the last few years. I do not have your experience and have been in the “SEO Game” for less than 5 years but I remember the days when keyword stuffing (including meta keywords) and directories submissions could be enough to rank. Nowadays it’s a new ball game with the search engines relying more and more on social signals. It’s a whole new user experience and it has made our job as SEOs much more complicated but also more exciting.

  7. There is definitely a weird timeline difference between history as a whole and the web. The latest seems to speed up things a lot. When I look back, it has been less than a decade things got serious.
    Very nice post of yours; really inspiring.

  8. Hi Bill,

    Great post – I really enjoy hearing about Virginia!

    On the thinking about the future – I think it really depends on how well my day is going.

    If I’m having a less than brilliant day I harbour a thought for the early days when things seemed simpler. Alta Vista and Lycos search engines, 56k modems and Geocities. When I guess I thought things were easier and perhaps a little less competitive.

    If I’m having a good day I think about how amazing it is for us . All the knowledge available along with the tools (if not the time) to really create, distribute, learn and discover as much as you want around almost any subject.

    I also think about all the discoveries that are yet to happen, the evolution of the way data is presented and the ease that it can be accessed.

    Occasionally – I think about how neat it would be to have bought shares in Apple in 1992, or to travel back in time with a touch screen phone and show it to nokia in 1998…

  9. The Yahoo Directory was one of the most important places to have your website listed in the 90s… I really think this is very important today too. I don’t know why but i tried many times to put my website to the yahoo directory but with no success…But do you think is so important for my site to be listed in those directory that demand payments ? should i pay all those money ?

  10. SEO has certainly changed the way I do my job. It’s hard to imagine what we were working with in the ’90s and how we felt like it was bigger than it ever should have been. I can remember dialing into a Prodigy hub in high school and thinking “Wow I have the whole world at my fingertips!” Where will be in another 10 years?

  11. Hi Bill
    Thanks for another wonderful post. I also remember those days when Altavista was on the boom. The question emerges in mind that can this happen to Google too. I mean after some years we see a new gaint cooming in? May be there could be possibility that inspite of bieng google the best search engine may be lacking some more refined algorithms and fast searching techniques the new comer may introduce and take place of google? like what happened with Altavista and others of that time?

  12. Hi Brent,

    Thank you for your kind words – they are much appreciated. I do try to present things in as straightforward a manner as I can here, and I do try to respond to comments, even though it sometimes takes a little while to get back to them.

    I try not to publish things here that I don’t learn from myself. Just presenting something that might be an echo of what other people write about doesn’t do that for me.

  13. Hi George,

    I wish I had been able to get onto the Web earlier than I had, but it just wasn’t around when I was a teenager. It is really interesting and exciting to see the evolution of the Web, and to think about where and how it has been growing.

  14. Hi Monika,

    Corel Draw was one of my favorite programs, and I remember installing many programs via floppy disk.

    Installation progress bars seem to fly along these days, but I remember spending hours watching them move as well.

    The tools may sometimes make SEO easier, but not the strategies behind them, which require using creativity, imagination, and experience.

  15. Hi Eric,

    I guess one of the truly fun parts of SEO is that it is ever evolving and changing. Some of the basics still remain true today, but there are more signals that search engines will look at. One of the things that I like about some of the changes taking place are that they do often require that businesses be more in touch with their audiences, and interact with them in meaningful ways. Social signals have been helping to do that, and it is more exciting.

  16. Hi Laurent,

    Thank you.

    The history of the Web is exciting, and the ability to share information and spread ideas is definitely part of an informational revolution.

    Radio and television acted to transform society in their days, and the Web builds upon that by making the sharing of information more of an interactive experience. We definitely live in interesting times.

  17. Hi Tom,

    Good to hear. I like learning about and writing about Virginia. I do find local history fascinating.

    If we knew about what the Web would have become today back then, we would have done things like registered for domain names for free that are now selling for millions of dollars, and invested in companies that sounded like they wouldn’t have come to much but did.

    The Web is growing, and becoming more sophisticated, but I think there are a lot of opportunities out there that are just waiting to be filled.

    The thing that amazes me is how much easier it is to access information than it ever has been in the past, to be able to develop expertises in subjects that many others might not see the sense in pursuing which could become valuable in their own rights, and the chance to communicate with people from around the world who might share common interests that I likely wouldn’t be able to talk with otherwise (thanks for that).

  18. Hi Samuel,

    I do remember using the Yahoo Directory a lot back in the 90s, but not so much these days.

    I don’t think that it holds as much power as it did. Chances were back then that you were more likely to get actual traffic from there to your website from people who used the directory to explore topics that they might not have known much about.

    There are so many other sites online that now curate links and information on specific topics, and social sites where you can interact with others and ask questions, that directories like Yahoo or DMOZ don’t have as much power.

    If you think that being in Yahoo might help, then you should consider trying. But there are a lot of other places to explore on the Web and see how they might help you that I would definitely explore those. For instance, if you have a site with a physical presence such as a storefront of office that you want people to visit in person, I’d definitely focus upon verification in Google Maps and listings in business profile and telecommunication directories first.

  19. Hi Kelli,

    I remember how excited I was when I found out I could dial my local library with my computer and search through the card catalog to see if they had books I was looking for before I made the trip to check the books out in person.

    And now I can search for the same information without having to go to the library, and stand a good chance of finding relevant information.

    SEO has worked to help make a lot of information easier to find online, and I think we’ll be surprised by how much easier it will be to find relevant and important videos and other sources of information in the future as well.

  20. Hi Zia,

    Thank you.

    I think there’s the possibility that something that will come along that provides better information, more relevant results, and easier to use interfaces than what Google does, but Google doesn’t seem afraid to innovate and try new things, so who knows?

    The Web does seem to be moving to mobile devices really quickly, and perhaps the next search engine that people will swarm to is one that takes the best advantage of that movement.

Comments are closed.