Understanding Search Engine Algorithms

Search technology has evolved in many ways within the past few years. The technology involved has
transformed from a simple mathematical science to what seems like an artificial intelligence.
Just a few years ago the algorithms involved in comprehending a web page consisted of a simple process of
examining meta tags (HTML code) and counting the words on a web page (Keyword Density). Today, this
process has evolved into a sophisticated comprehension of a web site and everything which surrounds it.
Count, spell, define, measure and more.
A sophisticated search engine like Google, can interpret a web sites overall theme while defining contents, checking spelling, calculating
relevance, measuring pagerank, translating languages, identifying it’s owner, tracing its geographic origin, and plenty more all within the
fraction of a second. In this document we will explore in detail some of the many processes that a search engine uses to comprehend a web
site.
—- Reader: Thank you for the banner and great placement on your forum, I will put links up for you on great sites for years to come. I will
help you a lot. Your a great guy to work with.

Website Marketing – Google Toolbar

If you have not done so already, I suggest you download and install the Google Toolbar. This is extremely important in finding information on
pagerank and other important factors concerning Google, it’s cache (last viewing) of your site, and directory information.
The Google Toolbar is free and can be downloaded here:
http://toolbar.google.com
Website Research – Alexa Toolbar

The Alexa Toolbar seems to be a great tool for website
related data such as site stats, user reviews, owner contact information, related links and archived copies via “The Wayback Machine”
(Internet Archive).
Alexa tracks and collects users web browsing activity when they download and install the toolbar. The results it gives are based much on the
data collected by millions of users. The toolbar itself is how Alexa estimates data on websites around the Internet. For those that do not
already know, Alexa is an extremely popular website which reports popularity on other websites around the web. Some professionals doubt
the accuracy of Alexa’s reports considering the information is only gathered by users who actually have the toolbar software installed on their
PC and that Alexa cannot gather information from those who do not. In any event, Alexa is still the worlds commonly excepted method for
measuring a websites traffic when the sites real stats are not available. The Alexa toolbar search feature is powered by Google.
The Alexa Toolbar is free and can be downloaded here: http://download.alexa.com Try Alexa stats here: www.alexa.com
SEO Search Engine Optimization / SEM Search Engine Marketing – The SEO Toolbar

SEO Incorporated, has released a serious add-on to the search engine marketing field with their SEO Toolbar software. This first of its kind
software enables a webmaster to simplify and enhance the difficulty of the everyday tasks of the serious marketing professional.

With one click of the “Get Data” button on the far right side of the toolbar, you are returned results of backlink data in multiple major search
engines and directories. Secondly, enjoy the ability to do multiple keyword searches in multiple engines within one browser by using the
search feature at left and selecting the target engine. It’s truly an amazing add-on to the field itself.
For more information on this toolbar, it’s features, and how to download and start using it, visit http://www.seoinc.com/toolbar/
—- Reader: I had a chance to check out the info on the link that you sent me, I am very impressed! Thanks for the help with getting this
new site up and going.
Just to rap up on toolbars, I personally use both the SEO TOOLBAR and the GOOGLE TOOLBAR – I suggest using both.

Just about every major search engine has basically 3 parts. The first is the spider, otherwise called a robot.
The spider visits a web page, reads it, then follows links to other pages within the site. This is what it means
when someone refers to a site being “spidered” or “crawled.” The spider returns to the site on a regular
basis, such as every month or two, to look for changes and updates. (If a site is updated often and is well
marketed, this will happen much more often, sometimes even every day)
Everything the spider finds goes into the second part of a search engine, the index. The index, sometimes called the database, is like a giant
library containing a copy of every web page that the spider finds. If a web page is different or appears to have changes, then the site will be
re-indexed and this “book” is updated with new information.
Sometimes it can take a while for new pages or changes that the spider finds to be added to the index. Thus, a web page may have been
“spidered” but not yet “indexed.” Until the new information is indexed, it is not available to those searching with the search engine.
The third, and most sophisticated part of a search engine is the ranking software (sometimes referred to as the algo or algorithm). This is the
program that sifts through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes
is most relevant. All search engines have the basic parts described above, but there are differences in how these parts are tuned. That is why
the same search on different search engines often produces different results.
—Reader: “I’ve think I’ve said before, I’ve always avidly read what you’ve written and admired what you done. To get your seal of approval
means a lot!!

Before I get started with my Traffic Blackbook Review I want you to let you know, why you should take advantage of paid traffic! It is getting harder and harder to get free search engine traffic! Especially after the recent Google Algo updates aka Google Panda, it is tougher than ever before to rank pages, getting search engine traffic and making money.

Here’s a search engine optimization concept that most people don’t think about: make sure you have keywords and key phrases in your TITLE tag. You know what the TITLE tag is, it’s the tag that gives you the name of the page on the Window frame in your browser, and it’s remarkable how few sites pay any attention to what’s in that critical search engine optimization (SEO) field.

Let’s take a quick tour of some big sites and have a look, shall we? HBO.com has a title tag of “HBO Online”. ESPN.com has “ESPN.com” as their title. No kidding. NYTimes.com is better, with “The New York Times > Breaking News, World News & Multimedia”, Microsoft has “Microsoft Corporation”, though, and, finally, BMW.com has “BMW International Website”.

What’s wrong with these? The problem is that each and every word in a TITLE tag is considered quite important by search engines (e.g, Google) when they figure out what your page is about and how relevant a given topic is on the page. Keyword density is definitely important in this regard, but one of the easiest ways to become more relevant to a given search result is to ensure that the keywords or key phrases you want to match are in the TITLE tag.

The downside is that sometimes the TITLES look a bit weird – as is demonstrated on this very site – but the upside is that if you want to have a site that Google thinks is an excellent match for, say, “acupuncture information”, then having a TITLE like “Acupuncture Information for Everyone” will yield a definite improvement.

If nothing else, please, do me a favor and don’t use “Welcome to”, “Home Page”, “Website” or any other empty words in your TITLE. After all, with all due respect to BMW, I think it’s pretty obvious that if I’m looking at their information on the Web with a Web browser that it’s a Website. So why bother saying so in the TITLE?

Frankly, for BMW, I think I’d suggest that they have a TITLE more like “BMW:Luxury Automobiles and Sports Cars from Germany for over 80 Years” which is still readable and friendly, but now it includes other keywords that can help with searches, making it a more relevant match for “luxury cars”, “luxury automobiles”, “sports cars”, “German cars”, etc. See how that works? Simple, but surprisingly effective.

So take five minutes and think about your TITLE tag. Is it doing the job you want? And keep in mind that Google and other search engines look at pages, not sites, so you need to ensure that the TITLE on every page of your site is helping your relevance with search engines.

This is still just search engine optimization (SEO) 101, but it’s important.

Search engine optimization, or “SEO” in the biz, isn’t only for people trying to turn their Web site into a revenue machine, to make money online, but can really be useful for everyone building Web sites. There are lots of different facets to writing, designing and adjusting your Web pages to maximize the chance of them being a top result for search terms, but one of the best – and easiest – is to work with keyword density.

What is keyword density? It’s basically a measurement of how relevant a given keyword “topic” is to a page of material. For example, this page is quite relevant to the word ‘keyword’ and the phrase ‘keyword density’ because both occur many times. More importantly, the ratio of their occurances to the total number of words or phrases on the page is reasonably high because, well, they occur a bunch of times.

That’s what keyword density is about. The keyword density of the word “keyword” is calculated by counting the total number of words on the page, then figuring out how many of them are “keyword”. Typical highly-ranked sites have at least a 2-3% keyword density for the key search word or search phrase.

But don’t take my word for it. Check out the keyword density of your favorite Web page at Search Engine World with their terrific – free - keyword density analyzer. To keep your sanity, I suggest that you set it to ignore words of five letters or less.

Of course, SEOs will tell you that keyword density isn’t the only factor to consider when building your page. Among the other important search engine optimization topics are so-called keyword prominence, that is, where on your page the keyword or keywords appear. A title tag is considerably more prominent than the alt text of an image, for example.

Nonetheless, it’s quite informative to search for a key phrase that you would like to have match your own site and then use the keyword density analyzer to see the density of top matched pages versus your own. Then add the phrase a few more times on your page, perhaps in the title or a h1 tag or similar, and try again.

And don’t be surprised if this change all by itself helps boost your site ranking on the search results.

Now that MSN has removed the wraps on its new search engine (beta.search.msn.com), intended to compete with both Google and Yahoo, the obvious question on the minds of SEO people everywhere is: what algorithm is MSN going to use for their pagerank calculations?

Microsoft is being predictably coy: their FAQ states: “The MSN Search ranking algorithm analyzes factors such as page contents, the number and quality of sites that link to your pages, and the relevance of your site?s content to keywords. The algorithm is complex and never human-mediated.”

Nonetheless, there are some useful tips that give you a little bit of insight into how MSN is approaching search. This is all quoted from their site, and broken into three categories.

Technical Recommendations

• Use only well-formed HTML code in your pages. Ensure that all tags are closed, and that all links function properly. If your site contains broken links, MSNBot may not be able to index your site effectively, and people may not be able to reach all of your pages.
• If you move a page, set up the page’s original URL to direct people to the new page, and tell them whether the move is permanent or temporary.
• Make sure MSNBot is allowed to crawl your site, and is not on your list of web crawlers that are prohibited from indexing your site.
• Use a robots.txt file or meta tags to control how MSNBot and other web crawlers index your site. The robots.txt file tells web crawlers which files and folders it is not allowed to crawl.
• Keep your URLs simple and static. Complicated or frequently changed URLs are difficult to use as link destinations. For example, the URL www.example.com/mypage is easier for MSNBot to crawl and for people to type than a long URL with multiple extensions. Also, a URL that doesn’t change is easier for people to remember, which makes it a more likely link destination from other sites.Dave’s comment: In case MSN didn’t notice, the majority of traffic to a site are from search results, so the complexity of a URL doesn’t matter as much as they are saying here. It’s an interesting insight into their ranking criteria, imo.

Content Guidelines

• In the visible page text, include words users might choose as search query terms to find the information on your site.
• Limit all pages to a reasonable size. We recommend one topic per page. An HTML page with no pictures should be under 150 KB.
• Make sure that each page is accessible by at least one static text link.
• Create a site map that is fairly flat (i.e., each page is only one to three clicks away from the home page). Links embedded in menus, list boxes, and similar elements are not accessible to web crawlers unless they appear in your site map.
• Keep the text that you want indexed outside of images. For example, if you want your company name or address to be indexed, make sure it is displayed on your page outside of a company logo.

You can learn more at the MSN Search Site Owner Help. Competition is always good, so it’ll be interesting to see what theories arise about how they’re ranking and ordering search results!

Maybe it’s just me, but affiliate programs for books, DVDs, even computer gear make sense to me. Even an affiliate program for a flowershop or bicycle dealer. But did you realize that matchmaking sites also offer affiliate programs? To find out more about it, I signed up for eHarmony.com, one of the top-rated dating sites.

The signup process was through CommissionJunction, one of the best affiliate management systems, and it took about two weeks to be approved. Once I was accepted by eHarmony, though, I found out these nice details:

“Join our Affiliate Program now and get up to $100 for every subscriber you refer. In addition, you will get paid $.50 for every registered user that completes our relationship questionnaire.”

A very nice payoff if you have a site that has anything to do with dating or finding compatible partners.

As is typical with affiliate programs, the eHarmony program has lots and lots of different banners and they make it pretty darn easy to integrate their advertising into your existing content. For example, here’s one of my favorite graphical adverts:

You can also utilize text links like this:

Dating For A Serious Relationship

which means that you can easily integrate your affiliate information into your prose – perhaps even in a weblog entry about dating, finding a good partner, soulmates, etc. It might look like this:

“After three months of trying the personals section of the local alternative newspaper, I was just about ready to call it quits. I mean, who knew there were so many weird women around? Fortunately, a pal of mine confided that he found his girlfriend by signing up for a great online matchmaking service. I was a bit self-conscious about it, but, yeah, I signed up, and ….”

You can see how that can be much more effective than a banner or graphical advert with many audiences!

Also, specifically with eHarmony, it’s good to know the target audience for the site before you begin to market aggressively. The introductory letter qualifies this very well:

“In order to successfully promote our service, it is essential to market the eharmony service to our target audience with an appropriate message. eHarmony is a service geared towards serious, mature-minded singles who are looking for a long-term relationship. We are especially successful with adult singles between the ages of 30 and 55. From time to time we will send you tips and suggestions for promoting our service and to help you maximize your referral efforts.”

So if you were managing a Web site for 19-24 year olds, this wouldn’t be the best choice for a dating service, nor would it be for the AARP crowd!

Fortunately, Commission Junction has that covered too. A quick click on “show me advertisers like this one” and you find out that there are a ton of different choices in this segment, from Yahoo Personals to Cupid Junction, Dating Direct to Platinum Romance.com. And even more unusual options, like Get Married Now, Gay.com and NeoDates (presumably, dates with people who really liked the film The Matrix?).

All joking aside, if you have a site where singles visit, you can definitely monetize that traffic and offer your visitors a valuable service too. And that’s a win/win in my book!

I bumped into the following tips and ideas about maximizing your placement of affiliate / pay-per-click advertisements on your page and thought they were some very sensible suggestions. The source of this material is the Equifax Affiliate Newsletter, which highlights a valuable, if rarely considered, additional upside to joining some of the major affiliate programs too: it’s in their best interest to help you identify how best to position and present your affiliate links, so they’ll often help you create the best possible pages for their products.

On with their suggestions:

While there are many different ways to place affiliate links, placing links in context with the theme of your site or within a specific category is a very successful technique for many affiliates.

Smart sellers know that product placement is essential to generating sales. This same rule applies to affiliate links on your site. Evaluate your site and organize your links in a way that makes the most sense for your audience to see, click, and take action. Keep the following guidelines in mind when designing your pages:

• Links placed in the upper right-hand corner of a site are more likely to be seen and clicked. Include special offers and new content in this location to maximum exposure.

• Try adding descriptive content around a link to improve link performance and let visitors know why the link is there.

• When incorporated into content, text links and product links have been proven to be much more effective link types than banner ads. When placed in context, product links can be closely matched to a visitor’s interests. Similarly, text links can be included directly into site content or product recommendations, providing a direct and simple transition between information that you provide and a product that you are advertising.

• Add a call-to-action in your site text or within the affiliate link. The words “buy this here,” “order now,” and “click here,” alert visitors to a commerce opportunity.

While the savvy among you may be saying “that’s pretty obvious”, it’s surprising how many sites offer affiliate links like this:

 Equifax Credit Report

Unsure about your credit rating? We highly recommend that you check out the Equifax Credit Report service today. It’s the best $9.00 you’ll spend this week!     [click here to learn more]

You can clearly see the difference, I’m sure. In publishing it’s the difference between “advertising” and “advertorials”, and the latter performs quite a bit better than the former.

After hearing a lot about it, I went over to SitePoint and read an interesting article by a “search engine optimization expert” wherein he enumerates his list of fifteen of the most egregious techniques by which companies and individuals “spam” the search engines. What’s search engine spam, you ask? It’s using inappropriate and deceitful methods of manipulating the HTML or other elements of a site to generate a higher ranking than the site would otherwise be granted by a typical search engine relevance calculation.

Now don’t get me wrong, I absolutely am against search engine spamming and other forms of ‘cheating’, so it’s not that I don’t agree with the premise of the article at all. What I have a problem with is more about whether the technologies and techniques that are singled out are really search engines spamming or not.

Wikis

For example, Wikis are singled out as a bad technology, yet a Wiki is just a minimalist shared white board, a technology that lets a group of people share the maintenance of Web-based content. The most popular is probably Wikipedia, which is a fabulous resource, but even Net-savvy publisher O’Reilly has a Wiki that they use to manage the interaction between the company, their authors, and user groups.

The argument of the article author, though, is that Wiki’s are dangerous because anyone can — theoretically — add content and therefore add bogus links back to a third-party site. Are Wiki’s therefore bad because people can “spam” them? Of course not.

Just like comments on a weblog or entries in a guestbook, pages on a Wiki should be monitored to ensure that the information thereon is relevant.

Networked Blogs

Another area of complaint: so-called “networked blogs”. Again, the article’s author arbitrarily decides what is and isn’t legitimate content, stating: “some spammers start a blog, plug it full of garbage content such as comments on what they thought at 5:15, along with a link or two and some keyword rich text.”

There are undoubtedly some people who exploit that idea, but I am far more reticent about deciding that a weblog where they talk about what they were thinking at a given time is garbage. As a quick example, perhaps the 5:15pm thoughts were interesting because the weblog writer had just gotten off work and knew that something frightening (or wonderful) was going to happen at 5:50.

Writing weblog entries or articles in a manner that allows certain keywords or key phrases to be repeated with some frequency seems like more of a smart way to ensure that your musings are rated as relevant with a search engine than otherwise. After all, in typical prose you might mention the subject of the comment once, then just refer to “it” and “the problem”, “the company” or similar, making it impossible for a search engine to know what is the subject of the article in the first place. (and that’s a good argument as to why you should also craft good titles for your entries too)

Who Makes the Judgment Call?

What bothers me about these two techniques that are highlighted is that we’re sliding from the overt spam techniques cogently discussed in the Sitepoint article (techniques including invisible text and link farms, both discussed in detail in Three Ways to Adversely Impact your Google Pagerank) to techniques that are really more of a judgment call. I imagine that the author of the original article, for example, would find a comment added to this weblog entry that pointed to someone else’s site offering ten smart ways to improve your search ranking to be spam, even though I wouldn’t necessarily agree.

What I’m trying to say here is that I’m in agreement when we’re talking about objective search engine spam techniques, but when we move into subjective search engine spam techniques, I’m a lot less comfortable with the entire topic and am confident that what I think is SES is going to be different to what you, the reader, would think is SES.

So my closing thought on this topic is that it’s always important to take what anyone says about search engine optimization — including what I say — with a grain of salt. Think for yourself, read the published criteria from important search engine sites like Google, and make your own decisions on how to approach the problem and what kind of results you seek.

The article on Sitepoint: Latest Search Engine Spam Techniques by Gord Collins.