If you look at the source for this web page you will see the following:

<a href="http://youlookfab.com">this link to Angie's site</a>

Very simple, but this little fragment is the basis of the revolution that is the world wide web. Without it there would be no web sites, no web surfing, and no Google search.

There are essentially two benefits that Angie gets when I create a link like this to (youlookfab.com) YLF. First, the obvious one. People will click on the link and end up on her page. So I am sending her additional direct traffic.

The second benefit is more subtle, but also more powerful. By creating this link to Angie’s page I am telling Google that YLF is important. As a result Google may place YLF closer to the top of their search results, and then send Angie additional search traffic. This second bump in traffic has the potential to have a far greater impact. There are only a few people today on TheBlogEasy, but there are millions of people doing Google searches.

In order to explain the implications of this I will now make some gross oversimplifications about the way Google search works.

Google Search and Pagerank

Google has a little piece of software called the crawler. This crawler goes from page to page, following all the links and building a giant database that captures all the links on the web. Another piece of Google software then uses this giant database to answer your search queries when you type them in on the Google homepage. But how does Google decide which search result goes on top? That’s where the links come in!

Google looks at the number of links coming in to a page and uses this as a reflection of the page’s importance. This makes sense – popular pages will have many incoming links. Unpopular pages that no-one cares about? They won’t have any incoming links at all. So by linking to YLF on this page, I have sent a little signal to Google that YLF has some importance. Google uses the term “pagerank” to describe this importance measurement that is based on incoming links.

Paid Links: An Industry is Born

People are smart, and when they realized that Google was sending sites a lot of traffic they starting to think about ways that they could influence the Google search engine. They needed to convince it to put their site closer to the top of the search results so they could get more traffic. One obvious way to do this is to convince other sites to link to you so that the crawler would find these links, their pagerank would increase and Google would send them more search traffic.

This worked so well that people started to pay each other for these links.

And therein lies a problem. Google’s algorithm is relying on the fact that the links are natural in order to use them as an indicator of a site’s importance. When people start buying links, they are buying importance. Even if their site is horrible to look at and contains unreliable information, with enough money they can trick Google’s search engine into thinking that the page is important.

Then Google starts sending people to crappy sites and they lose confidence in the search engine. Not good.

Policing Natural Links and Punishing those who Distort Them

Needless to say, Google is very concerned about anything that distorts the natural order of things. This includes paid links and also link networks, where everyone in a community links to each other in order to get more pagerank. So a few years ago they started to lay out guidelines about what webmasters should and shouldn’t do with links. And they started penalizing sites that they judged to be breaking these rules.

Google isn’t transparent about their penalties, but there are widely believed to be three main ones. The -30 penalty forces your site to the 30th place in the search results even if it would normally rank much higher than that. The page 99 penalty (or -950 penalty) relegates certain pages on your site to the very end of the results (page 99) for certain keywords. The third and most serious penalty is exclusion from the index.

One good example here is the company Text Link Ads. They provide a marketplace for people to buy and sell paid links. That is, they make money by distorting the natural order of things that Google cares about so much. If you search for “text link ads” in Google you will not find their website. It has been banished.

Interestingly, the topmost result when I searched for “text link ads” a few minutes ago was an article reviewing the Test Link Ads service. So they are managing to get good placement despite having been banished.

The Nofollow Exception

It is possible to create paid links that don’t anger Google. All you need to do is add the “nofollow” attribute to any unnatural link. Modifying the link I gave above, this would look as follows:

<a rel="nofollow" href="http://youlookfab.com">this link to Angie's site</a>

The rel=”nofollow” bit tells the Google crawler to ignore this link, so it won’t be counted when calculating the pagerank of YLF. Of course, this doesn’t solve the problem for people who buy and sell links because they want links that do impact the pagerank.

What this Means for You

I believe that if you care about getting traffic from Google, you need to care about their guidelines. For this reason we do not pay for natural links to YLF, nor do we accept money to put natural links on YLF . When we do link to an advertiser in a post thanking our sponsors or in a review article where the advertiser provided free merchandise, we use the “nofollow” attribute. We are also upfront with advertisers about this practice.

Note that while buying paid links is one example of Search Engine Optimization (SEO) — the set of techniques one can user to make ones site rank well in the search engines — I am not implying that all SEO is bad. There are many good SEO practices that make Google a better search engine by making it more knowledgeable about your site’s content (e.g. using a sitemap). What is bad is anything that distorts the natural link order.

Could you get away with buying and selling links? Probably. Is it worth it? If you are highly dependent on Google traffic, probably not. You might also agree with Google that the web is better off with people linking based on quality and relevance of content, rather than pure dollars. In fact, if paid links started to dominate the web, then Google would have a much harder time providing good search results. And then there would be no SEO-related  reason to pay for links in the first place.

Tragedy of the Commons

This brings us to the tragedy of the commons. From Wikipedia:

The tragedy of the commons refers to a dilemma described in an influential article by that name written by Garrett Hardin and first published in the journal Science in 1968. The article describes a situation in which multiple individuals, acting independently, and solely and rationally consulting their own self-interest, will ultimately deplete a shared limited resource even when it is clear that it is not in anyone’s long-term interest for this to happen.

Will this be true of links on the web? Will commercial interests ultimately ruin the relevance that one can infer from a hyperlink?

What is the web server?

This is the PC that you connect to the Internet to “serve” web pages. Other PCs on the Internet then send it requests for a page (the home page, for example) and the web server sends that page back to them. People use a web browser to send the request, and then the browser also interprets your web server’s response and displays it nicely on their screen.

So the “web server” is this server PC, but when people say “web server” they also mean the software running on this PC that interprets incoming requests and sends back the response. In our case, we are using the “Apache” web server, but there are also other options, like Microsoft’s Web Server offering.

What gets served?

Obviously the web server is not all you need. You also need some content for it to serve. This is as simple as adding the content to a folder on the server PC that the web server software knows about. Normally the content takes the form of files in the HTML format and the associated images that they use. In the early days of the web, it was *only* HTML text files and images that web servers “served”, but these days the web servers are way more sophisticated and can serve much more interesting content (videos, applications, etc.).

What is a web application?

There was a time when web servers only served up static content. You put a bunch of text files, images and videos into that special folder that the server knows about, and then people could view that content with their web browsers. At some point someone added some functionality to the web server that allowed it to dynamically serve up different content under different conditions. So the web pages now included logic, not only content. They became “applications”, as opposed to mere “pages”.

Why the database?

This is easiest to explain with an example. Take a blog. You could make your blog by manually creating a page for the front page, and then pages for each individual page. Each time you added a blog entry you would manually add it to the front page, move the last blog entry off the front page, and manually create the separate entry in a separate page. Clearly this is a crazy amount of work for each entry.

With a database, you can store the entry text as a record in the database. Then you have a blog “application” running on your server PC, working together with your server software. It takes entries from the database and formats them nicely for visitors to your blog. This application can format the blog entries in an infinite number of ways, with no change to the entries themselves! So if you wanted your blog entries to appear in a grid instead of the normal chronological order, you would only need to change the logic in the application.

What is WordPress?

WordPress is a blogging application that runs on a web server. It has a back-end part that makes it easy for you to add entries to your database, and it has a front-end part that displays these entries to your visitors. It is a big pile of HTML, CSS, javascript and images, but you install it the same way you would install the simplest of HTML web pages – you put it in that special folder that your web server knows about. There is some additional configuration, most importantly setting up the database that it will use to store your blog entries, comments, etc., but it is essentially just a web application that runs on top of your web server.

What is hosting?

You could run your own web server at home. You would buy a PC, install Apache or Microsoft Server, and install WordPress. This would probably be unreliable though. You probably don’t have a proper cooled environment, with backup servers and a high bandwidth connection to your home. So people normally let a company that specializes in managing web servers do it on their behalf. This company “hosts” your web server. Logically, it is as if they put your server PC in their datacenter and managed it for you. You can log in remotely and add content to your web server, but they look after it and keep it running.

In my case Media Temple is hosting the web server. I installed WordPress on the server, but they are hosting the server itself. You can take this hosting concept even further. Sites like Blogger.com also do hosting, but they host the web server AND the blogging application. There is a big difference, because in this situation you have way less control over your blog, and you don’t have the option of installing applications other than the blog (a forum, for example).

One way to find out how your site ranks is to do a search on Google and see where you end up. This is time consuming, however, and there are online tools that make things a lot easier. The SEOBook rank checker is one such tool. Just type in the keyword and your site URL (in my case, the URL is “www.youlookfab.com”) and click “Check Rank”. The result comes back pretty quickly.

Here are some examples this morning (rankings in parentheses):

• smart casual style (1)
• structured clothing (1)
• what to wear to a black tie event (7)
• pretty tops (9)
• smart casual (10)
• how to get rid of deodorant stains on darks (10)
• dress over leggings (10)
• clothes to make you look slim (28)
• body type (44)

There are some mysteries about Google keyword ranks. For example, the keyword “what to wear” doesn’t rank at all for YLF. It only appears in the paid results on the right hand side of the page (because we pay to put it there – more on that in a future post).

There are many other tools like the one I used above. Some of them will also check other search engines, like RankChecker, which gives results for Google, Yahoo and MSN (Live). Note that most of these sites are covered in advertising for products and services that promise to improve your rankings in the search engines. Be very wary of these promises – there are a lot of good search engine optimization (SEO) professionals out there, but there are also a lot of snake oil salesmen.

Other default characteristics you can specify:

• Width and height
• Background color
• Headlines only versus Headlines and articles
• Whether images are displayed

Note these are only defaults. Your user can change these characteristics to suit their blog. Bear this in mind when creating your widget and try the different options to see whether they look OK.

Once you have created your widget it is immediately available on Widgetbox. See the YLF widgets here.

Likes:

• Quick and easy. It takes a few minutes to create a widget, and the result is easy for your readers to deploy.
• Monitoring tools. There is a nice little tool to monitor the performance of your widgets.

Dislikes:

• Limited creative. Widget customization options are a bit limited. I would like to make my widgets more graphic and “flat” (less gradient and less glass effect), but this isn’t possible. There is no way to change the text headlines to a different color, so you are stuck with their choice of blue. It might also be nice to have a background image.
• Ugly thumbnails. The thumbnail representation of my widgets on the Widgetbox site look pretty ugly, and they also reflect an older version of the widget in some cases. I think this might hurt my ability to get widget users, but I don’t want to design to their thumbnail.
• Neutered backlink. You don’t get a real backlink from the widget. People clicking on the widget will come to your site, but the Google crawler won’t recognize that link when calculating the ranking for your site. So if a blogger deploys your widget on their site instead of putting you in their blogroll, you lose the positive impact that the link would have had on your search engine rankings.

One thing to watch like a hawk with any widget is the overhead it adds to your user’s page load time after they have installed your widget. The widgetbox impact looks reasonable to the naked eye, but the widget does make quite a lengthy exchange with their server. Reload this page and watch the messages that appear at the bottom of your browser (e.g. “Read widgetserver.com” and “Transferring data from flash.quantserve.com…”). If I had a lot of other scripts on this site I might not be happy with this additional overhead.

Still, it is a very quick and easy way to create some nice looking, functional widgets. I think we will use these on YLF until we do something more sophisticated of our own.

The giant among search engines is Google. Nobody outside of Google knows exactly how their algorithm works, but if you search around the web you will find some guidelines on what to do and what not to do. The whole topic of making your site search engine-friendly is called SEO, or Search Engine Optimization.

There are two broad categories of SEO techniques:

1. Black Hat SEO: these are techniques that try to trick the search engines into sending people to your site. Don’t use these techniques. Getting caught might result in Google excluding you from their index, which will be disastrous for your traffic.
2. Legitimate SEO: these techniques just try to make it easier for the search engines to understand your site and its importance. Adding more relevant content to your website, making it easy for Google to figure out what your site is about and marketing your website to create awareness and incoming links are all legitimate ways to increase your ranking.

Keywords are one way to tell Google what your site is all about. You provide these keywords by putting some HTML before the <body> tag on your webpage. In most WordPress themes you would do this in the “header.php” file. For example, in the YLF theme I have the following code before the <body> tag:
<META name="keywords" content="fashion, body type, what to wear, style, casual wear, dress codes, wedding wear, career wear, footwear, fuller figure, instant style tips, party wear, swimwear, underwear" />
Now, there is a temptation to provide a large number of keywords so that Google associates your website with many different topics, but the wisdom from around the web is that too many keywords will reduce your ranking. Based on this I cut down the YLF keywords a short while ago. It used to be:
<META name="keywords" content="advice, fashion, body type, what to wear, pear shaped, dressing well, trends, style, fashion you can wear, shopping, dress sense, fashion persona, casual wear, event wear, dress codes, dresses, wedding outfits, look slimmer, accessories, power dressing, feel good fashion, look good in your clothes, career wear, celebrities, clothing care, color, european style, fabrications, footwear, fuller figure, global shopping, individual style, fashion industry insider, instant style tips, maternity wear, men's style, party wear, swimwear, underwear" />
Our organic search traffic from Google is still down, so I think I’m going to switch back to the longer list of keywords for a while to see if that helps. You should experiment with different sets of keywords to see what works for you. Note that it can take Google quite a long time to review your site, so any one experiment with keywords should probably be run for at least a month.

I’ve never seen much direct traffic from these directories, but getting listed might help to raise your position in search results pages, particularly if the directory itself has a high page rank. Search engines like Google use the number of incoming links to your site as one factor they take into account when deciding how important your page is on the web. More links from popular sites (like a highly ranked directory) will make your site more popular in their eyes.

Today I added YLF to…

• www.lii.org
• www.turnpike.net
• www.searchwiz.com

…then I ran out of steam. Traffic2MyPage.com has a list of the top 500 directories sorted in page rank order.

1. Created a database on one of our servers
2. Downloaded WordPress 2.6 from WordPress.org
3. Set up the “wp-config-sample.php” file and renamed it to “wp-config.php”
4. Used ftp to put the contents of the unzipped WordPress folder in the html root folder
5. Browsed to the installation script and followed the steps
6. Changed the default “About” page text to something more appropriate
7. Wrote the first blog entry (this one)
8. Enabled pretty permalinks

Yep, definitely the blog easy.

[update] Easy maybe, but this sure looks ugly. Next step will be to install a better theme.

[update 7/27/2008] Installed the wp-andreas01 theme and modified it a bit – much happier now.