.comment-link {margin-left:.6em;}

Pinging Your Blogs to Search Engines

Tuesday, July 26, 2005

What is the difference between a Blog and RSS feed?

There seems to be a general amount of confusion over the difference between a blog and an RSS feed. While it's true that RSS feeds originated with blogs, and often are accompanied by blogs (just like this one), RSS feeds have much more importance than simply being an extention of a blog.

Let me start with a general description of what a blog is and what an RSS feed is.

A blog is really just a web page, usually in the form of a journal, guest book, or news page. The page is frequently updated with articles, maybe daily. The difference between a normal web page and a blog is really not that great. It is just a fancy term picked up in recent years to describe the type of content in the web page. Ok, so thats what a blog is.

An RSS feed is an XML formatted file containing article content. This content can come from anywhere, but usually it comes from a web page, such as a blog. An RSS feed provides web page content in a standard format so that 3rd-party programs called "aggregators" can read and display the content to users in the way they choose to see it.

Why would users want to read an RSS feed when they can just see your web page? Because the RSS feed is direct and to the point. There are (usually) no banner ads, no graphics, no navigation bars, etc. Just content. And the content is displayed how the user wants.

Why would a web site want to offer an RSS feed? Wouldn't this make less visitors come to their site if they just read the feed instead? Actually, stastics have shown that *more* users visit the web site through links in the feed. RSS feeds provide a convienient method for users to read more web sites in less time. This means more users have the chance to read your web page content, which leads to more clicks back to your site, and more repeat visitors.

Now for the web masters out there. You probably want to know which is more important to have and promote - a blog or an rss feed? The answer is both!

Having a blog gives you constantly updated content (as long as you have the time to update it), interested readers, and most importantly - a blog to ping to the blog search engines like Technorati. These blog search engines instantly re-index your site when you ping them. This is great for traffic and search engine rankings. I highly recommend starting a blog even if you can only update it once per month. Instant search bot gratification.

Having an RSS feed gives you the benefits already discussed and allows you to get listed in over 100+ RSS-specific directories. This leads to higher search engine rankings and more traffic from readers.

In summary, it is important to start up a blog and get an RSS feed going. Promote both items by pinging your blogs and submitting your rss feeds.

Monday, July 18, 2005

RSS spam and how to deal with it

RSS feeds are starting to appear everywhere as people realize the great traffic asset that they can provide. However, just like any other attention getter, RSS feeds are facing a growing threat from spammers making RSS spam.

What exactly is RSS spam?

RSS spam is an RSS feed which usually contains confusing paragraphs of text for each article, meant to bypass bayesian spam filters. The articles usually look like authentic RSS articles, but if you read them, you can tell they make no sense and just contain keyword-rich content.

Clicking any of the article links take you over to a similarly spamish web page filled with keyword-rich junk content and, of course, Google Adsense ads. How else do the spammers make money?

I thought I would jot down some thoughts on possible ways to handle RSS spam on the RSS search engines like ReadABlog http://www.readablog.com and the others.

The first method is the typical blacklisting of spammy keywords like viagra, replica watches, etc. But RSS spam tends to be much more variable, covering many topics just to get google clicks from unsuspecting users.

A better method would be to implement a bayesian spam filter to check the RSS feeds against "good" RSS feeds and "bad" RSS feeds. This may provide better results, as spam feeds are almost certain to contain keywords to indicate it as spam. But what about the feeds which are created to bypass these filters?

A third method could be a combination check. This check would first include scanning the RSS feed URL and article URLs for spammish characters. Lots of dashes in the URL tends to lead to a spam web site such as viagra-pills-low-viagra-cost.com. A second part would include fetching the target web page of each article link in the feed and seeing if a Google Adwords script is located in the resulting page. Perhaps a check on the width, height, and position of the Adwords box would better help the check. Spam web pages usually contain a large Google Adwords box at the top of the page (near the top of the HTML code) and almost always use the two largest Adwords options for size. Legitement web sites rarely use such large Adwords boxes, and if they do, they are usually located further down in the page.

Perhaps an even better solution would be similar to what is trying to be implemented with email using the authentication check "Received-SPF". This is a way of telling if an email is coming from who the sender says it is coming from. An update to the RSS schema might allow a way of signing and authenticating an RSS feed, although I am not sure how this could be done without bringing in a 3rd-party to validate a feed, similarly to how software is signed by Verisign etc - RSS feeds could be signed.

These are just some ideas on RSS spam checking. The problem is only going to grow as RSS becomes more popular.


Tuesday, July 12, 2005

Is Podcasting Here to Stay?

Recently, I've been experimenting with podcasts and trying to find out what the hype is all about. Podcasting is really just creating an RSS feed with embedded audio or video links, also called enclosures. Users can download your podcast, just as they do with an RSS feed, and listen to your audio recording, such as a news channel or radio station etc. For instance, try using GetAPodcast http://www.getapodcast.com

Big deal, I can turn on my radio and hear that too, so what's the big deal?

Well, the big deal is beginning to gently sit on a little device called the iPod. The iPod plays music, namely mp3s. The iPod is tiny, light, and extremely convenient. The iPod is just like a walkman, except it doesn't pickup radio. Bingo.

Podcasts of your favorite web DJ's, radio shows, or information channels can be downloaded to your iPod. You can then go out to the gym, walk your dog, whatever, and listen to your podcasts instead of to a song.

What a revelation!

Yes, it's true. iPods are not just for listening to music, but for news and information too!

What does this mean for those of you looking to grab hold of the budding podcast technology? It means as more users purchase iPods, more users will thirst for audio other than music. People will begin seeing how darn handy it is to listen to all of their RSS feeds play while they drive their car in traffic, instead of sitting in front of the computer to read it.

What do you think? Will podcasts take off? Maybe. The technology is still too new to really tell. One thing is for sure - RSS technology is definitely beginning to make its mark.


Monday, July 11, 2005

Speed of getting Googlebot and Yahoo Slurp to visit a new site

I recently performed an experiment of setting up a brand new web site domain name and hosting, creating a simple blog for it, and a simple rss feed. I then did a ping over to the blog search engines using Blog Blaster and submitted the RSS feed using RSS Submit.

Would you believe that within 30 minutes of this new domain being registered, I already had over 30 different bots visiting the site? Not only that, two of the bots were Googlebot and Yahoo! Slurp bot.

Side note - yes, the domain and web hosting was already online within 30 minutes and available to the world. New domains resolve incredibly faster than the 72-hour wait time they used to about 3-4 years ago.

I'm not sure how Google learned of the new domain that fast. Perhaps, the bot was already crawling one of the blog sites at the time, or they might have some kind of back-end monitoring of one of the big blog sites like Technorati, looking for new URLs.

Either way, 30 minutes is the fastest I have seen Google and Yahoo visit a site - and I have setup quite a number of new sites. The usual wait time is about 2 weeks.

Just goes to show you, once again the power of blogging and RSS feeds wins the day!


Sunday, July 10, 2005

A few podcast search engines

I've been catching up on the whole podcast movement, very cool stuff. I just thought I would provide a few links to some of the better podcasting search engines I've uncovered.

GetAPodcast.com http://www.getapodcast.com has a lot of interesting real-time statistics on their site such as podcast traffic, number of media files per feed, active keywords. They also appear to have instant podcast inclusion if you care to submit your own podcast. Submit your podcasts at http://www.getapodcast.com/addfeed.aspx

PodcastPickle.com http://www.podcastpickle.com provides fairly good quality podcasts. They review each feed by hand to weed out low quality feeds. While this is nice, I appreciate the more raw search results that getapodcast.com provides because it gives voice to the big and the small podcasts.

I know many of you will be asking, just what is this "podcasting" business all about? I will be posting another article shortly on this. Until then, check out the above sites.


Wednesday, July 06, 2005

Searching blogs with ReadABlog and BlogDigger

Technorati is great, I'm sure many of you already search using it. But there are also a few other good players out there. I've decided to write about a few of my favorites.

ReadABlog http://www.readablog.com is an RSS and blog search engine which happens to provide real-time information on feeds being viewed, popular listings, and the most interesting part - active keywords being used. Their results seems to provide a more raw approach to RSS and blogs, which is nice for a change. They have a built-in reader to view the RSS and blogs, so no external feed reader is required.

BlogDigger http://www.blogdigger.com is another nice blog search engine. Many of you have probably already seen this one. However, have you checked out their media search? It provides a blog and podcast search for feeds containing enclosed media such as images, movies, and audio files like mp3. I know you can just do a search on Flickr http://www.flickr.com if you want photos, but searching a media or podcast directory provides a greater variety of types.

Speaking of Flickr, their site happens to be going through an update today and is not currently available. If you are not familiar with Flickr, they are quite an amazing photo search engine. If you get bored or have some extra time, try a search on their site once they are back online.


Tuesday, July 05, 2005

How to monitor RSS feed traffic

Since an RSS feed is just a plain XML file, by itself, it has no way of recording hits. How would you know if anyone has subscribed to your RSS feed? How do you know you're not just writing to an empty audience?

Well, there are a few ways to track RSS feed traffic. One way, would be to just look at your web server statistics and see how many hits your RSS feed has. However, if you have a blog like this one, hosted by a free blogging service, you probably don't have access to web logs.

Instead, there is a pretty cool service available, and it's free. It's called FeedBurner http://www.feedburner.com.

What you do is, visit their web site, sign up for a free account using the standard stats account. You type in the URL to your RSS feed. They then provide you with a new RSS feed URL. This new URL goes through their servers, allowing the ability to keep track of hits, traffic, content, and a lot more. You would then go back over to your blog, and insert a little RSS graphic link on your page, indicating users can subscribe to your RSS feed. For the URL to your feed, just stick in the FeedBurner URL they provided to you.

Here is what the link in your blog might look like:

What's even cooler, your new FeedBurner URL does a lot more than just redirect to your RSS feed. It can actually let users who don't have an RSS reader still read your feed by "pretty-printing" it in the web browser. It also displays a friendly message explaining how to get an RSS reader. Seeing as how RSS readers are not mainstream yet, this is certainly a handy feature.

Another cool feature is FeedBurner's counter. You can link to a special counter that displays the current number of subscribers to your feed. It looks just like a web page counter - except it's an RSS feed counter!

So, what does all this mean for your blog? Well, it means now you have a way to count your RSS subscribers. Now you'll know how many people are listening!

What's that you say? You don't have an RSS feed yet? Well then you better find out if you already have an RSS feed or make one. Also make sure you submit your feed too.


Monday, July 04, 2005

How do I use Technorati Tags?

I know it's a holiday July 4th, but I have a few minutes for a quick lesson blog on using Technorati Tags.

You may have noticed at the end of each article in this blog are a few Technorati tag links. You might have been asking yourself, what is the meaning of a tag?

In the most basic sense, a Technorati tag is just like a keyword meta tag that you would put on your web site. The tag is a way of specifying a category keyword to classify your blog article. Since blogs do not normally use meta keywords, you can instead use tags.

Currently, Technorati is the only blog search engine (that I am aware of) that uses tags for searching.

Ok, here is where it gets cool. Cool means traffic to your blog.

A user can go to Technorati http://www.technorati.com and enter keywords for a search. However, instead of just clicking the Search button, he can click the Tags button located at http://www.technorati.com/tags/. The results returned will be the most recently posted blogs containing the tags he specified. The user will also see tags relating to the search.

What does this mean for you? It means by entering a few Technorati tags in your blog, you can easily categorize your content and attract readers interested in that category, just like meta tags, except since blogs are a smaller game, you get better results!

Hopefully more blog search engines will be picking up on the tag idea, which will make it even more important to include tags in your blogs.

Now you probably want to know how to use Technorati tags in your own blog? It's pretty easy. Just include the following html somewhere in your blog and replace the keyword with your own:

< href="http://technorati.com/tag/technorati tag" rel="tag">technorati tag< /a>

That would be a tag for the keyword "technorati tag" . Replace both instances of "technorati tag" with your own keyword.

As always, after adding the tags or making any changes to your blog, make sure you ping your blog to the blog search engines using your favorite tool. I prefer Blog Blaster.

Check out the tags at the end of this article.


Saturday, July 02, 2005

Live8 and The Power of the Blog

If you have any doubt about the power of RSS and blogging, perhaps the world concert event of Live8 or Live 8 will help clear that up. A simple blog search on any of the engines for "Live8" returns an amazing number of up to the minute results. For example, just searching on ReadABlog http://www.readablog.com or Technorati http://www.technorati.com, I was able to find quite a few blogs already written on Live8.

Most people are upset with the media coverage of Live8, specifically MTV and VH1. They are showing too many commercials and documentary, and not enough of the actual music. I certainly agree. However, no matter the opinion, the point is clear that blogs are certainly a powerful entity.

Are you looking for the latest conversation and news (by latest, I mean minutes old), then you need to search the RSS feeds and blogs. Forget about trying Google or Yahoo, their content is days old at best. Be sure to publish your own blog and RSS feed and don't forget to ping it to all of the aggregators.


Friday, July 01, 2005

Recent Ping-o-matic Server Problems

For those of you not familiar with it, Ping-o-matic is a serivce which allows you to ping your blogs to 12 blog directories by just submitting it to them once. You can submit your blog through their web site, or even better, through their XML-RPC interface. (The XML-RPC interface is a way to ping your blogs through a direct channel instead of visiting their web site each time)

However, as their service is new, the web site and their XML-RPC service goes down a bit too often, leaving you without the ability to ping the blog aggregators.

There is a nice software tool available called Blog Blaster http://www.dummysoftware.com/blogblaster.html, which allows you to ping over 30 blog aggregators with one click. This software includes Ping-o-matic in their list of sites to ping, in addition to a lot more. I highly recommend checking it out as an additional asset or even as an alternative to just using Pingomatic.