Comments are always welcomed and appreciated. However, comments are also one of the biggest causes of spam on websites. If you do not have a proper comment moderation and protection system, your site will be flooded with links to spam, phishing and other unwanted sites.
Akismet as always does a pretty good job at tracking most of the comments. However, recently I have started seeing a new way of comment spam which uses a different tactic. With this new form of spam, spam commentators have started to use short URLs such as bit.ly and is.gd among others as their website URLs, as well as within the comments itself.
The problem with such URLs is that, it is usually a bit hard to tell where it is linking to without actually clicking on the link, and this in turn leads many users to skip checking it, specially on highly commented blogs. If that URL turns out to be spam, it would end up affecting your readers who would click on that link.
I do have a plugin idea which will detect short URLs and then try to fetch the final destination of the URL and display it in the comments administration panel, unfortunately I do not have much time to work on it, so it would be great if someone could come up with a similar plugin.
Are you too seeing this form of comment spam on your blog? Do you verify all the shortened URLs used in website field or inside the comment?
Update: I had written about a similar form of spam back in July 2009. However, it was only happening when you pulled in twitter comments into your blog. You can read more about it at Twitter Spam Spreading to Blogs
I use several online services to reveal the destination of shortened links. One such service is expandmyurl.com There are many others.
@Len what about embedding loic into a url shortner and loading the target via an iframe
http://spareclockcycles.org/20.....shortener/
I use the same as Len. However, I pay attention to the message, than name of the commenter, the email, and the url given as well. You cannot fully rely on Askimet and other spam related plugins to remove everything. If a site owner cares about their readers, they will check, even if there are a multitude of spam comments. I remember when I was a cofounder of 3 top 10 Yahoo! Clubs (the largest well over 50K and the smallest of the three over 23K. And this was back in 2000) I cut a lot of spam out per day… more than a couple hundred, not including the deletion of the spam accounts. It really was an exercise for skimming…lol.
Why restrict the plugin to short URLs? Why not make it follow all URLs to find out their final destination?
Should be dead easy to implement.
@Jeremy – Short URLs usually lead to unknown places, when you can visually view the full URL there is actually no need to know the final destination.
Yeah, but it can also use 301 redirection so you won´t see final destination in those cases.
I have a function which takes a big portion of the short url searvices and replaces that url in comments with the real one. although akismet picks up most of them on its own
I always delete the URL formfield in the comments area of websites I build. Plus I always encourage 100% moderation of all comments before they are posted (unless the commenter has 3 or 4 previously approved comments). This way, a commenter can only provide a Name and Email. Then, if a spambot comment posts a comment and I see a URL, I automatically know it is spam and I can reject it (this is if Akismet does not notice it as spam).
I haven´t encountered this type of spam actually.
But I mainly look to the comment body. If it doesn´t appear to be reasonal to the topic of the post, it´s just weird or anything that makes me feel odd about it then I mark it as spam.
So with a shortened URL as a link from the commentators name will most likely be marked as spam. It isn´t hard to write down your URL once and then just use the dropdown menu that appears when typing in URLs.
Given a short URL and a weird comment body and I will definately mark it as spam.
That plugin idea sounds pretty handy. I’d use it for sure.
I really haven’t thought about that before. But I do even follow some regular links (domain names are weird sometimes). For blogs like mine where I don’t get a lot of comments, following those links shouldn’t be a huge deal but what if you get 100’s or 1000’s of comments every day? A Plugin would definitely be handy.
I have my own short url service, but as far as checking others, I’m a little hesitant to click on a short url for fear of getting a virus, etc. I guess it’s a good idea to check the short urls for spam if there’s a better way without getting infected. Firefox seems to do a decent job of protecting users if you reach a malicious site, but I can’t say about IE since I haven’t use it in years.
I get so few links in comments, I still check every stinking one.
Why not write a message in the comment form that encourages people to write the full URL instead of some short URL.
A plugin can automatically reject known short URL services, asking the commenter to enter the full address instead. This is also not “very rude.”
As far as general redirects are concerned (well, anyone can use a custom domain for redirection), we can’t automate screening them to a large extent.
Since volume of comments I get are less, I generally visit the target website to see if it’s “at least a website.” A comment body can easily tell you the real intention of the commenter in most cases. You’ll soon develop a knack to differentiate between “real comments” and “filler comments.”
I haven’t seen this sort of spam, yet, but I also have every comment with 1 or more links put to moderation just in case.
If this short URL issue becomes more prevalent on my blogs then I will definitely be looking for a more effective method of review than clicking through (which could be putting money into the spammers’ pockets!).
Honestly, i one of them before. I spam many shout box with this technique. But — yeah — i already aware that’s only good to give bad reputation so i do not do it anymore.
I don’t have this spam problem yet, but it’s a interesting idea for a useful plugin and like jeremy said should be really easy. I’ll give it a try.
I don’t see why Akismet doesn’t just have a crawler that checks the HTTP headers of the short URLs and grabs the original URL…
I haven’t seen this particular technique yet, but I’m seeing shortened URLs in email spam, so it’s surely just a matter of time before they show up in comments.
Thoughts:
1. Akismet should be smart enough to expand the URLs internally and act according to that info. Maybe it is, maybe it isn’t, but I’ll bet that they can add that on the backend anyway.
2. A plugin that auto-expanded short links into long ones for comments would be a good idea. It should actually replaced the links too, I don’t want short links in my comments at all, really.
2a. Problem: Too many shortlink services. How do you expand links? Many link services have APIs to do this, but even those are somewhat difficult. bit.ly, for example has an api which can do expanding links, but it requires the user to sign up and get an api key. A generic way would be preferred.
2b. One generic way would be to do a get on the link and see if it is a redirect. This is probably highly problematic for other reasons.
@Otto – Agree with both 1 and 2
WRT 2a and 2b, yes the simple solution as suggested earlier by @Milan would be to check HTTP headers and detect a 301 and 302 error. However, that is again an expensive operation as it would really slow down everything as you have to load each and every URL before actually displaying it.
So all in all, if Akismet does not have this feature built in right now it would be one that should definitely be a part of it.
Well, actually I’d do all that at the time the comment was posted, not in real time when it was displayed. Resolve the URL to the final URL before sticking it into the database.
I thought of that too, not checking on each comment load.
I mark as spam every comment where the user sets their URL fields to a shortened URL. I don’t even bother checking. If a reader can’t tell where the URL goes to, I consider it spam. Short URLs are appropriate for Twitter/identi.ca but not other places.
I always try to check links if comment and name seems to be “normal”.
But i have had a big problem with spam links in comments but it dissapear by it self for a while ago.
Well that was easy. here is the first version of the Preview comments short URL plugin http://onlinevortex.com/previe.....-comments/
I don’t get that many comments with attempts to link. If it has a url that appears bogus, it is deleted. Even if some appear OK, I don’t want it going back to another companies site anyway. So 99.9% of links are deleted.
Not usually a problem for me… and I would most likely delete it if it were from someone I didn’t recognize.
I don’t mind legitimate commentors linking out if they are adding to the discussion and not directly competing with me economically.
My comments are on moderation – ppl don’t like it, oh well.
I look at the e-mail and the content of the comment.
I moderate all comments, except for previously approved commenters.
On a newspaper website I disabled the URL field. I agree with previous commenters who said look at the email address and content.
Akismet is letting a few more through lately, I noticed.
Thanks for listing expandmyurl.com I had some suspected comments today and i couldn’t remember the site address.
I delete a comment if its got a URL in it of ANY kind!
The way I see it, a discussion forum is the place to post URL’s to people, not a comment on a blog, so therefore, I delete ALL URL’s. I consider a URL of ANY kind in a blog comment SPAM.
I’d just remove it. I don’t like these ugly URLs
I dont see any legitimate reason to use a shortened URL in a comment.. Havent seen them being used much, but I instantly delete comments that use them. Wouldn’t want to waste my time and effort by checking the intentions of the commenter (verifying that it is indeed spam).
A ‘Short URL’ in a blog comment does trigger major red flags. Personally I always check them because there is no reason at all to shorten URL’s for a blog-comment.
But then again, you’ll have to take the time to do so. And if you have larger websites with more visitors I can understand why people don’t want to do it (manually).
It could be potentially dangerous to and I feel that anyone leaving a URl should leave a full URL and therefore I’d probably not check nor accept the comment.
I always delete comments that place a URL in the body unless I can see straight off that the commenter is specifically helping the discussion thread by pointing us to something of use. I’ve not come across bit’ly URL’s yet but don’t know why people would use them in a comment as there is no restriction on space, unlike Twitter, so I would be immediately suspicious.
I get these occasionally but do not have the interest nor the time to track them down, so they are simply deleted. I don’t mind when someone leaves a comment that is at least adding something constructive. The shortened urls are a liability as you can never tell where it will lead until its clicked. If you don’t want anyone to see where it is heading there most likely will be something fishy on the other end.
I think what bothers me more than the shortened urls are the comments that are clearly and blatantly not even remotely close to the subject of the post…instant delete from me.
Look at the commenter’s name and the message of it, it’s not always that short urls are bad rigth?
I have never thought about this issue. Most of the comments that looks genuine and related to the post will get approved. Usually, I will check the link manually if it spells badly with meanings smells like adult, gambling or pharma related niches. Never thought of this before. Thanks for sharing this, it is time to check my recent comments with all those tinyurls…
To me, I always read the comments first and if they don’t have the quality I’m looking for, and if they link on to their site using the comment box, I always delete them.
Anyways, I learned alot on this article. Thanks.
If i see a suspicious shorten url, I check it manually by clicking it before approve it as a legit one.
I’m using a comment form by IntenseDebate. The plug-in automatically holds for moderation any comment with at least 2 links (shortened or not), including the link to the commenter’s name.
I am using akismet and another plugin to detect spam comments. Yes, they are effective but what if Real Humans are that ones who are posting this spam comments on our posts? These are really hard to control, we really need to manually check every comments and delete the ones with links on it, especially those bitly links. Most of them consist of unwanted programs or contents that can ruin our site’s reputation. If I can see spammy links on my posts, definitely deleted it or editing it and removing the links away.
So what I need to be doing is checking my site better. I really hate the fact that if they are using tiny url’s it could be hurting me by linking to sites that aren’t anything to do with barbecue.