This post will give a bit more depth on SmartScreen® and our approach to spam filtering, and on a particularly evil kind of spam called a phishing attack. I'll' also explain why individual e-mail accounts experience different levels of spam, and what you can do to reduce spam in your own account.
A deeper look behind the numbers
In my last post, I gave two key measurements:
Here's the math:
There are different approaches to measuring spam. Our approach is to use real user data to measure how much spam gets through our system. We select a cross-section of customers who reflect the broad population of Hotmail customers in several dimensions-such as age of account, country or region-and invite those customers to participate in our Feedback Loop program. The participation rate in the program is high, with more than 50% of participants classifying messages every day.
Another approach, which you see reported by some e-mail services, is simply to measure the rates at which users report spam using the "Junk" button or an equivalent. This approach suffers from a few flaws as a reliable metric, including false positives, biases in the feed, susceptibility to gaming, and dependence on customer enthusiasm to report spam. For many spam attacks, the rate of spam that gets reported is less than 2% of the total messages delivered. That's because most of the messages are never even opened or reported as spam. So, using this method of reporting can result in a deceptively low rate of spam ("under 2%!") even with no filter whatsoever.
We think our approach of measuring true spam by using a feedback loop is better. Our approach yields a statistical view of how Hotmail customers perceive a random selection of e-mail that we send to them for classification. It's the difference between a controlled experiment and observational studies. (But for those of you interested in the rate of "Junk" reports at Hotmail, it's consistently under 0.5%, which is comparable to what you'll see from other services.)
It can be difficult to make an apples to apples comparison of spam-fighting effectiveness across e-mail services. You're generally comparing your experience on different accounts, and there are several factors that affect the spam volume in an individual account, including:
Finally, it's worth talking a bit about perception and our approach to dealing with spam.
Each major e-mail service has a philosophy and criteria for dealing with spam. Some services put a large volume of messages in your junk folder. This might seem impressive, but it can make finding those false positives particularly hard, because you have to dig through so many spam messages. Hotmail, on the other hand, has a policy of aggressively deleting spam (although we never delete a message unless we're sure it's spam), and we also delete messages that have been in the Junk folder for more than 10 days. A customer who uses Hotmail as their primary account will see very few spam messages either in the Inbox or the Junk folder, and will occasionally find false positives in the Junk folder. Of course, classifying those false positives helps us make the system even better.
We think our approach of aggressively deleting spam and giving you the tools to control spam in your account makes a lot of sense.
The spammers aren't sitting still
One thing that makes fighting spam challenging is that spammers are constantly working to find new ways to exploit Hotmail and other e-mail services. Spam is big business and it only works if the spam messages (even just a tiny percentage of them) get delivered. There's a strong profit motive for spammers to find holes to exploit, so we see the techniques evolve every day.
What worked to prevent spam yesterday might not work today. That's why we take a balanced approach between long-term investments that will yield lasting improvements and short-term efforts to react to the latest attack. So, while we're working to keep the latest "cheap electronics store" message from showing up again, we're also focused on stopping any and all spam, period. It's a long battle.
Here's one of the graphs from my last post. The green triangles show when Hotmail released new spam-fighting technology. Those are typically the more strategic, long-term bets that yield major improvements, and you can see that they usually result in a substantial drop in spam. The blue circles highlight the day-to-day challenges of constantly evolving spam attacks-and the measures we took to tackle those attacks, as well as sustain the gains from our strategic investments. Those blue circles also highlight another reality of spam: there can be significant day-to-day variance in the amount that gets through the system. Some days you'll experience more, some days less.
Phishing
One of the most egregious forms of spam is phishing–a type of spam attack that attempts to acquire some sensitive information (like your password or credit card number) through fraudulent, misleading e-mail.
You've probably seen some of these phishing scams. Common scams include:
Our SmartScreen technology fights phishing scams by aggressively deleting or filtering these kinds of messages. SmartScreen uses several techniques, including:
SmartScreen is also built right into Internet Explorer, so you get the benefit of being alerted to phishing sites and other suspicious sites whenever you're browsing the web.
Here are some tips to avoid getting scammed:
You may recall from my last post that graymail refers to those messages in your inbox that are unwanted, but that aren't unsolicited. Common types of graymail include newsletters, social networking notifications, and alerts.
The "right" way to handle graymail is not so black and white; different recipients will disagree on whether or not a given message is spam. In fact, it's neither the content of the message nor the sender of the message that best determines whether or not the message is wanted; rather, it's your own relationship to the content or to the sender that determines whether or not you want to see the message in your inbox. What is perhaps most interesting is that your opinion on whether a particular e-mail is spam can actually change over time. (Advertisements for TVs are annoying until you're looking for a new TV.)
The good news is that Hotmail puts you in control. We provide several tools that help you decide what messages you do and don't want in your inbox, including:
Looking ahead
That's all for now. Next time around you'll hear from John Scarrow, my counterpart on the Windows Live Safety Platform team.
Until then, I hope you'll keep using Hotmail and keep the comments and feedback coming.
Dick Craddock,
Group Program Manager, Windows Live Hotmail
More...
A deeper look behind the numbers
In my last post, I gave two key measurements:
- Hotmail filters out 98% of all spam it receives.
- Spam in the inbox is under 5%, meaning that fewer than 5% of the messages in your inbox are spam.
Here's the math:
- We get about 8 billion messages every day. We deliver about 2.5 billion messages into the inboxes of our customers. That means we're filtering out about 5.5 billion messages. We know (from measuring after the fact) that about 110 million messages that get delivered are spam. So, that's a bit less than 5%:
110 million ÷ 2,500 million = 0.044, or less than 5% - We filter out all but 110 million spam messages out of 5.5 billion we receive each day, so that means we're filtering out 98% of spam:
110 million ÷ 5,500 million = 0.002, or about 2%
There are different approaches to measuring spam. Our approach is to use real user data to measure how much spam gets through our system. We select a cross-section of customers who reflect the broad population of Hotmail customers in several dimensions-such as age of account, country or region-and invite those customers to participate in our Feedback Loop program. The participation rate in the program is high, with more than 50% of participants classifying messages every day.
Another approach, which you see reported by some e-mail services, is simply to measure the rates at which users report spam using the "Junk" button or an equivalent. This approach suffers from a few flaws as a reliable metric, including false positives, biases in the feed, susceptibility to gaming, and dependence on customer enthusiasm to report spam. For many spam attacks, the rate of spam that gets reported is less than 2% of the total messages delivered. That's because most of the messages are never even opened or reported as spam. So, using this method of reporting can result in a deceptively low rate of spam ("under 2%!") even with no filter whatsoever.
We think our approach of measuring true spam by using a feedback loop is better. Our approach yields a statistical view of how Hotmail customers perceive a random selection of e-mail that we send to them for classification. It's the difference between a controlled experiment and observational studies. (But for those of you interested in the rate of "Junk" reports at Hotmail, it's consistently under 0.5%, which is comparable to what you'll see from other services.)
It can be difficult to make an apples to apples comparison of spam-fighting effectiveness across e-mail services. You're generally comparing your experience on different accounts, and there are several factors that affect the spam volume in an individual account, including:
- How the account is used. Do you use this address on lots of websites, for making purchases, etc.?
- The age of the account. Older e-mail addresses have had more time "out there" in the world, making them more likely to be on spammers' lists.
- The size of the e-mail service.
Finally, it's worth talking a bit about perception and our approach to dealing with spam.
Each major e-mail service has a philosophy and criteria for dealing with spam. Some services put a large volume of messages in your junk folder. This might seem impressive, but it can make finding those false positives particularly hard, because you have to dig through so many spam messages. Hotmail, on the other hand, has a policy of aggressively deleting spam (although we never delete a message unless we're sure it's spam), and we also delete messages that have been in the Junk folder for more than 10 days. A customer who uses Hotmail as their primary account will see very few spam messages either in the Inbox or the Junk folder, and will occasionally find false positives in the Junk folder. Of course, classifying those false positives helps us make the system even better.
We think our approach of aggressively deleting spam and giving you the tools to control spam in your account makes a lot of sense.
The spammers aren't sitting still
One thing that makes fighting spam challenging is that spammers are constantly working to find new ways to exploit Hotmail and other e-mail services. Spam is big business and it only works if the spam messages (even just a tiny percentage of them) get delivered. There's a strong profit motive for spammers to find holes to exploit, so we see the techniques evolve every day.
What worked to prevent spam yesterday might not work today. That's why we take a balanced approach between long-term investments that will yield lasting improvements and short-term efforts to react to the latest attack. So, while we're working to keep the latest "cheap electronics store" message from showing up again, we're also focused on stopping any and all spam, period. It's a long battle.
Here's one of the graphs from my last post. The green triangles show when Hotmail released new spam-fighting technology. Those are typically the more strategic, long-term bets that yield major improvements, and you can see that they usually result in a substantial drop in spam. The blue circles highlight the day-to-day challenges of constantly evolving spam attacks-and the measures we took to tackle those attacks, as well as sustain the gains from our strategic investments. Those blue circles also highlight another reality of spam: there can be significant day-to-day variance in the amount that gets through the system. Some days you'll experience more, some days less.
Phishing
One of the most egregious forms of spam is phishing–a type of spam attack that attempts to acquire some sensitive information (like your password or credit card number) through fraudulent, misleading e-mail.
You've probably seen some of these phishing scams. Common scams include:
- "You've won the lottery!" (and now you need to send us a small fee to collect your winnings).
- "You're cute! Want to go out?" and similar messages that attempt to get you to navigate to some for-pay site.
- "Your Hotmail account has been compromised and you need to re-enter your credentials." This scam is particularly heinous because it preys on your fear of having your account compromised in order to compromise your account! You also see this type of scam targeting e-mail services, bank accounts, and other financial institutions.
Our SmartScreen technology fights phishing scams by aggressively deleting or filtering these kinds of messages. SmartScreen uses several techniques, including:
- Authentication. The sender must prove they are who they say they are.
- URL reputation. SmartScreen inspects the URLs and links contained in the message.
- Content filtering.
SmartScreen is also built right into Internet Explorer, so you get the benefit of being alerted to phishing sites and other suspicious sites whenever you're browsing the web.
Here are some tips to avoid getting scammed:
- Never provide account credentials when a request comes through e-mail. Hotmail never asks for your account credentials in e-mail. (No other reputable service should, either!) Treat any such request as highly suspicious; generally, these messages should just be deleted. If you're unsure, visit the website in question by manually typing the address into the browser's address bar (don't click the link in the e-mail).
- Learn to recognize phishing scams by looking for commonly used techniques, like embedding links that look legitimate but actually take you to a fraudulent site.
- Be cautious when dealing with any e-mail involving credentials or financial information.
You may recall from my last post that graymail refers to those messages in your inbox that are unwanted, but that aren't unsolicited. Common types of graymail include newsletters, social networking notifications, and alerts.
The "right" way to handle graymail is not so black and white; different recipients will disagree on whether or not a given message is spam. In fact, it's neither the content of the message nor the sender of the message that best determines whether or not the message is wanted; rather, it's your own relationship to the content or to the sender that determines whether or not you want to see the message in your inbox. What is perhaps most interesting is that your opinion on whether a particular e-mail is spam can actually change over time. (Advertisements for TVs are annoying until you're looking for a new TV.)
The good news is that Hotmail puts you in control. We provide several tools that help you decide what messages you do and don't want in your inbox, including:
- Unsubscribe. Hotmail was one of the first e-mail services to offer the ability to unsubscribe from a newsletter. SmartScreen automatically detects newsletters and provides an unsubscribe link right in the header when you're viewing the newsletter. SmartScreen is, well, smart enough to do this only for reputable senders, thus avoiding another way for spammers to validate your e-mail address!
- Contact list. Your contact list is for real people whose e-mail you trust. Hotmail always puts e-mail from your contacts in your inbox.
- Safe senders list. Think of the safe list as the place to add commercial senders that you trust. Hotmail makes it easy by including a Mark as safe choice right in the message header. E-mail from these senders goes to your inbox.
- Blocked senders list. You can block e-mail from any sender by using the Mark as junk link in the message header, or by adding the sender's address or domain to your blocked senders list. Hotmail deletes future e-mail from the sender.
Looking ahead
That's all for now. Next time around you'll hear from John Scarrow, my counterpart on the Windows Live Safety Platform team.
Until then, I hope you'll keep using Hotmail and keep the comments and feedback coming.
Dick Craddock,
Group Program Manager, Windows Live Hotmail
More...