dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
1512
share rss forum feed

kungpow

join:2002-10-06
Canada

Shaw SMTP - not generating NDR

We are using shawmail.vc.shawcable.net for our SMTP and I notice that it doesn’t send a bounced email back to the sender if the recipient email address doesn’t exist. It doesn’t send any NDR reports. It looks like the shaw smtp server is dropping the email completely when the user doesn’t exist. It doesn’t even hit the recipient’s mail server. How can we get the bounced email when the recipient doesn’t exist? I tested this with telus smtp and it works fine (I get the smtp bounced email saying the user doesn’t exist).

For example:
Smtp server being used: shawmail.vc.shawcable.net
From: TestUser@hotmail.com
To: TestUserNoExist@hotmail.com

The sender (TestUser@hotmail.com) doesn’t get any bounced email when sending to an non-existent email address TestUserNoExist@hotmail.com.


rotohoto

join:2012-03-31
canada
kudos:1

It won't send an NDR if the remote recipient doesn't exist, and the from address is not an @shaw.ca one. That's a pretty standard configuration choice to limit backscatter.
Frankly I'm surprised to hear Telus isn't doing that as well.

As for how you can get them to change that behaviour... pretty sure you won't have any luck there


kungpow

join:2002-10-06
Canada

said by rotohoto:

It won't send an NDR if the remote recipient doesn't exist, and the from address is not an @shaw.ca one. That's a pretty standard configuration choice to limit backscatter.
Frankly I'm surprised to hear Telus isn't doing that as well.

I'm surprised Shaw has SMTP backscatter enabled. If I send to an non-existent email address, how would I know if I had a typo in the email address if I don't get a bounced email? This makes troubleshooting 10 times harder.

I did a test with hotmail, gmail, and office365 and they all provide NDR reports for non-existent email addresses.

kevinds
Premium
join:2003-05-01
Calgary, AB
kudos:3
Reviews:
·Shaw
reply to rotohoto

Agreed, that is the way it is supposed to be setup.

A good number of years ago before this was the standard, my (or anyone else's) inbox would be filled with NDR emails from emails I didn't send, the From hotmail address in this case.

-Posted from my phone.


kungpow

join:2002-10-06
Canada

said by kevinds:

Agreed, that is the way it is supposed to be setup.

A good number of years ago before this was the standard, my (or anyone else's) inbox would be filled with NDR emails from emails I didn't send, the From hotmail address in this case.

-Posted from my phone.

I don't think this is a good shaw standard to have backscatter enabled as NDR reports are not generated. Shaw's outbound smtp spam filter needs to be upgraded.

I have tons of email servers hosting all sorts of domains with SMTP spam filter protected inbound and outbound and I have yet to have an issue with backscatter.

kevinds
Premium
join:2003-05-01
Calgary, AB
kudos:3
Reviews:
·Shaw

said by kungpow See Profile
I have tons of email servers hosting all sorts of domains with SMTP spam filter protected inbound and outbound and I have yet to have an issue with backscatter.

How many open relays do you have setup, with NDR messages enabled?
--
Yes, I am not employed and looking for IT work. Have passport, will travel.

kungpow

join:2002-10-06
Canada

said by kevinds:

said by kungpow See Profile
I have tons of email servers hosting all sorts of domains with SMTP spam filter protected inbound and outbound and I have yet to have an issue with backscatter.

How many open relays do you have setup, with NDR messages enabled?

No open relays enabled. The smtp connection is dropped immediately if you have a signature that fits the spam profile.


Jumpy

@shawcable.net
reply to kungpow

said by kungpow:

I don't think this is a good shaw standard to have backscatter enabled as NDR reports are not generated. Shaw's outbound smtp spam filter needs to be upgraded.

I have tons of email servers hosting all sorts of domains with SMTP spam filter protected inbound and outbound and I have yet to have an issue with backscatter.

To clarify, 'backscatter' in an email context is the term for actively sending NDR messages for mailboxes that you don't control. Shaw hasn't enabled backscatter, they're intentionally preventing it. Spoofed source addresses, be they spambot sourced or not, are a real problem for enterprise and carrier class mail services. It is not in Shaw's best interest to even let you send email out using a non @shaw.ca source mailbox, but they do. As a consequence, they need to protect their environment from the damage a spam bot or malicious user can cause, hence the refusal to deliver NDR messages off-server.

Consider the following.
The average spam message size is broken down to as follows (source, Symantec Intelligence Report: August 2012):
5KB, 10KB : 25.5%
The average infection rate in Canada was 1.4 in 1000 in 2010 (source, Microsoft Security Intelligence Report for 2010).
Shaw had 1.9 million internet customers (and assume a single computer per connection), as declared in their 3rd quarter financial results for this year.

If you take the average upload performance as recorded by Ookla (1.85Mbit/s) and apply that 1.4/1000 metric, and assume that these infected machines are only available for 30 minutes per day, and only send for 1/4 of their active state on average, we get the following math:
1,900,000 computers * (1.4/1000 infected machines per computer) = 2,660 infect machines on Shaw's network
2,660 infected machines / * 30 minutes = 79,800 infected machine minutes.
79,800 infected machine minutes * (1/4 active) = 19,950 active machine minutes.

Now lets mix in the average upload speed.
1.85(Mbit/s)/machine * (60/1 seconds per minute) = 111 (Mbit/minute)/machine
19,950 active machine minutes * 111 (Mbit/minute)/machine = 2,214,450 active Mbit

Now, if we assume we're working with an average of 50KB per spam message (MUCH larger than what is actually observed), we get
50KB / message * 8 bits per byte = 200Kbit / message
200Kbit / message * (1/1024 Mbit/Kbit) = 0.1953125 Mbit / message
2,214,450 active Mbit / (0.1953125 Mbit / message) = 11,337,984 message(s)

So, conservatively, you can assume Shaw has to deal with 11 Million spam messages per day from their network. Almost none of these will have legitimate source mailboxes, and if Shaw were to deliver the NDR for each failed, dictionary created destination mailbox, they would not be able to offer email service to their customers.

There are a couple things should help in the future:

1 - If you receive a new recipient mailbox, vet it first. Ensure that the mailbox exists and that the recipient actually wants your mail. If you don't do this you are not correctly handling the email addresses you come across and will likely encounter throttling or relay denial in the future.
2 - If you want to ensure the deliverability of a message, use a local test account as the source; i.e. in Shaw's case use an @shaw.ca mailbox.
3 - If you are sending mail from an email address that exists within a domain that lets you relay through them, always use that available relay.


Jumpy

@shawcable.net

Looks like the less than and greater than signs messed things up.
Should have been:

The average spam message size is broken down to as follows (source, Symantec Intelligence Report: August 2012):
(less than) 5KB : 44.3%
(greater than) 5KB, (less than) 10KB : 30.2%
(greater than) 10KB : 25.5%


kungpow

join:2002-10-06
Canada
reply to Jumpy

said by Jumpy :

As a consequence, they need to protect their environment from the damage a spam bot or malicious user can cause, hence the refusal to deliver NDR messages off-server.

If I send an email that's over the size limit (eg. From: TestUser1@hotmail.com To: TestUser2@hotmail.com), how would I know that it is over sized if I get no NDR since shaw drops them?

rotohoto

join:2012-03-31
canada
kudos:1

Shaw doesn't drop them. They don't generate them when they are going to be sent to external (non shaw.ca) addresses.

If Shaw can't confirm that testuser1@hotmail.com sent the message, which they can't, they should not be sending an NDR to tell testuser1 that his message was too big for testuser2 to receive.

In this scenario testuser1 needs to make sure he's using the appropriate outgoing mail server for his domain, which would be hotmail, not shaw.


kevinds
Premium
join:2003-05-01
Calgary, AB
kudos:3
Reviews:
·Shaw
reply to kungpow

Because you are not running open relays (shawmail.vc.shawcable.net is an open relay - or open if you are using a Shaw IP address). You can trust the From addresses going through your servers are the ones sending them, and sure, leave the NDR's enabled.

Shaw's open relay servers, can not verify if the emails are From who the emails say they are, so disabling NDR's is the right thing to do.
--
Yes, I am not employed and looking for IT work. Have passport, will travel.


kungpow

join:2002-10-06
Canada

said by kevinds:

Shaw's open relay servers, can not verify if the emails are From who the emails say they are, so disabling NDR's is the right thing to do.

Because shaw disables NDRs, it makes troubleshooting 10x harder when email issues arise.

I don't know of any other ISPs that disable NDRs but shaw. Telus doesn't disable NDRs and when email issues arise, troubleshooting is much more streamlined.

kevinds
Premium
join:2003-05-01
Calgary, AB
kudos:3
Reviews:
·Shaw

Then send with either with a shaw.ca address and you will get the NDRs, or use your own mail server.

I'll have to look up when I get back to a computer, if there is a RFC document, recomending NDRs be delivered locally, but not sent to another server over the internet.

-Posted from my phone.


tlhIngan

join:2002-07-08
Richmond, BC
kudos:1
reply to kungpow

Email, like SMS, is an inherently unreliable protocol. It's a best-effort service, and even with NDRs, there's a chance that the server does not deliver the email to the recipient. Or delivers it multiple times. Hilarity ensues if it's a mailing list (most have protections against receiving reflected messages nowadays, as well as from auto-responders).

The only way to ensure delivery is positive acknowledgement.

If you want, here are some common examples:

1) Mail delivered to target mailserver, but before user picks it up, hard drive crashes. You hope it doesn't take it down, but who knows.
2) User receives mail, but THEIR hard drive crashes, or they get malware. Either way, email is irretrievable.

Oh, and as a holder of a domain, I appreciate that I'm not receiving NDRs because spammers decided it would be the day to put my domain on the From part. Or emails about spam failing antivirus checks. Or that I need to click a URL to make it onto a white list.

If you're wanting to verify a mailing list, do the sensible thing and auto-remove people after a year or two. You can send them an email reminding them to renew their free subscription. Chances are, by month 3, they auto-delete your newsletters or whatever, or stopped reading them altogether having lost interest and being too lazy to unsubscribe.