dslreports logo
site
spacer

spacer
 
   
spc
story category
Study: Carriers Inaccurately Track Wireless Usage, Overbill
Could be Fixed, if Carriers Actually Had Incentive
by Karl Bode 04:33PM Friday Sep 14 2012
Despite their rush toward caps and overages, landline broadband ISPs are fairly awful at accurately tracking broadband usage, and a new study suggests that wireless carriers aren't much better. According to a new study University of California, researchers found that two of the largest (unnamed) U.S. carriers consistently over-estimate subscriber usage and thereby overbill their users, specifically when the users travels in and out of roaming situations. From the report:
quote:
The researchers used a data-logging app on Android phones to check the data use that the carriers were recording. The carriers were found to usually count data correctly, but they tended to overcount—and hence potentially overcharge—when a person used applications that stream video or audio, and particularly when coverage was weak or unreliable. The researchers determined that even typical use of a phone could lead the data to be overcounted by 5 to 7 percent, Peng says."
Travel into unreliable areas means users are often being charged for bit delivery they never actually received:
quote:
They count data as it leaves the heart of a company's network and sets out on the journey to the mobile tower nearest a subscriber. That means data is counted whether a phone receives it or not. If a person on a bus is streaming video but enters a tunnel and loses her connection, for example, video that she never sees will already have been counted toward her plan.

Using a custom app made to demonstrate the flaw they had uncovered, the UCLA researchers racked up a charge for 450 megabytes of data they never received. "We wanted to explore how bad it could be, and stopped after that," says Peng. "There's apparently no limit."
Researcher Chunyi Peng says it would be relatively simple for carriers to fix the problem through phone software that delivers some feedback to central data loggers. Unfortunately with carriers making potentially millions in extra charges with the way systems work, and regulators who have shown us time and time again they do not care that consumers are over-billed for usage, there's no incentive to fix the issue.

view:
topics flat nest 

skeechan
Ai Otsukaholic
Premium
join:2012-01-26
AA169|170
kudos:2
Reviews:
·Clear Wireless
·Cox HSI
·Verizon FiOS

Fines...big big big BIG fines

Work on 'somehow' auditing (such as dummy accounts actually owned by auditors) but then make the fine 10000X the amount of the over-bill. So if you screw up and over charge a sub by $10...it's a $100K fine, 10% to the customer if the customer's account could be audited, 90% to the regulator to pay for the audits.

There comes a point where ISPs will police themselves or give up on the greed of overages in the first place.
old_wiz_60

join:2005-06-03
Bedford, MA

Re: Fines...big big big BIG fines

Won't happen - the ISPs OWN enough people in the FCC/Congress so they will get away with incorrect billing. I'm only surprised they don't tack on an extra 100MB here and there anyway.

skeechan
Ai Otsukaholic
Premium
join:2012-01-26
AA169|170
kudos:2

Re: Fines...big big big BIG fines

I can dream can't I?

cork1958
Cork
Premium
join:2000-02-26
said by skeechan:

Work on 'somehow' auditing (such as dummy accounts actually owned by auditors) but then make the fine 10000X the amount of the over-bill. So if you screw up and over charge a sub by $10...it's a $100K fine, 10% to the customer if the customer's account could be audited, 90% to the regulator to pay for the audits.

There comes a point where ISPs will police themselves or give up on the greed of overages in the first place.

Should do EXACTLY that with just about every big company in existence including our stupid government!!
--
The Firefox alternative.
»www.mozilla.org/projects/seamonkey/

skeechan
Ai Otsukaholic
Premium
join:2012-01-26
AA169|170
kudos:2
Reviews:
·Clear Wireless
·Cox HSI
·Verizon FiOS

Re: Fines...big big big BIG fines

Except the government would just raise taxes to pay the fines. For the government, the employees and politicians should go to prison if they violate the law...no wrist slaps or simple reimbursements for crap like those parties and junkets on the taxpayer dime.
zefie

join:2007-07-18
Hudson, NY

Fringe coverage

Explains how my usage counter would go up when I was on 2G and content never loaded and timed out.

88615298
Premium
join:2004-07-28
West Tenness

Devil's advocate

Why do we assume the data counting app is the one counting correctly?

vpoko
Premium
join:2003-07-03
Boston, MA

Re: Devil's advocate

They're both counting "correctly", you'd have to be quite the incompetent developer to write an app that makes up bits out of whole cloth. The issue is where they're counting - counting at the source can't account for packet loss.
rahvin112

join:2002-05-24
Sandy, UT

Re: Devil's advocate

If they are charging for bits that never make it to the phone they are improperly charging.
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Devil's advocate

Packet loss is a fact of life in wireless networks. Most Layer 2, and many Layer 3 protocols, have no way to detect it. The billing counters will never be completely accurate, your phone will overestimate usage when upstream packets are lost, while the network will overestimate it when downstream packets are lost.


vpoko
Premium
join:2003-07-03
Boston, MA

Re: Devil's advocate

And what percentage of your packets are downstream vs upstream?
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Devil's advocate

I'm not certain why that's relevant.

What you have here is a technical reality, it is not a deliberate attempt to screw people. There is no way to determine if the phone received a downstream packet or not. A few people have suggested installing software on the phone itself to track this, but there are two problems with that:

1) Software on the phone can be hacked by the end user to commit billing fraud.

2) Such software would impose increased overhead on the network and INCREASE the amount of data you use.
TheGhost
Premium
join:2003-01-03
Lake Forest, IL

Re: Devil's advocate

You have to admit that if this issue was going against the carrier, they WOULD find an acceptable fix.
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Devil's advocate

It goes against them on the reverse link (from phone to base station), lost packets here will not be accounted for.

vpoko
Premium
join:2003-07-03
Boston, MA
It's simple: if you can't accurately meter - and it doesn't matter why - then you can't use usage to determine charges. If the technology doesn't exist to implement your business model, move on to a different business model.
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Devil's advocate

They are accurately metering. There is no promise in any packet based network, not even a wired one, that your packet won't be lost somewhere along the way. If it's lost between the base station and your phone then it's going to be accounted for by their billing system. That's just a fact of life. It's worth noting that the electric meter on the side of your house does not compensate for resistance losses that occur after the meter.

vpoko
Premium
join:2003-07-03
Boston, MA

1 recommendation

Re: Devil's advocate

The electric meter is at the demarcation point where the power grid ends and your premises begins, so it makes sense that losses inside would be the consumer's problem. To say that the base station is analogous to the demarc would be silly.
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Devil's advocate

The resource bottleneck (forward link timeslots) exists between base station and phone. It's the last place where it's feasible to count bytes. You're using airtime, which is how cell phones have been billed for decades.

Packet loss on a cellular network adds very little to your data count, 0.01% to 1%, depending on the RF conditions you typically encounter. Even if I accepted your argument that only successfully delivered packets should be billed, the meter is already accurate enough to pass regulatory muster.

Anyway, if it bothers you so much, you've got two national (and a few) regional) alternatives. Vote with your wallet.
Kearnstd
Space Elf
Premium
join:2002-01-22
Mullica Hill, NJ
kudos:1
There has to be some method of accounting for lost packets in TCP-IP. I mean I know most MMORPGs have a little network status thing that will tell you packet loss between you and their server, so they must be using something. Unless their own network code knows how many are sent and the server tells the client how many arrived.
--
[65 Arcanist]Filan(High Elf) Zone: Broadband Reports

bear73
Metnav... Fly The Unfriendly Skies
Premium
join:2001-06-09
Derry, NH

Re: Devil's advocate

Yes TCPIP has designed into it a method for telling the server that particular bits did not reach their destination. That is a core part of the stack. Without it the internet wouldn't work. As for how the provider is measuring, they're using the simplest method, which is to look at what they send to you including all the retransmits and calling that your requested data. Even if you have a dinosaur phone that can't do internet like mine but just attempts to start a browser app that times out and fails. I have had to fight verizon multiple times that I don't have a phone that does internet. I had to go so far as go to a verizon store and get the manager to look at my phone, verify it CANNOT do mobile web and then put a note on my account to the same.
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Devil's advocate

said by bear73:

Yes TCPIP has designed into it a method for telling the server that particular bits did not reach their destination. That is a core part of the stack. Without it the internet wouldn't work.

The endpoints of a TCP conversation know if a packet was received. The routers between those endpoints do not know, or care, all they do is pass along the packet (if the relevant connection isn't congested) or drop it (if it is and the queue is full). Even NAT devices don't account for lost packets, they only track the state of the connection, it's still up to the endpoints to verify that the packet wasn't lost.

That's TCP, most other Layer 4 protocols (UDP, ICMP, etc) have no mechanism to deal with lost packets, they leave it up to the application using them for transport.
tmc8080

join:2004-04-24
Brooklyn, NY
Reviews:
·ooma
·Optimum Online
·Verizon FiOS

usage billing = scam

You the people let these two carriers called AT&T and Verizon wireless collude to become a duopoly and strongarm the industry into usage billing because (ALLEGEDLY) there's not enough spectrum do offer a fair distribution of unlimited data plans and claim the highest speeds which is total horse crap (scarcity of spectrum bandwidth). Add on top of the scam in usage based bililng overages and false metering to fraudulently bill overages leaves you with a duopoly that's worse than a monopoly.

The moral compass of earning a profit in the USA's been skewed so badly against the consumer (by some copmanies) it's as if between 2000 (2005 for telco) and beyond you've woken up in a different country. I know I'm not the only one who feels this way.

88615298
Premium
join:2004-07-28
West Tenness

Re: usage billing = scam

said by tmc8080:

You the people let these two carriers called AT&T and Verizon wireless collude to become a duopoly and strongarm the industry into usage billing because (ALLEGEDLY) there's not enough spectrum do offer a fair distribution of unlimited data plans and claim the highest speeds which is total horse crap (scarcity of spectrum bandwidth).

If you honestly think that Verizon and at&t can offer unlimited data to all their customers on 4G I have some gold mines in Florida to sell you. If you want to see 4G crawl to 1X speeds then offer unlimited but then don't bitch about slow speeds.

Now can they offer more than 2 GB? Sure the fact they offer a 20 GB tier is proof of that. So yes a 2 GB cap for $30 is BS but so is thinking they can or should offer unlimited for $30 a month. My cable ISP doesn't even do that. befoe they got rid of it a couple of months ago Charter charged $25 for 3 Mbps and 100 GB cap. And that's cable. Somehow you think wireless can do better than cable.
InvalidError

join:2008-02-03
kudos:5

1 recommendation

Simple fix?

Putting software on phones/devices to report successfully received data to correct the delta between sent data as seen by the carrier's billing system and confirmed-received data by the subscriber device would be a bad idea: most devices can be jailbroken and once broken, the subscriber could very well install non-official apps to fake usage reports or otherwise cause the carrier's app to not count usage anymore.

Subscriber-controlled devices cannot be trusted with tracking/reporting their own usage for billing purpose.
Gardener
Premium
join:2006-10-19
Burnaby, BC
Reviews:
·TELUS

Re: Simple fix?

said by InvalidError:

Subscriber-controlled devices cannot be trusted with tracking/reporting their own usage for billing purpose.

Carrier-controlled devices cannot be trusted with tracking/reporting customers' usage for billing purposes.

Perhaps a collaborative effort?

88615298
Premium
join:2004-07-28
West Tenness

Re: Simple fix?

said by Gardener:

Carrier-controlled devices cannot be trusted with tracking/reporting customers' usage for billing purposes.

If you actually belvie that why would you stay with that carrier?

vpoko
Premium
join:2003-07-03
Boston, MA

Re: Simple fix?

said by 88615298:

If you actually belvie that why would you stay with that carrier?

Because you have like, umm, 4 choices and none of them are trustworthy. This is an inherent problem with usage-based billing. If we just took the electric company's word for how much electricity we used, without having meters, we'd be in the same boat there.

88615298
Premium
join:2004-07-28
West Tenness

Re: Simple fix?

Except
said by vpoko:

said by 88615298:

If you actually belvie that why would you stay with that carrier?

Because you have like, umm, 4 choices and none of them are trustworthy. This is an inherent problem with usage-based billing. If we just took the electric company's word for how much electricity we used, without having meters, we'd be in the same boat there.

said by vpoko:

said by 88615298:

If you actually belvie that why would you stay with that carrier?

Because you have like, umm, 4 choices and none of them are trustworthy. This is an inherent problem with usage-based billing. If we just took the electric company's word for how much electricity we used, without having meters, we'd be in the same boat there.

except that mobile companies do provide meters. at least verizon does.

vpoko
Premium
join:2003-07-03
Boston, MA

Re: Simple fix?

The meters are owned by the carrier and we have to take them at their word that they're accurate. Electric meters are certified by an independent authority.
Crookshanks

join:2008-02-04
Binghamton, NY

Re: Simple fix?

Hate to break it to you but you've always had to take the phone company at its word. Ma Bell didn't have an independent agency verifying that her switches were probably timing your long distance calls. No cell phone carrier ever had such an authority accessing their switches to verify the length of voice calls and number of texts.

The people here accusing the carriers of cheating are essentially accusing Fortune 500 companies of committing fraud on a massive scale. These are outfits that employ legions of lawyers and compliance officers, alongside thousands of IT people, and others, all of whom have access to the data accounting and billing system. Have you considered the fact that not one of these people have come forward as a whistle-blower? Could it just be that there isn't deliberate fraud going on here?

I track my data usage with third party apps and it lines up to what Verizon tracks. They don't perfectly agree but the difference between the two is less than 2%. That's more accurate than my speedometer is required to be, so what exactly is the problem here? The difference is easily explained by packet loss and times when the app wasn't running for whatever reason (updates, waiting for the phone to boot, etc.)

bear73
Metnav... Fly The Unfriendly Skies
Premium
join:2001-06-09
Derry, NH
but the mobile meter is recording what is sent from the server not what is used by the device.
rahvin112

join:2002-05-24
Sandy, UT
We do take the power companies word for it. The difference is that there is an independently certified meter (a requirement of regulation) involved so that you could if you so wished check the power companies math.

See back in the days when the power companies came into being no one trusted companies not to put their finger on the scale to up the bill so regulation was put in place requiring that if there is a meter involved that the meter be certified to measure accurately. There is a political effort afoot to try to convince people that the company can be trusted (wink) and that the regulation should go away. Just like the banking collapse a few years ago we'll find that human nature is for the company to put their finger on the scale just like they did in the past to deserve the regulation in the first place. And much like the mortgage crisis greed will cause severe harm to innocent people.

If the providers want to have billing that relies on metering they should be required by law to have independently certified meters.
desarollo

join:2011-10-01
Monroe, MI
Why is it a bad idea? The customer has a vested interest in knowing if the amount being billed is accurate. The carriers don't and will never use that information for the reasons you cite, but that doesn't mean it isn't an important method for verifying billing accuracy. I doubt the device is *that* inaccurate when it comes to measuring the data.

The carriers insisted upon usage based billing, if they're unable to accurately bill, the it is time for investigations into fraudulent billing.
InvalidError

join:2008-02-03
kudos:5

Re: Simple fix?

said by desarollo:

The carriers insisted upon usage based billing, if they're unable to accurately bill, the it is time for investigations into fraudulent billing.

If you read the story, billing is accurate as far as the accounting systems and tower traffic are concerned.

Where the "problem" lies is with the data that gets lost in-transit from the tower to the phone. The network operator has no way to know about it without some form of feedback from the device but devices cannot be trusted to give that feedback since jailbroken devices can be made to return bogus information.

JohnInSJ
Premium
join:2003-09-22
San Jose, CA

2 recommendations

um...

if the data makes it to and out the transmit tower, you used the resources. You pay for that. Collecting the client side of the data usage and being shocked that it differs from the server side because packets got lost in the air? Wow - you gotta wonder if any of these people were network engineers.
--
My place : »www.schettino.us

joako
Premium
join:2000-09-07
/dev/null
kudos:6

Re: um...

So if someone calls me, that means I used network resources and should be billed... even if T-Mobile's spotty network causes me not to receive the call?
--
PRescott7-2097

Selenia
I love Debian
Premium
join:2006-09-22
Fort Smith, AR
kudos:2

I trust the meters

It is a very simple task to count packets. Most of the "inaccurate" usage statistics for both wireline and wireless isps can easily be written off by protocol overhead(counter on main network interface mitigates this minor discrepancy) and packet loss. In a perfect world, this should be less than 1%. However, I have seen very lossy connections, sometimes wired business connections, in the order of upwards of 30%. Bad wiring or oversaturated networks can do this to wireline. RF problems and oversaturation can do it to wireless. These caps are lucrative and the telcos won't risk losing this revenue stream in a future court case about usage accuracy. They will just hike their rates and introduce shared plans. Wait, they already did that! Well, they could just raise overage rates and say it's for the children so they don't have enough of a cap to download porn and they would be afraid of paying for going over for the rest of their lives! Johnny gets sued for $2000 phone bill from downloading Nymphos from Venus! That what deregulation does folks. Why would risking being sued for inaccurate meters be worth it?
--
A fool thinks they know everything.

A wise person knows enough to know they couldn't possibly know everything.

There are zealots for every OS, like every religion. They do not represent the majority of users for either.
ShellMMG

join:2009-04-16
Grass Lake, MI
Reviews:
·Frontier Communi..

Reminder

Go back several years to a story that got congress to actually *act* on "information abuse" by advertisers.

It seems that many clever businesses decided to send unapproved advertisements to fax machines. It cost the owners of said fax machines the use of their phone line, paper, toner and electricity to print the ad. Those costs actually resulted in a bill banning that nonsense.

It's 2012, and now we have internet data plans that are seeing faster and faster speeds, higher costs and shrinking monthly caps. You are not only paying for the ability to download the information you actually want - you're paying for somebody's advertisements as well! All those Flash ads, commericals, banners, pop-ups and pop-unders...YOU are paying for them. They add up over time and could mean the difference in paying $15 in overages for another gig (VZW). Trying to stop the ads is like playing a never-ending game of Whack-A-Mole.

Spoken by someone who suffered with satellite for two years and the 7.5GB cap. NEVER never again.
Kearnstd
Space Elf
Premium
join:2002-01-22
Mullica Hill, NJ
kudos:1

Re: Reminder

Actually this is why I am partially against bill by the byte on home broadband connections. Websites today love rich media ads, But if I have to pay for the data transfer than I am paying for the advertising.
--
[65 Arcanist]Filan(High Elf) Zone: Broadband Reports
WhatNow
Premium
join:2009-05-06
Charlotte, NC
Reviews:
·Time Warner Cable

Re: Reminder

Use Adblock Plus on Firefox and you will not see most of the ads. When I have to use IE on occasion I see what I am missing.

I know any software can be hacked but if the counter was part of the operating system or built into the hardware it should be easy to detect if it is hacked.

KrK
Heavy Artillery For The Little Guy
Premium
join:2000-01-17
Tulsa, OK

They'll NEVER make any of the logging on the phone side.

The reason is simple: A few people could hack their phones and therefore "cheat" them out of their overages. So they'll use this red herring and instead overcharge millions of users in order to avoid losing profit to a few hundred.

... and they'll get away with it, too.
--
"Fascism should more properly be called corporatism because it is the merger of state and corporate power." -- Benito Mussolini

KrK
Heavy Artillery For The Little Guy
Premium
join:2000-01-17
Tulsa, OK

The solution is even more simple.

Pick a number. Say 10%.

Then whatever the data usage is for the customer's account the tracking software automatically just removes 10%.

Call it done.
--
"Fascism should more properly be called corporatism because it is the merger of state and corporate power." -- Benito Mussolini
Crookshanks

join:2008-02-04
Binghamton, NY

the study gets at least one thing wrong....

The problem affects video and audio streaming apps in particular because they use protocols that don't require the receiving device to acknowledge the receipt of every chunk of data, or halt data transmission immediately, as Web browsers or many other apps do. That means a video app will keep sending data for some time, oblivious to the fact that a device can't receive it.

Factually incorrect. The majority of video and audio streaming occurs over TCP connections. TCP contains mechanisms both to detect lost packets and slow down transmission when packet loss occurs.

Many forms of two way communication (VoIP) occur over UDP, which does not contain such mechanisms, simply because a lost packet here or there is better than a complete pause in communications while the lost packet(s) are retransmitted. VoIP's data usage doesn't really amount to much though, and that's not what they were studying anyway.