Spice300
Premium
join:2006-01-10
| FAP Flap Follow-up
I have collected about a month and a half of data using Grover92000's Fapzilla program to read my WB usage meter and my own program that reads my actual received and sent data from "netstat -e." I have calculated my 30 day rolling download usage according to netstat (green curve) and plotted it with my usage meter (red curve).
My system: beam 31, Riverside gateway, using WB proxy, NRTC = gotsky
On Feb. 15 Wildblue under reports my actual usage by about 600 MB because the meter flat-lined on January 30 and 31 failing to record about 700 MB of actual download  .
For most of the graph WB adds usage within 1 to 3 hours of the actual download.
WB subtracts data about 30.8 days after it was actually added as can be seen most easily by the big dip on the 21st in the green curve and on the 22nd in the red curve. On my admintool page WB states that the FAP can take up to 24 hours to update, so they subtract usage within their obscurely stated range of 30 to 31 days.
Until the 26th WB is adding and subtracting my usage fairly accurately. Sometime on Feb. 26 my WB FAP meter stopped updating within a few hours and began updating every 12.5 hours. Others have reported this twice daily update of their FAP. I seem to remember there was scheduled maintenance on the morning of Feb. 26 for the Riverside gateway, but beldin's entry, »www.wildblue.cc/wbforums/attachm···70244620 , does not indicate it after being edited. Perhaps this twice daily updating is a characteristic of WB's new method of measuring the FAP. After two days of this new method of updating, the 600 MB gap closes to zero and appears to be overstating my usage or not accurately subtracting the data from 30 days ago.
I may have caught WB in the act of switching to their new, "more accurate" form of monitoring. I need to watch my meter for a few more weeks to be certain. If anything WB's FAP meter has been inconsistent. |
|
Spice300
Premium
join:2006-01-10
1 edit | I have collected just over two months of data using Fapzilla allowing me to calculate my actual rolling 30 day usage for an entire month. The red curve is my WB usage meter as reported by Fapzilla and the green curve is my actual rolling 30 day usage as reported by "netstat -e." For comparison with the rolling curves I have included the blue curve showing when I downloaded data according to netstat. No data has been subtracted from the blue curve.
WB Usage Compared to Actual
In February my WB usage meter subtracted data with a 30.8 day delay. In March it is subtracting data with about a 27.8 day delay. The data added on Feb. 8 (blue curve) was subtracted just before Mar. 8 (red curve). That sequence continues until Mar. 17. A more appropriate name for the rolling 30 day FAP may be a rolling monthly FAP.
This varying interval for subtraction may account for many complaints about WB not subtracting data. People are expecting the data from 30 days ago to be subtracted when in reality WB is subtracting the data from the same calendar day of last month. This suggests that WB's FAP meter will behave oddly at the end of the month. On March 29, 30 and 31, how can it subtract data from February when there is no 29th, 30th and 31st?
The data from Feb. 27 to Mar. 6 is a jumbled mess that is hard to interpret. WB was not subtracting much data and eventually overstated my usage by about 700 MB. This is also about the same amount, 720 MB, that they did not record during that flat line period at the end of January. By Mar. 15 that gap was reduced to about 500 MB.
Except for Feb. 27 through Mar. 2 when my WB FAP meter was updating every 12.5 hours, WB continues to add usage withing a few hours of the actual download. |
|
lhookins
join:2007-01-05
Bonham, TX
| said by Spice300  :People are expecting the data from 30 days ago to be subtracted when in reality WB is subtracting the data from the same calendar day of last month. This suggests that WB's FAP meter will behave oddly at the end of the month. On March 29, 30 and 31, how can it subtract data from February when there is no 29th, 30th and 31st? If that's what they're doing, that's truly frightening. The only possible excuse for writing code like that is because someone didn't know how to determine a date 30 days prior. In the end, it's much more complicated because the months aren't the same length.
I've only met or heard tell of a few programmers with horsepower so low they'd try to write a function like that. It almost suggests it was written by a manager... and not a programming manager either.
I sincerely hope I'm wrong on this.
Hook |
|
alhanson
join:2006-01-29
Barneveld, WI
| I don't think so. Most like the head guy, because he has to keep his scrips secret. In his memo he call them "customer care scripts" - the scripts that write stuff to your computer registry that direct you to the DAMA. You have to reformat the Hard drive to get rid of them or just unplug the computer from the net work and use another one. with open DNS on the router and proxy setting all disabled. |
|
magicjimmy
join:2006-03-23
Tucson, AZ
| said by alhanson : In his memo he call them "customer care scripts" - the scripts that write stuff to your computer registry that direct you to the DAMA. This is incorrect. He was referring to the "scripts" on the cust service reps' computers. The ones that tell them what to ask and what to say. |
|
Spice300
Premium
join:2006-01-10
| reply to lhookins
said by lhookins :I sincerely hope I'm wrong on this. Wildblue's FAP meter is so erratic and unpredictable that maybe customer service rewrites the code on a monthly basis  For the last 3 months, my WB FAP meter has malfunctioned at either the end or beginning of the month: flat-lined at the end of December and January and was not subtracting data from mine nor updating hourly at the beginning of March. Fictional usage is added and subtracted occasionally. The FAP is a mess. What will next month bring??????? |
|
lhookins
join:2007-01-05
Bonham, TX
| said by Spice300 :Wildblue's FAP meter is so erratic and unpredictable that maybe customer service rewrites the code on a monthly basis That may not be far off the mark. If they're simply working off the same day from the last month, *someone's* going to have to go in there and manually massage the data.
Heck, if WildBlue got in touch with me, I'd write the function to return a date 30 days prior for them for free. All I need to know is the date format they want and the language they want it written in. Even if they're using some proprietary language (RBASE 3000 for example) I can probably work it out. The advantage of 30 years programming experience.
Hook |
|
seriouslywtf
@sbcglobal.net
| quote:
I don't think so. Most like the head guy, because he has to keep his scrips secret. In his memo he call them "customer care scripts" - the scripts that write stuff to your computer registry that direct you to the DAMA. You have to reformat the Hard drive to get rid of them or just unplug the computer from the net work and use another one. with open DNS on the router and proxy setting all disabled.
I can't even begin to count how many times I've heard this, or something like it.
Your computer is not directed to "the DAMA". The traffic-shaping and scheduling takes place at the gateway. Proxy or not, openDNS or not, *ALL* traffic passes through the gateways.
This kind of unfounded conspiracy theory that WB is manipulating the stack in nefarious ways is ridiculous. |
|
YellowHammer
join:2004-12-09
Jackson, AL
 ·WildBlue
| said by seriouslywtf :
The traffic-shaping and scheduling takes place at the gateway. Proxy or not, openDNS or not, *ALL* traffic passes through the gateways.
This kind of unfounded conspiracy theory that WB is manipulating the stack in nefarious ways is ridiculous.
Yep. But I agree that the FAP coding is absolutely incompetent at best. |
|
Spice300
Premium
join:2006-01-10
1 edit | reply to Spice300
I modified the data in my spreadsheet allowing me to adjust FAPPeriod, the interval of time when FAP data is subtracted. I had to modify some of the data to do it. For example, I changed entries with irregular time intervals like this:
Date (MST), Time, WB up,WB down,received,sent
2/1/2007,12:00:00,168,5901,9713587,800195
2/1/2007,12:06,,,10143849,814243
2/1/2007,12:30:00,168,5901,1735386,54916
into:
2/1/2007,12:00:00,168,5901,10143849,814243
2/1/2007,12:30:00,168,5901,1735386,54916
These alterations change the time at which data is added and subtracted slightly but do not affect the overall total. The precision is no better than 30 minutes anyhow.
Wildblue Download Usage Compared to Actual
For the green curve from Feb. 16 through Feb. 28, FAPPeriod is 30.79 days (exactly 1478, 30 minute intervals). On Mar. 1 at 00:00:00 GMT I changed FAPPeriod to 27.79 days (exactly 1334, 30 minute intervals). There is no adjustment for the delay in adding data which is usually a few hours later but sometimes is 12.5 hours or longer. The red and green curves now have similar shapes although there is a several hundred megabyte offset between them. In February Wildblue's usage meter under reported my usage due to the flatline period at the end of January. In March WB is over reporting my usage by about 800 MB or 11% of my Value Pack allocation.
When changing FAPPeriod to 27.79 days on Mar. 1, Wildblue skipped subtracting 3 days of usage mostly on Jan. 29 to 31 amounting to a whopping 843.1 MB. About 720 MB of this data was added during the flatline period. The difference grew during the first week of March. The big increase on Mar. 4 was about 480 MB with WB reporting 600 MB, 120 MB too much. A little bit of data was subtracted in the green curve in the middle of Mar. 4 but nothing was subtracted in the red curve. On Mar. 6 and 7 WB failed to subtract about 100 MB. These little errors continued to accumulate until Mar. 8 when the overstating amounted to 840 MB. After that the gap fluctuated around that amount.
On Mar. 18 WB's FAP meter took about half a day to finish updating the downloaded data. This variability in the delay for updating can be confusing to someone watching his meter. One might think that some data was subtracted canceling out the addition allowing for more data.
To Wildblue's credit when they overstate usage, they generally subtract off the overstated amount as shown by the data added on Feb. 12 & 13 being subtracted on Mar. 12.
Downloading and watching streaming video tend to be reported accurately, but browsing is over reported.
Edit: fixed error: overstatement in March is about 800 MB, not 1000 MB. |
|
randyvsatus
Premium
join:2005-03-03
Monument, CO
 ·Qwest.net
| reply to seriouslywtf
said by seriouslywtf :
... The traffic-shaping and scheduling takes place at the gateway. Proxy or not, openDNS or not, *ALL* traffic passes through the gateways.
This kind of unfounded conspiracy theory that WB is manipulating the stack in nefarious ways is ridiculous.
Nice to hear from someone who has a clue as to what is really going on.......thanks!
--
1.2M Dish |4 watt|IA 8||Qwest DSL|7168 / 896 Kbps |
|
DrStrangeLov_
| reply to lhookins
said by lhookins :Heck, if WildBlue got in touch with me, I'd write the function to return a date 30 days prior for them for free. Start right here:
# San Diego, CA
# Corporate Headquarters
# Main Campus
# 6155 El Camino Real
# Carlsbad, CA 92009-1699
# Tel 760.476.2200
# Latitude: N 33° 07.614
# Longitude: W -117° 16.008
»maps.yahoo.com/maps_result?ed=ao···me=&qty= |
|
Spice300
Premium
join:2006-01-10
1 edit | reply to Spice300
As usual the red curve is my download FAP meter whose data was read using Fapzilla. The green curve is my download usage as reported by "netstat -e" with data subtracted from 27.81 days ago (FAPPeriod). I have not adjusted for the delay in adding data. Notice that my WB FAP meter has not been subtracting data since March 29 12:30 am GMT. For most of the period WB over reported my usage by a little over 700 MB suspiciously close to the amount of data that I downloaded during the flatline period at the end of January. Because WB has not been subtracting data for the last 2 days the difference has increased to over 1100 MB. Maybe Wildblue changed FAPPeriod again.
WB Download Usage Compared to Actual
|
|
Spice300
Premium
join:2006-01-10
2 edits | reply to Spice300
The red curve is WB's download FAP meter as it transitions from March to April. On March 29 00:31 GMT WB stopped subtracting but continued adding usage. On March 30 07:31 GMT through March 31 it flat lined at 6088 MB. It next changed at 08:31 GMT on April 1 when it added a fictional 472 MB spike. The next reading taken at 09:01 GMT subtracted most of the fictional usage to 6089 MB leaving a 1 MB increase.
To make it align I modified my actual usage (green curve) as follows:
Through the entire period the delay for adding data is 2 hours.
I had to alter the interval at which data is subtracted (FAPPeriod) several times:
From March 27 00:00 GMT to March 29 00:31 GMT, FAPPeriod = 27.81 days.
Starting March 29 01:01 GMT usage is added but not subtracted. This is equivalent to FAPPeriod increasing.
Starting April 1 08:31 GMT FAPPeriod is 30.81 days, the same as it was in February.
WB Download Usage Compared to Actual
Choosing a delay of 2 hours makes the best overall fit causing little features to appear, but there are several discrepancies. In WB's FAP the increase on March 30.3 occurs 1 hour before the actual increase meaning delay should be 1 hour at that spot in the graph. The increase at April 2.9 occurs 30 minutes early meaning delay should be 1.5 hours there.
Before March 29 WB over reportes my usage by approximately 750 MB. On March 29 the gap decreases to a little over 600 MB because WB under reported the increase in usage on March 29. This gap continues to decrease through the flat line period because the green curve shows the actual usage added. When data begins being subtracted again on April 1, there is a discontinuous drop in my actual usage while WB's shoots up with a fictional spike. By April 3 the gap decreases to about 550 MB as some over stated usage from the first week of March is removed. I expect this trend to continue for the next week.
The fictional spike is troublesome. If my usage had been comfortably below the maximum at 7028 MB, then the fictional spike would have exceed the limit and it might have FAPed me. It is the largest fictional usage that I have detected on my meter so far, and I would not have detected it if I had been sampling data at hourly intervals.
Even though the FAP meter was updating at 12.5 hours intervals on March 1 & 2, the usage is subtracted on April 1 & 2 as though it was recorded hourly supporting my idea that the WB portal read the WB FAP counter at 12.5 hour intervals. The FAP counter was still updating as usual. |
|
Spice300
Premium
join:2006-01-10
| reply to Spice300
Here is my download usage compared to actual for the last 44 days. To make my actual usage, green curve, align I modified it as follows:
From March 11 to March 26 23:31 GMT, delay = 0 hr
From March 27 00:00 GMT to April 23 00:00 GMT, delay = 2 hr
From March 11 to March 29 00:31 GMT, FAPPeriod = 27.81 days
From March 29 01:01 GMT to April 1 08:01 GMT, usage is added but not subtracted.
From April 1 08:31 GMT to end of the graph, FAPPeriod is 30.81 days.
WB Download Usage Compared to Actual
Through April WB usage meter has been over reporting my download usage by about 500 MB. During the flat line period at the end of March, I downloaded about 150 MB that is included in the green curve but not in the red curve. This means that WB is over reporting my usage that they actually record by about 650 MB. If TalonDancer at wildblue.cc is correct about the usage on March 29 being subtracted on April 1, then the gap is another 75 MB larger. I still can not account for this gap.
On March 16 WB's usage meter began updating at 12.5 hour intervals instead of the more typical hourly intervals. However, my NRTC usage meter continued to update whenever I checked it giving different reading from WB's portal which Fapzilla reads. Today WB's FAP meter began updating hourly again.
The delay of 0 hr for adding data in March is only used because I am too lazy to update the older data to the 2 hr delay. A delay of 1 to 2 hours produces a good fit through out the entire period. |
|
Spice300
Premium
join:2006-01-10
1 edit | reply to Spice300
April 27, 2007 23:31 GMT
WB FAP meter appears to stop adding data. However, I am not sure if the subtractions simply exceeded the additions during April 28. WB's FAP meter subtracted about 100 MB more than my actual curve, and I added 79 MB. Based on my data it is possible the meter stopped adding but continued subtracting usage for 2 days, April 28 and 29 GMT.
April 28, 2007 00:00 GMT
FAPPeriod = 30.81 days
delay = 2 hr
April 28, 2007 23:31 GMT
My actual usage increases but there is no corresponding increase in WB's FAP meter. WB's FAP meter is definitely subtracting, but not adding, usage. Usage continues to be added and subtracted from the green curve.
April 30, 2007 06:31 GMT
WB FAP meter flatlines. No data is subtracted from the green curve but usage continues to be added with a delay = 2 hr.
May 1, 2007 04:01 GMT
WB FAP meter decreases by 14 MB in download ending the flatline.
FAPPeriod = 29.79 days
delay = 2 hr
The drop of about 100 MB in the green curve at this time is a jump discontinuity caused by changing FAPPeriod.
May 1 and half of May 2
WB FAP meter updates at irregular intervals ranging from 4 to 12.5 hours.
May 2, 2007 14:01 GMT
WB FAP meter begins updating at hourly intervals.
May 4, 2007 GMT
I finally have enough data to determine FAPPeriod for May as 29.79 days plus-minus .021 days.
WB Download Usage Compared to Actual
Between April 29 00:00 GMT and May 1 04:01 GMT I downloaded 1577 MB that WB does not show on their meter. The value could be 79 MB larger if the period of no subtraction began sometime on April 28. Consequently the WB FAP meter now underreports my download usage by about 1300 MB. Hopefully the discrepancy is large enough to determine of WB adds it later.
--
Value Pack, beam 31, Riverside gateway |
|
Spice300
Premium
join:2006-01-10
2 edits | reply to Spice300
WB Download Usage Compared to Actual
May 29, 2007, 00:00:00 GMT
One day prior to the flat line at the end of May.
FAPPeriod = 29.79 days
delay = 2 hours
My FAP meter is updating at approximately 12.5 hour intervals. Sometime between 20:01 on May 28 GMT and 08:01 on May 29 GMT, my FAP meter stops subtracting but continues adding usage.
I begin manually sampling download usage reported by my NRTC FAP meter which is plotted in the blue curve.
May 30, 2007
Sometime between 08:31 GMT and 13:01 GMT the meter flat lines based on the NRTC data. However, the WB FAP meter updates to the NRTC value at 21:01 GMT. I continue adding and subtracting data from my actual usage (green curve). I begin download data, mostly streaming video, non-stop for 36 hours.
May 31, 2007 09:31 GMT
I stop subtracting data from my actual usage which is the same as increasing FAPPeriod by the real elapsed time but I continue to add usage. I continue downloading data keeping a close watch on the Fapzilla data and my NRTC usage meter.
June 1, 2007
Sometime between 04:01 GMT and 10:01 GMT the flat line ends. None of the nearly 5,000 MB of data that I downloaded during the flat line appears on my usage meters even though my actual usage exceeds my Value Pack limit of 7,500 MB.
At 10:01:07 GMT I set the FAP variables to:
FAPPeriod = 30.79 days
delay = 1 hour
June 2, 2007, 10:31 GMT
The Fapzilla data stops updating at approximately 12.5 hours intervals and begins updating hourly. The data that is added is erratic, not corresponding to any actual usage. Small amounts ranging from 1 to 10 MB are added with little correspondence to actual usage. I can not determine the pattern. Delay appears variable.
June 4, 2007, 00:01 GMT
delay = 6 hours
The wild ride of the FAP continues. The erratic update interval that continued for several days makes it difficult to monitor usage closely. Until I plotted the data I thought usage was subtracted canceling out what was added. For us bandwidth hogs there is good news: Wildblue's usage meter does not record your usage during flat lines even if your actual usage exceeds your FAP limit, at least until they change the FAP algorithm. I am still not FAPed  
--
Value Pack, beam 31, Riverside gateway |
|
Spice300
Premium
join:2006-01-10
| reply to Spice300
WB FAP Meter Compared to Actual Usage
Green curve = rolling monthly download usage derived from netstat (actual usage)
Red curve = Wildblue FAP meter as read by Fapzilla
FAPPeriod = 29.79 days
delay = 30 minutes
Adjustment: the usage during the flat line at the end of June is removed from the green curve.
From June 1 to June 6 my WB FAP meter updated about twice per day.e
From June 6 to June 22 my WB FAP meter updated more rapidly than once every 30 minutes which is the fastest rate observed to date. This short delay began after my WB FAP meter was restored after the reset on June 26.
Linear Error
This is a plot of the data between July 6 and July 23 showing the linear relationship of the error between my actual usage and my WB FAP meter. Covering a range of more than 25% of my FAP threshold the error retains a linear relationship rather accurately. The pairs of approximately vertical dots extending from 5700 to 6500 on the x-axis correspond to the big decrease around July 14. The variations are caused by my sampling interval of 30 minutes being different than the actual update interval of my WB FAP meter and provide a measure of the accuracy of the data. The vertical dots at x-ordinate 5350 are caused by my WB FAP meter returning to twice daily updates starting July 23 GMT. I have omitted the data from July 1 to July 6 to reduce the error that twice daily updates introduce.
The line's slope of .923 corresponds to WB's FAP meter overstating my usage uniformly by about 8.3%. This could be caused by the "satellite wrapper" (a phrase coined by News). I interpret the intercept of -125 as the error introduced by canceling page loads and understatements of my netstat meter due to WIN XP locking up & shutting off my computer. 125 MB is about right for those errors. The intercept varies considerably when I add more data to the graph.
Here is proof that Wildblue's FAP meter consistently overstates usage relative to netstat without proving why.
--
Value Pack, beam 31, Riverside gateway |
|
dbirdman
Premium,MVM
join:2003-07-07
Eureka, CA
| What do you think the odds are that WB uses "Millions of Bytes" when they quote MB? We are pretty sure that Hughes does that with their FAP, just as disk drive manufacturers usually do. That introduces a 5% overage, since an MB standard is 1048576 Bytes.
--
W2K Server|Toshiba Satellite XP Pro|HughesNet IA8/1410/7000 2-watt Business Internet on .98 meter fixed | Datastorm .98 XF2 2-watt on 1990 Blue Bird Wanderlodge "Blue Thunder" 22 tons of rolling steel! |
|
BasilAR
WildBlue
join:2006-07-20
Parks, AR | You mean 1MB of allowed FAP is 1,000,000 bytes and
1MB of usage is 1,048,576 bytes?
--
Beam 35 - value |
|
