dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
3551
share rss forum feed


Den
Premium
join:2001-01-21
Cape Coral, FL

4 edits

8 recommendations

[F@H] Let's talk about milestone interval revision

As it stands milestones are too numerous for the people with higher output and/or score value. Their achievements shouldn't be ignored, but a milestone everyday is silly.

The problem is the scoring is so out of whack that there are people doing 100-300 a day and some doing 10,000+ a day

So it's a rough decision on how to award milestones. On one hand you have people working very hard to achieve a milestone and others basically getting one everyday.

So somehow it has to compare a persons work output and award milestones accordingly.

The output level of members is no longer a narrow range and the old "how big is the number, divide it by ten" milestone trick no longer works. Some people will wait forever for a milestone once they reach a certain level and others will be getting them daily.

It needs to give milestones frequently as a person starts and then become less frequent as time goes on

Static first tiers:
up to 1,000 - 100 point milestones
up to 10,000 - 1,000 point milestones
up to 100,000 - 10,000 point milestones

But once you reach above the 100,000+ score tier, there will be people that will only get a 10,000 point milestone every 8-10 weeks... and others getting it everyday. That's where it goes wrong. Somehow the code needs to choose an appropriate step based on estimated daily output.

To be fair...It needs to give awards on work done by a member based on their output capability.

So in the code it needs to decide what step size to use once you pass the 10,000 and 100,000 milestones.

1) Between 10,000 and 100,000 points it's either a 5,000 or 10,000 point step

2) Over 100,000 points it's a 5,000 / 10,000 / 100,000 point step based on output.

Englishfied code :-)
 
Notes:
# starts a comment
>= means greater than or equal to
<= less than or equal to 
 
#######################################
 
### Checking output level to adjust Milestone step
 
# checking between 10,000 and 100,000 point level
If "current_score" > 10,000 and "current_score" < 100,000 { 
    
    # then check output
 
     if "member_daily_output" >= 10,000 {
         # adjust step
         then "milestone_step" = 100,000
     }
 
     if "member_daily_output" <= 1,000 {
         # adjust step 
         then "milestone_step"=5,000
     }
 
}
 
# checking above 100,000 level
 
If "current_score" > 100,000 { 
    
    # then check output
 
     if "member_daily_output" >= 10,000 {
         # adjust step
         then "milestone_step" = 100,000
     }
 
     if "member_daily_output" <= 10,000  {
         # adjust step 
         then "milestone_step"=10,000
     }
 
     if "member_daily_output" <= 500 {
         # adjust step 
         then "milestone_step"=5,000
     }
}
 
# Then use "milestone_step" in code to check for award
 
 

As for scoring, some of you know I folded for years and then stopped. Over that time I reached 127,866 points with 1905 workunits (based on old stats page)

Today I just passed 1,000,000 and have 2370 workunits done

So that means...
872,941 points for 465 workunits
vs
127,866 for 1905 workunits

The old school members worked hard for their points and the milestone levels worked back then. They don't anymore.

So...after all my rambling I say update the code to handle some insane point level and then add the code to check output and for variable steps and be done with it.

Den

edit -
-changed last output check to less than or equal to 500 a day
-moved notes into code field. html was breaking it


sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

1 recommendation

Revising the milestone intervals needs to be done before leibold See Profile can make changes to a copy of Fishie's script or create a new one.

Thanks for starting this discussion, Den.
--
Join Team Helix * I am praying for these friends .



signmeuptoo
Bless you Howie
Premium
join:2001-11-22
NanoParticle
kudos:5
reply to Den

IMHO, it does seem to be a reasonable issue to concern with. I've wondered what could be changed and if it should be, I can't offer an opinion on what to do, but I certainly don't object to what Den_ is saying, I tend to agree I think.



rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2

1 edit

1 recommendation

reply to Den

These are SANE recommendations!

One of my bitches personal concerns, was posting Milestone points that are, today, (with new CPUs and GPUs(new), the extraordinary point accumulations, with relatively few WUs done to achieve those points!!

I'm all about recognizing folks, that "make a milestone". It's important to recognise those who do!

There needs to be a more graduated/non-linear calculation for the individual Milestone.

This may prove to be a daunting request for the script writer.

EDIT: I know, the "point values" are beyond our control. They are concieved, and assesed by F@H, and properly so.

Allowing this, Milestones WE recognize, should be considered. (jus' my humble opinion).
--
Hey, Obama!
Socialism is GREAT - UNTIL you run out of other people's money!



OSIU
Where is my "change"?
Premium
join:2003-11-12
Nowhere
Reviews:
·Armstrong Zoom ..
·Frontier Communi..
reply to Den

I agree that a sliding scale should be used so that all contributions, both large and small, are duely acknowledged.

What if we took the estimated ppd for each team member and multiplied by 10 (50, 100, SQRT(16)*5, or some other factor) for their next milestone? This way everyone has a reachable milestone and won't take less than a day for some or months for others?

My $0.02

O



DataRiker
Premium
join:2002-05-19
00000

1 recommendation

reply to Den

Totally agree.

I have a single core i7 on loan so my contributions are on the lower end of the scale.



onDvine
Don't Litter. Spay-Neuter.
Premium
join:2005-01-29
So. CA, USA
kudos:9
Reviews:
·Verizon Online DSL
reply to Den

I'm among the lowest-producing members. Don't do it for the milestones but less time between them would be fun.
--
Be happy for this moment. This moment is your life. Omar Khayyam



Starfish
Per Ardua Ad Astra
ExMod 2002-04
join:2000-12-28
Netherlands
reply to Den

Let me make a quick reply:

I like what you've written Den, good ideas

The only thing that, from a technical perspective, makes me ponder here is the following:

You mention the "member_daily_output".

Right now the script just looks at and processes one of Stanford's statistics pages. If I'm not mistaken it's this page:

»fah-web.stanford.edu/teamstats/team4.html

For each day, the user + credit total values are stored locally (compressed in a separate file, thus not using a database) on the computer of the person who is posting the milestones.

When a new milestone compilation is made, the script looks at the difference between the values of the current day and those of another day, usually the day before.

It would seem to me that, with output varying per day, that this should be something like a moving average over X number of days to determine a user's average output speed compared to the more "instantaneous" speed it's currently looking at.

So using those daily snapshots of user+credit total, how does one determine the "member_daily_output"?

I guess the script would then have to move to a database-based storage instead of a file-based storage.



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

1 edit

I think the simplicity of the script for posting daily milestones was elegant, Fishie. It was/is working well and didn't/doesn't require any complexities for the person(s) who brings/bring us those milestones. This script has a different purpose than some of the other milestone charts out there which are similar to the stats Den once created for Helix.

Just my opinion, but I think the main issue is due to the fact PG has greatly increased points over the years and our milestone markers have consequently become ridiculously low for the high end. I only have four older crunchers now and they put me on the daily charts five times over by the highest milestone marker (5,000). That is due to increase in points and isn't due to any upgrades by me in equipment, not since 2007 and even then they weren't huge upgrades.

IMO, most who are near or at the top would rather the emphasis be on those who haven't folded as long or don't have the equipment to generate the credits so quickly.

I suspect, if the team could decide on better milestones for this place in time, that leibold See Profile could either modify your script or, if you prefer not to provide the source, create another to do what yours has been doing for us for years.

--
Join Team Helix * I am praying for these friends .



suprleg
Abracadabra
Premium
join:2004-05-06
Cypress, CA
kudos:4
Reviews:
·Time Warner Cable
reply to Den

Perhaps milestones based on wu's rather than points would be a more realistic translation of a members contributions? Without doing the research I wouldn't be able to say that a wu generated by a gpu vs. smp vs. single core is of more, less, or the same value from a research standpoint, so it's somewhat difficult to gauge based on a broad- brush of quantities of work-units alone. However, it's something to consider. /my 2 cents
--
Team Discovery
Team Helix



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

Milestones based on WUs would probably put emphasis on those of us who have been folding a long time, which is what a number of us are trying to change.

I think we primarily need to fairly dramatically change the higher intervals, so that newer folders and those who are folding with all their might, but just don't have the heavy point making equipment, show up more often and higher in the daily reports.

Some of us (parkut has told me this repeatedly) don't really care whether we are in those reports or not. I know I once watched the stats. Now I just do what I can and I really only look at my third party monitor to make sure the equipment I do have is folding or if there is a problem I need to check into.

Starting out, though, the encouragement of the daily stones was a good thing as I recall.
--
Join Team Helix * I am praying for these friends .



rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2

Well, does anyone think, perhaps the daily milestone report has out-lived it's usefulness?

I know how you & parkut See Profile think about your milestones being published. I know several folks, myself included, that feel exactly the same. Only peek at it to determine if all machines are healthy.

There are many sites, along with Stanford that publish hourly reports for every team, but don't recognize, or single out individual "milestones".
--
Hey, Obama!
Socialism is GREAT - UNTIL you run out of other people's money!



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

1 recommendation

I think the milestones are a good thing, Russ. I just think we should change the intervals, that's all, to make people at the top of the charts (usually because we've been folding so long) show up far more infrequently and make sure everyone else shows up frequently enough to be encouraged and receive the honor they should for keeping up the good work.

Judging from the unique views on the milestone posts, I think it is obvious there are interested people.
--
Join Team Helix * I am praying for these friends .



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

1 recommendation

reply to rusdi

My puny little farm tends to get 22,000 to 27,000 PPD and the top milestone interval, if I recall correctly, is 5,000. That used to be meaningful, but the bonus points for SMP WUs has changed that.

Make me work for it? Change that 5,000 top interval from 5,000 to 100,000 or 200,000 or 250,000 or?
--
Join Team Helix * I am praying for these friends .



rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2
reply to sortofageek

said by sortofageek:

Judging from the unique views on the milestone posts, I think it is obvious there are interested people.

AGREE!!!!!!!!!

I think it would be good for leibold See Profile to comment, about how long it would take, and difficulty involved writing a new script that includes the new milestone triggers, (still to be determined).
--
Hey, Obama!
Socialism is GREAT - UNTIL you run out of other people's money!


sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

I don't see how he can do that until when/if he sees the code, Russ. Even then I don't think I would like to ask him that question about a volunteer job. I think it oughta' be when/if he can help us out in his spare time.

My son is a software developer, gets paid handsomely for that ability. If he could make the time to help us for free, I don't think I would be asking him for estimates of completion, just might lose him.

leibold See Profile has offered to help us out, one way or the other. I'm good with that.
--
Join Team Helix * I am praying for these friends .



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21
reply to rusdi

I'm wondering, though ... how long will it take us to decide on new milestone intervals? That might take more time than changing the script.
--
Join Team Helix * I am praying for these friends .



rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2
reply to rusdi

I certainly don't mean to rush anyone, especially one who has volunteered.

I really just wanted to get some idea of if,what we are wanting, is too difficult, or practical for a mere "service" provided.

EDIT: Oops. Hit the wrong "reply" button.
--
Hey, Obama!
Socialism is GREAT - UNTIL you run out of other people's money!



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

I'm sure it isn't too difficult for leibold See Profile. I'm just saying, there isn't really anything he can say until these decisions are made:

1. He can work with Fishie's source code or not.
2. What do we want the new milestone intervals to be?
--
Join Team Helix * I am praying for these friends .



Den
Premium
join:2001-01-21
Cape Coral, FL

1 recommendation

working on reply to Fishie



Den
Premium
join:2001-01-21
Cape Coral, FL

2 edits

1 recommendation

reply to Starfish

Fishie,

You can keep the flat text file storage and expand the fields using a delimiter like : or , if your not already doing so.

Read in from file
member_name:day1:day2:day3:day4:day5:day6:day7:milestone_old

All you need to do then is just shift the data fields when you dump it back out to the file.

#add in code if the day = "" then it equals "0" to get it going

Then when writing out to file shift the fields over:

member_name:day2:day3:day4:day5:day6:day7:score_current:milestone_current

You've just removed the oldest and added the newest to the file.

==== Word of Warning!!

For 'coding/testing/multiple runs per day' surround your write function with a boolean on/off flag if you plan on running it over and over to test. That will protect your old data from being shifted every run and overwritten with zeros for every field. It will also allow it to be run multiple times a day without destroying the history

create a special user name and use it to store data such as epoch date ( not epoch time) of last run.

So data file would look like:

crypticname:epoch_of_last_run
member_name:day2:day3:day4:day5:day6:day7:score_current:milestone_current
member_name:day2:day3:day4:day5:day6:day7:score_current:milestone_current
...
...

In the file read section have it look for "crypticname" and set the variable and then skip the rest of the read in function

if date = date then {
testing=1
} else {
testing=0
}

Then around your write function check for testing=1 and bypass writing to file if set.

#uncomment next to force run
#testing=0

if testing=0 then {
#file writing function here
} else {
# print "Test mode: Multiple Daily run detected : Skipping file write"
}

You'll need write out to file at least once to build the new storage file format. Then set it to 1 to code and test

=== Calculating "member_daily_output"

On read in, day1 through day7 would be used for the average calculation.

Then take all the "day" fields, add them together, and divide by number of days for a rough average. All that's needed is a rough estimate.

"member_daily_output" = ((day1 + day2 + day3 + day4 + day5 + day6 + day7 ) / 7 )

The heavy output members aren't going to change that much if they are over 10,000 a day. All the script needs to do is weed them out and raise their milestone requirement most of the time. Doesn't have to be crazy accurate. The low output members will get caught by the 7 day average very easily.

==== Checking for "milestone_step" value

The 7 day average would be fine to determine the following:

#reset to null each member checking pass
"milestone_step"="";

#then check "member_daily_output" against
greater than 100,000
greater than 50,000
greater than 10,000
less than 10,000
less than 1,000
0 to 500

"milestone_step" is now set

#Figure out next milestone level
"milestone_next" = ("milestone_old" + "milestone_step")

#Then check if they pass next level
if "score_current" > "milestone_next" then award

antbhill2
Premium,MVM
join:2001-02-28
Northern VA

2 recommendations

reply to Den

I'll preface my suggestions with my thoughts on milestones as they apply to me:

At this point, I don't find them important. I seem to have fallen into a range that the stats script doesn't handle, and that's fine with me. (Rusdi, no need to add anything manually.)

However, when I was starting out, I did check the milestones much more often, and did let out a little "yay" when I happened to be high (or occasionally top) on the milestone lists. (This was back in the early days when there were two projects.) Back at that time, milestones occurred much less frequently, and it was possible for someone low in the team ranking to be high on the milestones. But of course, it varied from day to day, and that made it interesting.

As to the current situation:

I agree the milestone tiers/intervals need adjustment. I don't think it is right to have the high producers on the top of the list every day.

However, I'm a bit concerned about the complexity of basing milestones on individual outputs. I am not familiar with the code, but I would assume that current tiers and milestone intervals are stored as constants someplace, and modifying them should be fairly easy. Other options are certainly doable, but someone has to dedicate the time to implement them. Also we would need to clearly explain to everyone how the system works. It may be confusing if Folder A gets recognized at a certain level and Folder B doesn't due to having more output.

I'd suggest we first adjust the tiers/intervals and try that for some period of time before going for a more complex solution. I'm starting a spreadsheet to try to analyze the effects of different tiers. Hopefully, can complete and post that this weekend, but no promises.

I may have missed it, but what are our current tiers/intervals?

If the team feels that the milestones are still out of whack after that period, we might add a restriction that a milestone gets dropped if the individual was recently recognized (5/7/10/14 days?). Although we might establish major milestones that are exempt from this (for example, the powers of 10). Filtering the milestones could be implemented in a program separate from the stats code.

Has anyone perused any other team's forums to determine how they do milestones?



rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2

My apologies antbhill2 See Profile.

I know you & parkut See Profile have indicated that you are "fine" with not being recognized in the daily milestones, (I disagree, and think ALL folks making a milestone SHOULD be posted. it really isn't much trouble to add manually..........so far.) I will honor your requests.

This is another good example/reason to talk about this, and as a Team decide what those "Milestone" intervals should be in the future.

There is a set trigger for the milestones.

»Team Helix FAQ »Folding@Home Milestones, what's that all about?

Folding@Home Milestones, what's that all about?
A: Team Helix Folding@Home Daily Milestones are posted in recognition of members reaching specific intervals in the accumulation of credits/points for Wu's completed, not the number of Wu's completed. Click here for a summary of information on the point values for Wu's of current projects.

The current Milestone intervals are:

1-10 points, a milestone for every 1 point
11-100 points, a milestone every 25 points
101-5000 points, a milestone every 250 points
5001-20000 points, a milestone every 500 points
20001-50000 points, a milestone every 1500 points
50001 points and up, a milestone every 5000 points

The milestones are pulled from here using a visual basic script written by Starfish . When this site is unavailable for whatever reason, the milestones cannot be posted.
--
Hey, Obama!
Socialism is GREAT - UNTIL you run out of other people's money!



leibold
Premium,MVM
join:2002-07-09
Sunnyvale, CA
kudos:10
Reviews:
·SONIC.NET

1 recommendation

A bunch of thoughts on the topic, not necessarily connected or in any particular order:

I would like to suggest to keep the discussion focused on what team members would like to see with regards to milestone recognition and not worry too much on how difficult it may be to implement. There are obviously some challenges with making the milestone intervals dynamic, but most of those can be overcome by keeping more historical data to determine things like 'average daily production over the last 30 days' or whatever data we may need. Starfish See Profile's program already keeps the data for 3 days (if I remember correctly) and it would be easy enough to extend that to a longer period if needed.

Dynamic milestone steps can be perceived as unfair by a member that has a higher daily production and therefore doesn't get recognized for the same intermediate milestones that a slower team member would be recognized for. How do the members of the team feel about that ?

The suggestion from Den See Profile to base the milestone interval on daily production is a very interesting one. Given that peoples ability to contribute changes over time (computers get replaced with more powerful models, new computers are being added or old computers are retired and so on) I would suggest that lifetime daily production (total points/days since first workunit) is not a good metric to use.
However given that some workunits take a very long time to compute (I remember some that took me two weeks each) daily production cannot be based on the last 24 hours either or the milestone interval is going from one extreme to the other depending on whether the previous day was one without results or one with the large result for a multiday workunit with high point score. If we want to use daily production as the factor to determine the milestone interval we need to decide over what period (number of days) we average the daily production.

Dynamic milestone intervals will effect users who contribute seasonally (e.g.: turn off computers when it gets too hot in summer). When the folding is started again their recent daily production is none/low and they will get a number of quick milestones until their daily production number is at the correct level again. Is this an issue ? It will do the same for a member returning to folding after some absence and perhaps it is actually a nice side-effect to welcome them back with more frequent milestones initially.

Dynamic milestone intervals can be based on other factors as well, it doesn't have to be the daily production. If the goal is to try to recognize members not too frequently and not too rarely the factor to base the milestone interval on could be the number of days since the last recognized milestone (starting with the highest step value the day after a recognized milestone and then gradually dropping the step value).

Dynamic milestone intervals can cause a big milestone (such as 1.000.000) to be skipped! A user with 990.000 points and low daily production would have his next milestone at 995.000 (based on earlier suggested 5000/10000/100000 steps). If that user then increases production to get their big one million milestone quicker (e.g. asks a friend to help) it may push them to 10000 point steps putting the next milestone at 1.005.000. This is assuming the steps are the number of points gained since the last recognized milestone. I would find that undesirable.
--
Got some spare cpu cycles ? Join Team Helix or Team Starfire!



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

said by leibold:

Dynamic milestone steps can be perceived as unfair by a member that has a higher daily production and therefore doesn't get recognized for the same intermediate milestones that a slower team member would be recognized for. How do the members of the team feel about that ?

Personally, I have no problem with that.

As for everything else you mentioned, I hope everyone who wants to see daily team milestones will voice opinions. I'm fine with whatever the majority of my team mates would like to see.
--
Join Team Helix * I am praying for these friends .


rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2

1 edit
reply to leibold

said by leibold:

Starfish See Profile's program already keeps the data for 3 days (if I remember correctly) and it would be easy enough to extend that to a longer period if needed.

Not sure about that. You see, I delete, (clean out) the site where the posts, and folding records are kept. I usually clean 'em out every two weeks or so, but I always leave the last three days of both posts, and folding, otherwise there would be an enormous number of files, on NZs site.

I think when you first viewed that, you may not have realized that what was there, was what I left from recently deleting.

I don't think I told you that, when we spoke before.

Anyway, if it becomes necessary to leave those files, for whatever method of point milestone is chosen, I suppose they could stay. They are only txt. and not very large, there would just be MANY of them.

EDIT: I guess to be clearer, I upload those data files to that site daily from my computer. I don't think, (I could be wrong) the script "keeps" anything. It merely generates the reports, then I upload 'em to nozero See Profiles site, and post them here in the Daily Milestone post.
--
I didn't build this boat, I just help pull an oar or two.


rusdi
American V
Premium,MVM
join:2001-04-28
Flippin, AR
kudos:2

2 recommendations

reply to Den

Well, here's my suggested scheme.

1-10 points, a milestone for every 1 point
11-100 points, a milestone every 25 points
101-5000 points, a milestone every 250 points
5001-20000 points, a milestone every 500 points
20001-50000 points, a milestone every 1500 points
50001-100000 points, a milestone every 5000 points
100001-500000 points, a milestone every 10000 points
1000000 a day and higher, a milestone every 50000 points.
For those folks with many, many smokin' machines, or have been at it a really LONG TIME, with one machine.

I think that would cover EVERYONE folding for Team Helix, as of today.

My Main wish, along with seeing folks who, perhaps don't have but one, or two machines still being able to achieve a milestone and be recognized, also to "future-proof" the program as much as we can now, and make it relatively easy to modify the script, in the event things happen again as they have now.

This doesn't take into account, an exponential gradient for those that quit, and return.
--
I didn't build this boat, I just help pull an oar or two.



signmeuptoo
Bless you Howie
Premium
join:2001-11-22
NanoParticle
kudos:5
reply to Den

I wish I knew how to code already, I'd love to help!



sortofageek
Runs from Clowns
Premium,Mod
join:2001-08-19
kudos:21

I'm guessing the best way to help is to help decide what the intervals should be.



signmeuptoo
Bless you Howie
Premium
join:2001-11-22
NanoParticle
kudos:5

Yeah, and that is a difficult one. Looks like Russ put some good thought into his suggestion!

I suppose one way to do it would be some sort of geometric curve, and then get the rate from that. But certainly not a log curve, that is too extreme.

My deepest apologies that, in fact, I am so rusty on math that I can't even do algebra any longer. That's why I am hoping to go back to school!!!

But that is what we could do, perhaps: Look at curves on graphs and see if there is one we like?
--
Join Teams Helix and Discovery. Rest in Peace, Leonard David Smith, my best friend, you are missed badly! Rest in peace, Pop, glad our last years were good. Please pray for Colin, he has ependymoma, a brain cancer, donate to a children's Hospital.