New 'single validation' of DDDT

[[XC] Adywebb]

Apparently within the next week or so WCG will start beta testing new 'single validation' DDDT WU's.

They have not yet announced yet how these new units will be issued credit, but if they only need one computation to validate and no quorum I'm wondering if this will mean you will get what you claim, or if they will have some kind of fixed credit

Anyway, watch this space........

[Phili]

That sounds awesome, that'll save so much CPU power!

[twilyth]

Thanks for the update - I haven't been checking the forums very often.

[Hymay]

I may have to set my amd to do do ddt exclusively then. For some reason it either overclaims, or I get paired with really badly benched machines. It virtually never gets its claim or higher... and gets "skunked" (30% rule crap with quorum of 2) pretty frequently. I think I've gotten over my claim twice since I got it running.
Downloaded the client directly from WCG.. so I know its not using leftover optimized files from the old days.
Fairly irritating.. but I just couldnt' stand having an all intel stable.

[anubis]

me thinks they send both units to one computer..one pc quorum

[[XC] Adywebb]

me thinks they send both units to one computer..one pc quorum

I doubt that very much - they might as well continue having 2 unit quorum validation in that case.

[DDTUNG]

Please keep us closely posted on this development. It would be nice if my quad farm could start getting what it claims for work done.

DDTUNG

[anubis]

I doubt that very much - they might as well continue having 2 unit quorum validation in that case.

but why would it be a bad idea? if the outcomes differ they can just send it to someone else. but youd get your points worth if the results match.

[sierra_bound]

They are probably going to send out just one work unit, with no duplicates. The Dengue project is supposed to run for another six months or so. I think the scientitsts just want to process as many WU's as possible during that time span.

[[XC] Adywebb]

but why would it be a bad idea? if the outcomes differ they can just send it to someone else. but youd get your points worth if the results match.

The idea of single validation is they only need to send out 1 unit for computation - if they still send out 2 even if its to the same computer then it defeats the object of the excercise.

Also if that PC is throwing up errors, aborts, doesn't crunch them in time etc, then they will have to send out another 2 to another computer - so they've now sent 4

[anubis]

but if they only send out 1 wu then how do they know it errored? sorry if its a stupid question

[sierra_bound]

I'm not sure what you mean by "error". If there is a computational error, the result will be thrown out anyway and you would receive no credit. On any result that looks medically promising, the researchers will probably re-run it again themselves for verification purposes.

[Hymay]

Any new word on these, or an ETA when they will start hitting?

[[XC] Adywebb]

Nothing more - they still haven't yet announced the start of the Beta testing.

[fallwind]

Just a quick post to say that these new WU's are now going out. I have a few crunching right now.

[D_A]

How long do these run and on what hardware?

[Naja002]

How can we tell the difference?

[fallwind]

Looks like 2.5-3.25hrs on my QX9650@4GHz, so their reasonable length. DDDT hasn't had any work sent out in quite a while so anything you have now will be the new single validation units. But if you want to check, go to my grid>>results status and filter the results for DDDT. Then click on the WU names to see if you're the only one in the quorum.

I'm interested too see how points are awarded. If it simply awards what you claim it would be great since my x64 machines get slaughtered in the quorum in all projects.

[fallwind]

Get a load of this! I'm the only one in the quorum, so I have no idea how they are calculating claimed vs awarded points.

[[XC] Adywebb]

This is an explanation about how single validation works from one of the Techs at WCG:

Single validation is going to work in the following way:

Workunits are loaded into BOINC with a quorum size of 1. This means that 1 replica is created and it will only take 1 successfully run result in order for validation to be attempted on the result.

However, there are a few checks in place:

1) The server maintains a value that serves to inform us of how 'reliable' a given computer is. When a result is assigned to the computer for a workunit with a quorum of 1, then the reliable measurement is checked to make sure that the computer is sufficiently reliable. If it is, then all is good. If it isn't, then the workunit is changed to have a min_quorum of two and a second copy is sent.

2) When validation is attempted, the value for the host is checked again. If the value has fallen below the required level, then the result is marked INCONCLUSIVE and another result is sent.

3) Additionally, during validation, there is a certain random chance that the result will be flagged to be checked again. Any result picked in this case will be marked INCONCLUSIVE until the validation with the additional result occurs. All computers are subject to random checking.

4) We have also added some additional checks within the research apps to detect errant results. Part of this is a short run of the application that computes a known result (this was part of what we ran last week). This short run will be used to help ensure that the computation ran correctly as well as it will be used to determine the appropriate credit to award.

Point 4 is the only place where it refers to credit allocation - its not very clear to me, however I am guessing they do a short run of WU's themselves to get some kind of average speed for computation and credit claim and then look at the efficiency of your machines and award granted credit according to some formula or other

That's the best I can come up with at the moment I'm afraid...

[Jaco]

I don't understand it. What is the reliable measurement ?

How is the reliability of computer checked?
Is there a small test at the beginning of each WU or are they keeping a database about how reliable your machine is (based on previous results) ?

[D_A]

That sounds like it might be the case. The latter thing that is.

[Ramaistro]

Well the new units are giving me a hard time allready have like 6 inconclusives and some that are valid. And indeed is see that another unit is send out now that mine is inconclusive.

Hope it has something to do with the wu's because all other wu's are acting normal on my machines.

[[XC] Adywebb]

I don't understand it. What is the reliable measurement ?

How is the reliability of computer checked?
Is there a small test at the beginning of each WU or are they keeping a database about how reliable your machine is (based on previous results) ?

I've asked the question over at WCG - so far 2 conflicting replies from CA's, so awaiting for a more definative answer from a Tech....however I think these are probably near the mark:

There is a little benchmark right at the beginning of a job, every job, that computes a known piece of science. That and the variation for each DDDT benchmark is used to build a claim rating for a device forming part of the computation of credit. Let me dig out if anything was written over what was said in point 4.

I've not done new DDDT yet, so cant say from hands on, but one CA achieved a claim to grant variation between 0 and 4.3%

The workings for the new FAAH will be the same. I just know from the 43 beta I got of last test, the difference was minimal.

And an update:

As what i wrote earlier it's definitely hinging on that little pre-job integrated benchmark of a reference piece of DDDT science. Do that well, you get equal or more than the claim.... do it not so well, you get a little less. Over time it's expected to converge, so at least the data from the beta test supported the theory.

[Jaco]

pre-job integrated benchmark.
That's for crediting purposes, i would think

That 'reliable measurement' still confuses me .