Suggestion for better task progress bar

Message boards : Number crunching : Suggestion for better task progress bar
Message board moderation

To post messages, you must log in.

AuthorMessage
pschoefer
Avatar

Send message
Joined: 1 Jan 21
Posts: 9
Credit: 2,792,393
RAC: 993
Message 892 - Posted: 8 May 2021, 14:15:00 UTC

With checkpoints already appearing on the horizon, I think it is time to revisit another possible improvement of the application from the volunteers' point of view.

Right now, the shown task progress is just estimated by a standard BOINC routine based on the initial runtime estimate. If the runtime is overestimated, tasks will jump to 100% from much smaller percentages, whereas the progress will crawl asymptotically toward 100%, if the runtime is underestimated. Both behaviours can be confusing for new volunteers in particular, and while the runtime estimate may settle for a realistic value in the long-term, it is most likely off for newly attached hosts.

CMDock reports a more useful progress estimate in its logfile, and I guess that something along the lines of reading that estimate and converting it to a fraction that can be read by the wrapper was the purpose of the script that was removed before I joined this project. For a more stable way to get the progress estimate in a wrapper-readable format, I suggest a small modification to CMDock.

As a proof of concept, walli and I have come up with the following patch:
diff --git a/src/exe/cmdock.cxx b/src/exe/cmdock.cxx
index 62adbe8..689ea5e 100644
--- a/src/exe/cmdock.cxx
+++ b/src/exe/cmdock.cxx
@@ -647,13 +647,19 @@ int main(int argc, char *argv[]) {
         std::cout << "Ligand docking duration:      " << recordDuration.count()
                   << " second(s)" << std::endl;
         totalDuration += recordDuration;
+        std::size_t estNumRecords = spMdlFileSource->GetEstimatedNumRecords();
+        // set "fraction_done" after every record
+        double progress = (double)nRec/(double)estNumRecords;
+        std::fstream fraction_done_filename;
+        fraction_done_filename.open("fraction_done", std::ios::out | std::ios::trunc);
+        fraction_done_filename << progress << std::endl;
+        fraction_done_filename.close();
         // report average every 10th record starting from the 1st
         if (nRec % 10 == 1) {
           std::cout << std::endl
                     << "Average duration per ligand:  "
                     << totalDuration.count() / static_cast<double>(nRec)
                     << " second(s)" << std::endl;
-          std::size_t estNumRecords = spMdlFileSource->GetEstimatedNumRecords();
           if (estNumRecords > 0) {
             std::chrono::duration<double> estimatedTimeRemaining =
                 estNumRecords * (totalDuration / static_cast<double>(nRec));

With this patch, CMDock writes its progress as a simple fraction in a fraction_done file that can than be specified as fraction_done_filename in the job.xml.

I successfully finished two tasks using the patched CMDock on this host, and am also testing it on a Raspberry Pi right now. While the reported progress is not completely linear (because some ligands take more time than others), it is much better than BOINC's progress estimate. I even set the fraction_done_exact option, so that the remaining runtime is estimated based on the reported progress. While the estimated remaining runtime jumps a lot in the beginning because the progress report is not very fine-grained, it becomes a very realistic estimate after the first few ligands.
ID: 892 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
koschi

Send message
Joined: 23 Oct 20
Posts: 3
Credit: 9,087,508
RAC: 15,274
Message 898 - Posted: 9 May 2021, 22:01:04 UTC

Good job pschoefer, I hope this makes it into the next CmDock release!
ID: 898 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 9 Oct 20
Posts: 185
Credit: 2,782,517
RAC: 50
Message 900 - Posted: 10 May 2021, 10:10:15 UTC - in response to Message 892.  

Nice work!!! It will definitely go into the next release. Also, if you are willing, you may make a proper progress bar at the command line output. We would be grateful!
(Natalia, on behalf on Marko and devs team)
ID: 900 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
walli

Send message
Joined: 2 May 21
Posts: 7
Credit: 28,958,563
RAC: 173,639
Message 950 - Posted: 22 May 2021, 15:30:57 UTC
Last modified: 22 May 2021, 15:38:28 UTC

Hi Natalia,

what a pity that you released the unaltered v0.1.2 by now. The progress estimation basically comes from CmDock itself and is quite accurate after ~10% runtime (-61 records => 61/500=0,122 => 12,2%), see:



What exactly do you mean with "... proper progress bar at the command line output."? You could simply put the "progress" variable somewhere in the "std::cout" within the "if" statement below :).

Or give us a concrete example of what you imagine and we'll take another look...

walli
ID: 950 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
walli

Send message
Joined: 2 May 21
Posts: 7
Credit: 28,958,563
RAC: 173,639
Message 971 - Posted: 26 May 2021, 0:26:12 UTC - in response to Message 950.  

Btw., to be on the safe side, don't use the C style typecast in

double progress = (double)nRec/(double)estNumRecords;

as i did, but

double progress = static_cast<double>(nRec) / static_cast<double>(estNumRecords);

(or at least for nRec.)
ID: 971 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Suggestion for better task progress bar

©2024 SiDock@home Team