ROOT LIT JINR

PROOF

PROOF and object of class TTree. Example.

PROOF - Parallel ROOT Facility is an extension of ROOT that allows parallel analysis of large sets ROOT files on multiprocessor machines. That is, PROOF can parallelize tasks, which can be formulated as a series of independent subtasks.
PROOF provides:

The absence (or a very small amount) of differences between the analysis session running on the local ROOT and a remote parallel session PROOF. A typical parsing script should work the same way.
No limit on the number of processors that can be used in parallel.
Adaptability of the session to changes in the remote environment (load changes on the nodes cluster, network interrupts, etc.).

In the PROOF implementation, the slave servers - "workers" - are the active components that request the main server for getting new work whenever they are ready to do it. the main goal PROOF - to minimize the time to perform a common task in the presence of all working workers ending their assigned task at the same time. In this scheme, the performance of parallel processing is a function of the duration of each small task, the "package" - the volume work assigned to the worker, and network bandwidth. The main setting parameter is the size of the package. If the packet size is too small, the effect of parallelism will be leveled. waste of time to transfer service data caused by the movement of many packets on the network between master and slave servers. If the packet size is too large, the effect of performance difference insufficiently expressed. Another very important factor is the location of the data. In most cases it is necessary to process a large number of data files that are distributed across different cluster nodes or separated from each other geographically. To group these files together uses a chain that provides a single logical view of many physical files.

A package is a simple data structure of two numbers: “initial event” and “number events. The size of the package is determined dynamically after the work of workers in real time. The main server - the "master" generates a package when requesting the slave server, taking into account the time spent processing the previous batch and the size of the files in the chain. Wizard saves the list all generated packets to one of the slave servers, so in case any "worker" "dies" during processing, all its packages can be recycled by the rest of the "workers". In principle, a package can be as small as the main processing unit an event.

To use PROOF, the user (client) must start a PROOF session. Practically it means the creation of a TProof object or a pointer to it.

root[] TRpoof *proof = TProof::Open("url")

This class manages Parallel ROOT Facility on a cluster. It starts up workers servers, keeps track of their work, status and other practices, sends messages to all workers, collects results, etc. A full description of the TProof class can be viewed. here . We will only consider main methods of TProof.

Method

TProof * TProof::Open(const char * cluster = 0, const char * conffile = 0, const char * confdir = 0, Int_t loglevel = 0)

runs a Proof session on a specific cluster. The first parameter is the URL representing the network address of the cluster master. The second parameter - conffile - is the name of the configuration file that describes the remote PROOF cluster. (this argument allows you to describe various cluster configurations). The default file is proof.conf. The third parameter, confdir, is the directory in which there is a configuration file and other PROOF-related files (for example, motd and noproof files).

Method

TProof::Print (Option_t opt)

displays the final status of the session. By default it shows information about the client and master, ROOT version, platform, location of user directories, number of active and inactive workers, real work time and CPU time and I / O used in the session, etc. etc.

Method

displays the final status of the session. By default it shows information about the client and master, ROOT version, platform, location of user directories, number of active and inactive workers, real time and CPU time and I / O used in the session, etc. etc.

Method

TProof :: Exec (const char * cmd)

allows you to execute ROOT commands on workers or on master. For example, PROOF diagnoses CINT commands that require a file ('.L', '.x' and '.X'), and make sure that the updated file version exists on nodes. The file is loaded only if necessary.

Method

TProof :: SetParallel (Int_t nodes))

tells PROOF how many auxiliary nodes to use in parallel.

When you run a macro for the first time, you may notice some waiting time due to its distributions by workers. When run a second time and beyond, if the macro has not changed, then it goes much faster. By default, the command is executed only on workers, not on master. To perform it on master, you can do the following:

root [3] proof-> SetParallel (0)

The Process method has several interface types:

Long64_t TProof::Process(TDSet * dset, const char * selector, Option_t * option = "", Long64_t 	nentries = -1,
Long64_t first = 0)

Processes a dataset implemented in ROOT by a special TDSet class using the specified selector file. (.C) or Tselector object.

Long64_t TProof :: Process (TFileCollection * fc, const char * selector, Option_t * option = "", Long64_t nentries = -1,
Long64_t first = 0)

Processes a data set (TFileCollection) using the specified selector file (.C) or a TSelector object.

Long64_t TProof :: Process (const char * selector, Long64_t n, Option_t * option = "")

General (not based on pre-selected data) selector processing: the Process () method of the specified selector object (.C) or TSelector is called "n" times. What is a selector?

Selector

To be able to parallelize at the event level, PROOF must adjust event flow. This requires that the code be defined in advance, but at the same time it has a flexible structure. In ROOT, this requirement is provided by the Selector structure defined by the abstract TSelector class, which defines three logical steps:

Begin - Begin where the input data, parameters, file are set for withdrawal; running on client and workers;
Process - The process where the actual work is done; is caused for every event on workers. This part can be parallelized;
Terminate - Final where results are processed (fit, visualize ...); caused by client and workers.

That is, you can schematically represent the structure of the Selector as follows:

void Begin() - client initialization is executed once on the client

void SlaveBegin() - node initialization, executed once on each subordinate node

void Process(Long64_t entry) - calculation, specified number of times (entry) on each subordinate node

void SlaveTerminate() - final processing on the node, executed once on each slave node

void Terminate() - processing and output of data on the client, executed once on the clien

Packetizer

Packetizer is responsible for load balancing between workers. He decides where each piece is. work - package - must be processed. The packetizer object is created on the master node. Work of workers, as well as the speed of transfer of various files can vary considerably. To dynamically balance work distribution, packetizer uses pull architecture : when workers are ready for the next handling, they ask packetizer for the next package.

Consider an example in which a one-dimensional histogram is created and filled with random numbers. obeying the Gauss distribution. First we write a script for a normal ROOT session without use PROOF.

	
#include "TH1F.h"
#include "TRandom.h"
#include "TCanvas.h"
void gauss(Int_t n=100000){
TH1F *fH1F = new TH1F("FirstH1F","First TH1F in PROOF",100,-10,10);
TRandom *fRandom=new TRandom3(0);
for(Int_t i=0;i<n;i++){
Double_t x=fRandom->Gaus(0.,1.);
fH1F->Fill(x);
}
TCanvas *c1=new TCanvas("c1","Proof ProofFirst canvas",200,10,400,400);
fH1F->Draw();
c1->Update();
}

The red color in the script highlights the definition of the histogram and random number generator. In blue - direct work of the script - in this case, the generation of a random number and the filling of the histogram with it. Data output is highlighted in green - creating a canvas and drawing a histogram on it.

Now, to perform the same work in parallel, using PROOF, let's create the class ProofFirst, derived from the abstract class TSelector. Description and methods of the TSelector class methods can be viewed. here . The header file ProofFirst.h looks like:

#ifndef ProofFirst_h
#define ProofFirst_h

#include < TSelector.h >
#include "TH1F.h"
#include "TRandom.h"
#include "TCanvas.h"

class TH1F;
class TRandom;

class ProofFirst : public TSelector {
public :

   TH1F   *fH1F;    
   TRandom *fRandom; 

   ProofFirst();
   virtual ~ProofFirst();
   virtual Int_t   Version() const { return 2; }
   virtual void    Begin(TTree *tree);
   virtual void    SlaveBegin(TTree *tree);
   virtual Bool_t  Process(Long64_t entry);
   virtual void    SetOption(const char *option) { fOption = option; }
   virtual void    SetObject(TObject *obj) { fObject = obj; }
   virtual void    SetInputList(TList *input) { fInput = input; }
   virtual TList  *GetOutputList() const { return fOutput; }
   virtual void    SlaveTerminate();
   virtual void    Terminate();

   ClassDef(ProofFirst,2);
};
#endif

and implementation file ProofFirst.C -

#include "ProofFirst.h"
#include "TH1F.h"
#include "TRandom3.h"

//_____________________________________________________________________________
ProofFirst::ProofFirst()
{
   // Constructor
   fH1F = 0;
   fRandom = 0;
}
//_____________________________________________________________________________
ProofFirst::~ProofFirst()
{
   // Destructor
   if (fRandom) delete fRandom;
}
//_____________________________________________________________________________
void ProofFirst::Begin(TTree * /*tree*/)
{
}

//_____________________________________________________________________________
void ProofFirst::SlaveBegin(TTree * /*tree*/)
{
 
   fH1F = new TH1F("FirstH1F", "First TH1F in PROOF", 100, -10., 10.);
   fOutput->Add(fH1F);
   fRandom = new TRandom3(0);
}

//_____________________________________________________________________________
Bool_t ProofFirst::Process(Long64_t)
{
     if (fRandom && fH1F) {
      Double_t x = fRandom->Gaus(0.,1.);
      fH1F->Fill(x);
   }
   return kTRUE;
}
 //_____________________________________________________________________________
 void ProofFirst::SlaveTerminate()
 {
 }
  //_____________________________________________________________________________
void ProofFirst::Terminate()
{
   TCanvas *c1 = new TCanvas("c1", "Proof ProofFirst canvas",200,10,400,400);
   fH1F = dynamic_cast(fOutput->FindObject("FirstH1F"));
   if (fH1F) fH1F->Draw();
   c1->Update();
}

We announce the histogram and random number generator as data members of the ProofFirst class (red color). In the constructor class, initialize the 0 histogram and generator pointers to 0. In the destructor we destroy the random generator. numbers The histogram belongs to the output list, so it is not destroyed. The SlaveBegin () method creates instances of the histogram and random number generator (red). Main job performed by the Process () method, in this case it is the generation of a random number and its filling of the histogram (blue color). Finally, the Terminate () method shows the result on the terminal (green).

Now we are ready to handle this selector. Therefore, we run PROOF

TProof * proof = TProof::Open ("lite://","workers = 12")

and call the method

 TProof::Process("ProofFirst.C+", 1000000000)

Here is what you should see on your screens:

A more complicated example can be found in the following link : tree reading, calculations, histogram filling, fitting.