SLAC PEP-II
BABAR
SLAC<->RAL
Babar logo
Workbook HEPIC Databases PDG HEP preprints
Organization Detector Computing Physics Documentation
Personnel Glossary Sitemap Search Hypernews
Unwrap page!
Wkbk. Search
Wkbk. Sitemap
Introduction
Non SLAC
HOWTO's
Introduction
Logging In
QuickTour
Detector
Info Resources
Software Infrastructure
CM2 Introduction
Unix
OO
SRT
Objectivity
Event Store
Framework
Beta
Modifying Code
Writing and Editing
Compiling
Debugging
Analysis
Framework II
Analysis
Find Data
Batch Processing
PAW
PAW II
ROOT I
ROOT II
ROOT III
Advanced Infrastructure
New Releases
Workdir
Main Packages
Event Displays
Gen/Sim/Reco
Contributing Software
SRT and CVS
Coding
Advanced Topics
Make CM2 Ntuples
New Packages
New Packages 2
Persistent Classes
Java
Site Installation
Check this page for HTML 4.01 Transitional compliance with the
W3C Validator
(More checks...)

Workbook for Offline Users: New Packages: Making a new Analysis package based on BetaMiniUser

This chapter describes how to create a new analysis package based on the BetaMiniUser package. This is useful for users wanting to write their own Beta analysis and to use the powerful CVS tools to maintain the package. In this section step-by-step instructions are provided to help users make all the necessary changes required to get their new package into a state that it can be included in the BaBar CVS repository as a unique analysis package.

At this stage, the analysis package you will create will belong to you only, and will not yet exist in the BaBar CVS repository.

The following workbook chapter New Packages: Getting your package into CVS describes how to get your new, tested and operational, package into the BaBar framework and into official releases.


Contents


Introduction

The general procedure to set up a new package and integrate it into the BaBar framework is a two-step process:
  1. Create and test your new analysis package. You may have been modifying BaBar packages for your own use for a while now, in which case the instructions below will just give step-by-step instructions of how tochange the package to something uniquely named so that it can be put into the BaBar framework. Or you may be relatively new to editing code, in which case this example may teach you a bit about how a BaBar package works.
  2. Add your package to the BaBar repository by testing it carefully, requesting the creation of a new package (basically telling the computing gurus that you have a new package called TopUser) and starting to use your new package in CVS

The first step is described in this chapter, while the second step is explained in the next WorkBook chapter New Packages: Getting your package into CVS. Making a new analysis package is usually a good idea as it allows several people to work together on analyses and commit their changes using all the tools of CVS, and also makes available the tools developed by the group for the whole collaboration. Additionally, making source code available can be useful at the review stage to make all steps in the analysis transparent and testable. Additionally, CVS allows your code to keep in step with different releases, allowing you to tag specific versions of your code that work with specific BaBar code releases. Note also that you should NEVER check code back into BetaMiniUser (or any other package) without specific permission of the package coordinators.


Package Naming

You should name your package to give a clear indication of its role, and all the files within that package should be similarly named. Three-letter acronyms are favoured within BaBar.

Analysis packages written for specific physics groups are commonly named as SomethingUser, for example, BetaMiniUser, TauUser, CharmUser, etc.

Packages that are related to detector subsystems should be named with the three-letter acronym that is appropriate to the subsystem, such as Emc, Drc, Dch, etc.

To allow specific examples to be used throughout this chapter, we shall describe the procedure to set up a test release called top-a30, based on analysis-30 and create a new analysis package that we shall call "TopUser", based on the BetaMiniUser package.


Setting up the TopUser package

To start, follow the WorkBook QuickTour and check out a BaBar software release: newrel -t etc., with the test release name top-a30 , (see the QuickTour section of the Workbook). It is recommended at this stage that you base your test release on analysis-30.

Now check out the workdir and BetaMiniUser packages in the usual way and set up the workdir package:

~> cd top-a30
top-a30> srtpath <enter> <enter>
top-a30> addpkg workdir
top-a30> gmake workdir.setup
top-a30> addpkg BetaMiniUser

Now copy the files in BetaMiniUser into a new directory called TopUser:

top-a30> mkdir TopUser
top-a30> cp BetaMiniUser/* TopUser

The system will probably respond,

cp: omitting directory `BetaMiniUser/CVS

That is because this command copies only the files (not the subdirectories or their contents) to TopUser. But that is what you want, because BetaMiniUser's only directory is the CVS directory, which holds information about the CVS repository where the BetaMiniUser package resides. Our new package isn't yet in CVS (it doesn't actually exist yet), so it would be inappropriate to have a CVS directory at this point.

To check what went in there:

top-a30> cd TopUser
TopUser> ls

AppUserBuild.cc             MyReadList.hh
AppUserBuildBase.cc         MyReadList.tcl
AppUserBuildBase.hh         MyReadListSnippet.tcl
AppUserBuildRoo.cc          MyReskimStreamPhysics.tcl
BetaMiniPatches.tcl         MyTimePointAnalysis.cc
BetaMiniPidKilling.tcl      MyTimePointAnalysis.hh
BetaMiniUserProduction.tcl  MyTimePointAnalysis.tcl
GNUmakefile                 NamedScalers.cc
History                     NamedScalers.hh
MyDstarAnalysis.cc          README
MyDstarAnalysis.hh          RewriteMini.tcl
MyDstarAnalysisSnippet.tcl  bdbMini.tcl
MyDstarMicroAnalysis.tcl    bdbMiniPhysProdSequence.tcl
MyDstarMiniAnalysis.tcl     bdbMiniPhysics.tcl
MyDstarPhysics.tcl          bdbMiniQA.tcl
MyHistManager.cc            bin_BetaMiniApp.mk
MyHistManager.hh            bin_BetaMiniRooApp.mk
MyK0Analysis.cc             bin_BetaMiniUser.mk
MyK0Analysis.hh             binlist
MyK0MicroAnalysis.tcl       btaMini.tcl
MyK0MiniAnalysis.tcl        btaMiniPhysProdSequence.tcl
MyMiniAnalysis.cc           btaMiniPhysics.tcl
MyMiniAnalysis.hh           btaMiniQA.tcl
MyMiniAnalysis.tcl          link_BetaMiniUser.mk
MyReadList.cc

If you have followed the workbook up until this point, then you will recall the PExample module that you made in the Editing Code and Compile,link,run sections of the Workbook. In the following example you will add the PExample module to your new TopUser package. So you need to copy the PExample files to your TopUser directory:

TopUser> cp $BFROOT/www/doc/workbook/NewExamples/PExample/* .
(Don't forget the dot "." at the end.)

Now you will have two new files in TopUser - PExample.cc and PExample.hh - as well as modified versions of AppUserBuildBase.cc and MyMiniAnalysis.tcl. You may also have a file named header.inc. This is a Workbook file that makes Workbook pages look nice - it has nothing to do with PExample, so you should remove it.

The next step is to get rid of all of the files that are not needed in your new package. Your goal is to produce a package with just one module - the PExample module - plus any modules needed to run PExample. As you saw in the Quicktour, you need the MyMiniAnalysis module, but the others can go:

TopUser> rm MyDstar*
TopUser> rm MyK0*
TopUser> rm MyTime*
TopUser> rm MyRead*

You also need to edit AppUserBuildBase.cc and MyMiniAnalysis.tcl to remove all reference to these modules. For example, for the MyDstarAnalysis module, you will need to remove the lines

#include "BetaMiniUser/MyDstarAnalysis.hh"
and
  theBuild->add(new MyDstarAnalysis("MyDstarAnalysis", "Dstar analysis module"));
from AppUserBuildBase.cc, and remove
module disable MyDstarAnalysis
from MyMiniAnalysis.tcl. The lines for the other modules are similar (or nonexistant).

It is probably a good idea to remove the History and README files as well. These files are not part of the code, they just provide information about the package. But as they contain many references to files in the package, they may get in the way when you are tracking down dependencies with grep.

TopUser> rm History
TopUser> rm README

With regard to the other files, it is not obvious whether they need to be removed or not. To find out, use grep (the unix search utility) to see where they are used. For example, let's grep all the files in TopUser for BetaMiniPatches:

TopUser> grep BetaMiniPatches *

BetaMiniPatches.tcl:# $Id: BetaMiniPatches.tcl,v 1.3 2003/04/08 18:13:04 brownd Exp $
BetaMiniPatches.tcl:#       puts "BetaMiniPatches: Setting DchBuildEnv::dedxCalibBkgStrategy to false"
btaMini.tcl:set patches BetaMiniPatches.tcl

The first two lines are in BetaMiniPatches.tcl itself, so you do not need to worry about them. However, the third line shows that btaMini.tcl uses BetaMiniPatches.tcl. So if you are going to keep btaMini.tcl, then you will also have to keep BetaMiniPatches.tcl. Similar investigations of the other files will reveal that you can remove the following:

TopUser> rm BetaMiniPidKilling.tcl
TopUser> rm BetaMiniUserProduction.tcl
TopUser> rm MyReskimStreamPhysics.tcl
TopUser> rm RewriteMini.tcl
TopUser> rm bdb*
TopUser> rm btaMiniQA.tcl
TopUser> rm NamedScalers*
TopUser> rm header.inc
TopUser> rm MyHistManager*
Now you are left with the following files:
TopUser> ls

AppUserBuild.cc      MyMiniAnalysis.hh      binlist
AppUserBuildBase.cc  MyMiniAnalysis.tcl     btaMini.tcl
AppUserBuildBase.hh  PExample.cc            btaMiniPhysProdSequence.tcl
AppUserBuildRoo.cc   PExample.hh            btaMiniPhysics.tcl
BetaMiniPatches.tcl  bin_BetaMiniApp.mk     link_BetaMiniUser.mk
GNUmakefile          bin_BetaMiniRooApp.mk
MyMiniAnalysis.cc    bin_BetaMiniUser.mk

As you edit, your system may automatically save backup copies of some files. These files have the same name as the original, except for a tilde (~) at the end. I find it easier to delete these files as they appear, so that they do not cause problems when I use grep to check for dependencies.


Renaming source code and tcl files

The first thing you should now consider doing is renaming your source code files so that the names are indicative of belonging to a package called TopUser. You also need to do a bit of renaming to let the files know they are now in the package TopUser, not BetaMiniUser any longer.

To do this, grep for all instances of BetaMiniUser in the source code.

TopUser> grep BetaMiniUser *.cc
TopUser> grep BetaMiniUser *.hh
TopUser> grep BetaMiniUser *.tcl
Doing so reveals that there are six files that you need to edit, and change each instance of BetaMiniUser to TopUser:
AppUserBuild.cc
AppUserBuildBase.cc
AppUserBuildBase.hh
AppUserBuildRoo.cc
PExample.cc
MyMiniAnalysis.cc
MyMiniAnalysis.tcl
btaMiniPhysProdSequence.tcl
btaMiniPhysics.tcl

The TopMain module

BaBar packages generally have source files named with three-letter acronyms (TLAs), which identify those files as belonging to the package. In our case, "Top" is appropriate to use.

Now we should rename PExample.* so that the cc and hh files look like they belong to a package called TopUser. For now, let's just go with TopMain, since this is the main analysis source code for our package:

mv PExample.cc TopMain.cc
mv PExample.hh TopMain.hh

You now must go through TopMain.* and change all instances of PExample to TopMain. You also need to change the two lines with PEXAMPLE in capitals to TOPMAIN in TopMain.hh.

The last step is to check for and edit any other files that depend on PExample:

TopUser> grep PExample *
This will reveal that you have two files to edit: AppUserBuildBase.cc and MyMiniAnalysis.tcl. In both files you need to change all instances of PExample to TopMain.

Taking these steps has renamed the module PExample to be now a module called TopMain.

Other files

There are a few other files that should be renamed to reflect their new roles as members of the TopUser package:

mv BetaMiniPatches.tcl TopPatches.tcl
mv btaMini.tcl topMini.tcl
mv btaMiniPhysProdSequence.tcl topMiniPhysProdSequence.tcl
mv btaMiniPhysics.tcl topMiniPhysics.tcl
A check for references to these files
TopUser> grep BetaMiniPatches *
TopUser> grep btaMini *
reveals that you have five files to edit:
TopPatches.tcl
MyMiniAnalysis.tcl
topMini.tcl
topMiniPhysProdSequence.tcl
topMiniPhysics.tcl
and change BetaMiniPatches to TopPatches, and btaMini to topMini.

AppUserBuild.cc

AppUserBuild*.cc are the files used the build your executable. Since they are such important files, it is a good idea to check them to make sure you did not forget anything before you continue.

Many BABAR packages come with an AppUserBuild.cc file. The setup in the analysis-30 version of BetaMiniUser is a bit unusual: instead of one AppUserBuild.cc file, there are three: AppUserBuild.cc for older "bdb" code, AppUserBuildRoo.cc for newer "roo" code, and AppUserBuildBase.cc for code common to bdb and roo. In most packages, however, you will have only one AppUserBuild.cc file.

The original AppUserBuild files had "#include BetaMiniUser/file.hh" statements for several BetaMiniUser modules. Make sure that you have changed all instances of BetaMiniUser to TopUser.

PExample also appeared in the original AppUserBuildBase.cc in two places: in an "include" statement, and in the constructor of the AppUserBuildBase class. Make sure that you replaced all instances of PExample with TopMain.

In the constructor, you can also change the description of TopMain if you like, for example, to "New package example" - the second argument of this line isn't that important, while the first argument is what the module will be referred to and must not contain spaces.


Dependency and make files

Dependency files

The dependency files are:

link_BetaMiniUser.mk 

bin_BetaMiniUser.mk
bin_BetaMiniApp.mk
bin_BetaMiniRooApp.mk

link_<package>.mk; files tells the compiler what packages your package depends on. bin_<package>.mk; files tell the compiler what packages your binaries depend on. Most packages put all of their dependencies in one bin_<package>.mk file. However, some packages (like BetaMiniUser) with multiple binaries have one binXXX.mk file per binary.

From time to time you will need to update these files as your dependencies change (for example, if you add some code to do a complicated task, you may require another BaBar package for this particular code to work correctly. Fortunately help is at hand, as the SoftRelTools (Software Release Tools) package contains a script that you can use at any time to update your link_ and bin_ The way to use this script is to go into your release directory and issue the commands:

gmake installdirs
workdir/PARENT/SoftRelTools/make-linkfiles -p TopUser

This will create two new files:

link_TopUser.mk.NEW
bin_TopUser.mk.NEW
in your TopUser directory with updated dependences. All you need to do next is to satisfy yourself that these files are sensible and then overwrite the existing *_TopUser.mk files.

Of course, in order to know if the new versions are sensible, you need to understand a bit more about how they work. Here is a brief explanation.

bin_TopUser.mk contains lines of the form:

override LINK_BaBar   += $(PACKAGE)GNUmakefile
If you were making this file from scratch, then to find out which lines are needed in here, you would look through the .cc file that is compiled to produce an executable. In most applications, this file will be AppUserBuild.cc. (In BetaMiniUser, you have to check all 3 AppUserBuild*.cc files.) There is a LINK_* entry for each package for which there is a file included with
#include "Package/file.hh"
Although you may get away with only including lines for a subset of these files, it is likely that it will catch up with you one day when you change operating system or release.

link_TopUser.mk is also a required file for each package. For each package from which something was included and will be compiled into the libraries, there is a line of the form

override LINK_BaBar                += TopUserGNUmakefile
If you were making this file from scratch, you would have to go through all of the .cc files apart from the one that compiles into your binary (executable) and put a line in the link_TopUser.mk file for each package that is included with a line like
#include "Package/file.hh"
as it is these lines that tell the compiler that your package needs information from the other listed packages.

In this case, if you compare the files produced with make-linkfiles with the older ones, you will find that references to many packages have been removed. For example, in link.TopUser.mk.NEW, the lines for TrgTools, TrkBase, and TrkFitter (among others) have been removed. These are packages that were used by one or more of the modules that you removed. Only the modules needed by MyMiniAnalysis and TopMain will remain. This is exactly what you want.

The make-linkfiles command produced only one bin_TopUser.mk file, not three. But if you compare it to the original 3 bin_XXX.mk files, you will find that all of the dependencies from the original files have been included in the single new file. If your PExample module had introduced new dependencies for the binaries, then these would have been included as well. Now that we've confirmed that the new files are OK, let's replace the old files:

mv link_TopUser.mk.NEW link_TopUser.mk
mv bin_TopUser.mk.NEW bin_TopUser.mk

rm link_BetaMiniUser.mk 

rm bin_BetaMiniUser.mk
rm bin_BetaMiniApp.mk
rm bin_BetaMiniRooApp.mk

Related documentation

GNUmakefile

Next you should sort out your GNUmakefile. The original BetaMiniUser package could produce two binaries: BetaMiniApp, and the "extra" binary BetaMiniRooApp. (An extra binary is one that is compiled only if the user explicitly requests it via "gmake BinaryName.bin".) Here, we simply rename the binaries: BetaMiniApp becomes TopMiniApp, and BetaMiniRooApp becomes TopMiniRooApp.

(You will notice that the GNUmakefile contains references to the AppUserBuild files. This is where the GNUmakefile tells the compiler what source files are needed to build the executables.)

In TopUser there is also a one-line file called "binlist". You need to change BetaMiniApp to TopMiniApp in this file.

When you start writing new code to go into your new package, it is strongly recommended that you put all of your source code in a single package directory. Subdirectories should be used only for things like documentation, auxiliary files, and macros. If you break from this convention, you will need to modify your GNUmakefile accordingly.

Related documentation


Finishing touches

If you have not already, now is a good time to remove any backup files (files with a tilde ~ at the end) in your TopUser package.

If you were really planning to add this package to CVS, you would also want to go through each file and edit the comments. For example, there may be references to BetaMiniApp that should be changed to TopUserApp. These will not harm the program, so we can leave them in this case. But in a real package you want your comments to be accurate.

Also, a real package would come with its own History and README files, to tell the user about the evolution of the package and how to use it. So if you were planning to add this package to CVS, you would need to write those files.

Now is also a good time to remove your BetaMiniUser package.

top-a30> rm -r BetaMiniUser

Telling the framework about your package

Now you have a unique package with appropriately named files. All you need to do is tell the BaBar framework about your package, and you can start your analysis.

Because yours is a new package, and not part of the release contained in - and checked out from - cvs, you need to tell the framework that the package exists. In this preliminary step before putting your package into the BaBar CVS repository, you do this by checking out the package PackageList:

top-a30> addpkg PackageList

PackageList contains a file called link_all_reco.mk. You need to edit this file to include the existence of your own new package. This is easy - just copy a chunk of the repeated code in link_all_reco.mk and put your details in there. Be sure to put your package information above any packages that yours depends upon. If you are setting up a standard analysis package, then a good place to start is to look for the entry for TauUser and put it either above or below that entry. The syntax you need to introduce into link_all_reco.mk is:

PACKAGELIST += TopUser
ifneq ($(LINK_TopUser),)
        LINKLISTDEPENDS += [LINK_TopUser, $(LINK_TopUser)]
        override LOADLIBES += -lTopUser
        -include TopUser/link_TopUser.mk
endif

If you forget to do this step, or if you subsequently introduce a dependency on another package, you might get compiler messages about "symbol referencing errors" - so check that indeed your package is listed in link_all_reco.mk above all the packages that it depends upon. Further information is available in HOWTO-dependency in the SoftRelTools package


Compiling and running your new binary

Before starting, it is useful to issue a

top-a30> gmake installdirs
command from your test release directory, just to make sure that the required directories for temporary binary and library files exist. Also don't forget that in each session, each time you log in, you must issue the command:
top-a30> srtpath
Finally, as in the Quicktour, you will need a file called .bbobjy in your test release directory, containing the single line,
FD_NUMBER = ****
where **** is one of your 4-digit FDID numbers. To find the FDID numbers assigned to you, use the command
GetFDID.

Compile your code

Now you can compile and link as in the WorkBook Quicktour, with:

top-a30> bsub -q bldrecoq -o all.log gmake all
If there are any error messages, you must deal with them first, but you can generally ignore warnings.

If you edit any header files you will need to recompile the library files with gmake lib (remember: all gmake commands should be issued from the release directory). gmake bin is used whenever source code is changed. Executables are stored in /bin/$BFARCH. To check that everything worked, you can do:

top-a30> ls -l bin/$BFARCH
and make sure the binary file you find there, TopMiniApp, has a time stamp of only a minute or two ago (i.e. that it was just made).

From time to time doing gmake clean to clean up files is a good idea - remember that it removes existing executables, but also cleans up library files - which can get rid of seemingly inexplicable errors.

To check that your binary is ok, you can now follow the steps in the Workbook QuickTour to see that your job runs ok. (Note that you will need to copy your snippet tcl file to TopUser, and edit it to change BetaMiniUser to TopUser. The run command also changes; now it will be "TopMiniApp snippet.tcl".)

If the resulting ROOT file, myHistogram.root, has only empty histograms, you should check that you have correctly followed all the steps in this section, and that you have not put "TopMain" in the place of "TopUser" or vice versa. The result of such an error is generally either an error message, or simply the failure of the BaBar framework to include your module in the full analysis sequence, thereby resulting in your histograms being booked, but never filled.

As a last step, we should also check that the "extra" binary, TopMiniRooApp, links correctly. To force an extra binary to be compiled, you must call it by name:

ana30> gmake TopUser.TopMiniRooApp

Once again, check the bin/$BFARCH directory to make sure TopMiniRooApp was produced.

An example of a working TopUser package, produced using the instructions in this section, is located at:

$BFROOT/www/doc/workbook/NewExamples/TopUser

Related Documents:

Getting help

While attempting to compile and link a new package, many unfamiliar error messages are likely to occur. The best place to start looking for help is one of the BaBar Hypernews groups. Depending on the particular question, one of the following groups might be most appropriate (it is best not to cross-post to more than one group):

When you do post to hypernews groups, you should usually state the packages you are using (mention that your own package is based on BetaMiniUser if this is correct), state which release your test release is based on, and what machine architecture also. You should also give the relevant part of any log files you have if you were running your analysis and it crashed and/or the traceback from the debugger. It can also be useful if you are running at SLAC to give the path to your files so that a helpfully-inclined person can go and get their hands dirty to help you.

Conclusion

Now your shiny new package is ready to be filled up with powerful analysis code. Since your package and executable are uniquely named, you can safely allow others to copy your package and modify files or simply test what's there. However:
  1. your package at this stage is only guaranteed to work for the release analysis-30 (it will work for other releases unless there has been an incompatible change in the basic BaBar code that is called by TopUser),
  2. your package is still not an official BaBar package, so whenever it is used, the PackageList package must also be checked out and the lines telling link_all_reco.mk about your package must be added.

Once you are happy the your package is working correctly, and that it does something useful, you can release it to the (BaBar) world. To get your new package included in the BaBar CVS repository and in future BaBar releases, follow the steps in the next section of the BaBar WorkBook New Packages: Getting your package into CVS.


Back to Workbook Front Page

Authors:
Jenny Williams

Last modified: 21 January 2006
Last significant update: 17 July 2005