Difference: AnalysisSoftwareInstructions (21 vs. 22)

Revision 222009-06-17 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="MuCapGroup"
-- PeterWinter - 18 Dec 2007
Changed:
<
<
This page will describe in the future all necessary steps, to setup the various analysis tools. -- BrendanKiburg - 18 Dec 2007
>
>
This page will describe in the future all necessary steps, to setup the various analysis tools. -- BrendanKiburg - 18 Dec 2007
  -- PeterWinter - 09 Jan 2008

Setting your environment variables

In the following sections, you will often have to set some environment variables and/or change the PATH. Depending on the system you have to do this in different locations:
Changed:
<
<
  • On your own cluster, PC, laptop etc. you can set them in your ~/.bashrc (or equivalent like ~/.tcshrc). To effect the changes execute
>
>
  • On your own cluster, PC, laptop etc. you can set them in your ~/.bashrc (or equivalent like ~/.tcshrc). To effect the changes execute

 > source ~/.bashrc
Changed:
<
<
Please refer to your shell manual for details.
  • On NCSA, you edit the ~/.soft file and execute
  • >
    >
    Please refer to your shell manual for details.
    • On NCSA, you edit the ~/.soft file and execute

     > resoft
    Changed:
    <
    <
    Please visit the softEnv manual at NCSA for more details.
    >
    >
    Please visit the softEnv manual at NCSA for more details.
     

    Have Root installed:

    If your system has Root installed, go on to the next section. If not (like on tungsten.ncsa.uiuc.edu), then install it locally:
    Changed:
    <
    <
    1. Downloaded source tar for root versions 5.08 from root.cern.ch
    2. tar -xzvf root_v5.08.00.source.tar.gz
    3. cd root
    4. > ./configure linux --enable-shared --enable-thread --disable-ssl
      > make
      > make install
    >
    >
    1. Downloaded source tar for root versions 5.08 from root.cern.ch
    2. tar -xzvf root_v5.08.00.source.tar.gz
    3. cd root
    4. > ./configure linux --enable-shared --enable-thread --disable-ssl
      > make
      > make install
     
    1. Set $ROOTSYS to your current working version of Root (e.g. ROOTSYS = /u/ac/bkiburg/root)
    2. Add the Root libraries to LD_LIBRARY_PATH: LD_LIBRARY_PATH += $ROOTSYS/lib
    3. Add Root to your path: PATH += $ROOTSYS/bin
    Line: 61 to 48
     

    Midas:

    Even if you have MIDAS already installe on your system, you should probably recompile it with the mana.c from the mu/misc/ to avoid checksum errors (see one of the following steps).
    Changed:
    <
    <
    1. If you don't have the MIDAS sources yet you can either get them from the web or you check out the daq/ repository from CVS: > cvs co daq
    >
    >
    1. If you don't have the MIDAS sources yet you can either get them from the web or you check out the daq/ repository from CVS: > cvs co daq
     
    1. To compile MIDAS, ROOT must be installed first (and ROOTSYS defined). See section above to do that.
    Changed:
    <
    <
    1. Check out the mu/ repository from CVS: > cvs co mu
    2. Copy mana.c from mu/src/misc/mana.c to avoid checksum error when analyzing skimmed data.
    >
    >
    1. Check out the mu/ repository from CVS: > cvs co mu
    2. Copy mana.c from mu/misc/mana.c to avoid checksum error when analyzing skimmed data.
     
    1. Check the Makefile in daq/midas/ to use cc as compiler (not icc)
    2. Run make in daq/midas/. Before, you might need to adapt the Makefile a bit to set some library paths correctly. Especially, you probably need to change the path MYSQL_LIBS for the libmysqlclient.a to match your system. If this library is not installed, you have to install it first (on my Ubuntu laptop, I had to install the package libmysqlclient15-dev for that).
    3. Set your MIDAS paths to point to a working version
    Line: 93 to 78
     To be added:
    • decompressor.c needs md5 library: vpath needs ../src/uky and UNCOMPRESS_LIBRARY needs -L/home/winter/installation/md5-lib/ -lmd5 (on NPL the -L is needed)
    • Make sure to have the same compiler consistently in all used Makefiles (icc and icpc OR cc and gcc)
    Changed:
    <
    <
    • Adding a UIUC module
    >
    >
    • Adding a UIUC module
     

    Compiling the tree processing analysis: ./mta

    Changed:
    <
    <
    • This part of the analysis comes with the checkout of the module mu/ from CVS. To compile the ./mta, you have to go to mu/src/uiuc/macros/EventTree
    >
    >
    • This part of the analysis comes with the checkout of the module mu/ from CVS. To compile the ./mta, you have to go to mu/src/uiuc/macros/EventTree
     
    • Before you compile, it's a good idea to check the Makefile and adapt certain variables for your working environment, such as:
      • $UIUCDIR must be set to the directory of the ./mu sources, e.g. $HOME/mu/src/uiuc
      • $LIB_DIR is the directory where you compiled the ./mu executable, e.g. $HOME/mu/work
    • You should check that your version will do and execute all subroutines in the /src directory which you want for the analysis:
    Changed:
    <
    <
      • main.cpp
    >
    >
      • main.cpp
     
      • files.make
    • Compile the ./mta in mu/src/uiuc/macros/EventTree:
      > make clean
      > make
    • If it compiles, you can run the analysis on a tree output root file from the ./mu. Refer to the section below how to do that. If not here are some hints what could have gone wrong:
    Changed:
    <
    <

    >
    >
     

    Running the tree generating analysis: ./mu

    • The basic analysis should include the following input parameters:
    Line: 116 to 101
     
    • Additional common options
      • -h : for help
      • -n : 20 (do the first 20 blocks)
    Changed:
    <
    <
      • -n : 20 100 (do the first 20 midas blocks starting at block 100)
    >
    >
      • -n : 20 100 (do the first 20 midas blocks starting at block 100)
     

    Running the tree processing analysis: ./mta

    • The basic analysis should include the following input parameters:
    Line: 125 to 110
     
      • For example, from ~/mu/src/uiuc/macros/EventTree/ , run: ./mta /data/processed/tree/tree56555.root /data/processed/mta/mta56555.root

    ODB settings

    Changed:
    <
    <
    • You should use the default odb for the Illinois based analysis. It contains several parameters that are critical for detector timings. Any additional timings/parameters that are determined to be run-dependent should be added to the /Uiuc_analysis/Parameters/RunN branch of the odb
    • To get an up-to-date odb, from /mu/odb/, change to this directory and type:
      • cvs update master.odb
      • Before you can load this master.odb file, you have to check that your ODB is large enough in size. Go to your ODB directory (usually defined by $MIDAS_DIR or in $MIDAS_EXPTAB) and execute:
        • > ls -ltar
        • You should see among other lines one like:

    >
    >
    • You should use the default odb for the Illinois based analysis. It contains several parameters that are critical for detector timings. Any additional timings/parameters that are determined to be run-dependent should be added to the /Uiuc_analysis/Parameters/RunN branch of the odb
    • To get an up-to-date odb, from /mu/odb/, change to this directory and type:
      • cvs update master.odb
      • Before you can load this master.odb file, you have to check that your ODB is large enough in size. Go to your ODB directory (usually defined by $MIDAS_DIR or in $MIDAS_EXPTAB) and execute:
        • > ls -ltar
        • You should see among other lines one like:

     -rw-r--r-- 1 winter npl 30135232 Feb 8 13:55 .ODB.SHM
    Changed:
    <
    <
  • If your .ODB.SHM has a size less than the one above, you should resize the ODB. To do that, first save your current ODB:
    > odbedit
    > save current.odb
  • Then, in the ODB directory, delete all .SHM files:
    > rm -f .*.SHM
  • Create new default ODB with size specification
    > odbedit -s 30000000
  • Go back to the mu/odb directory and follow the next step
  • Now you can load the master.odb from mu/odb. Before it might be good to save your current ODB:
    > odbedit
    > save current.odb
    > load master.odb
  • * You can then optionally overload the master.odb settings with your old ODB settings > load current.odb
    >
    >
        • If your .ODB.SHM has a size less than the one above, you should resize the ODB. To do that, first save your current ODB:
          > odbedit
          > save current.odb
        • Then, in the ODB directory, delete all .SHM files:
          > rm -f .*.SHM
        • Create new default ODB with size specification
          > odbedit -s 30000000
        • Go back to the mu/odb directory and follow the next step
      • Now you can load the master.odb from mu/odb. Before it might be good to save your current ODB:
        > odbedit
        > save current.odb
        > load master.odb
    * You can then optionally overload the master.odb settings with your old ODB settings > load current.odb
     

    Using NCSA

    Changed:
    <
    <
    ======== IN PROGRESS =========
    >
    >
    ======== IN PROGRESS =========
     

    Setting up the Analysis on Abe

    Changed:
    <
    <
    ======== IN PROGRESS ========= This is still fairly preliminary so proceed with caution.
    >
    >
    ======== IN PROGRESS ========= This is still fairly preliminary so proceed with caution.
     

    Accessing Tungsten With FTP

    There was an e-mail sent to the mulan and mucap collaborations yesterday which metioned there were problems accessing Tungsten from Abe via ftp. You can see that e-mail for details, but the long and short of it is that you may use two commands to get access to Tungsten via ftp, which works around the authentication problems which were reported to administrators.
    Line: 181 to 140
     

    Root

    Changed:
    <
    <
    The second e-mail mentioned some matters concerning the various software packages we need to compile the mu and mta analysis programs. I currently have set up root v5.18 The key bit of information which is unique for Abe is that the configuration of the makefile to compile root should be set up at follows.
    ./configure linuxx8664icc --enable-shared --disable-ssl --disable-opengl --with-fftw3-libdir=$HOME/fftw-3.1.2/.libs/
    >
    >
    The second e-mail mentioned some matters concerning the various software packages we need to compile the mu and mta analysis programs. I currently have set up root v5.18 The key bit of information which is unique for Abe is that the configuration of the makefile to compile root should be set up at follows.
    ./configure linuxx8664icc --enable-shared --disable-ssl --disable-opengl --with-fftw3-libdir=$HOME/fftw-3.1.2/.libs/
     make
    Changed:
    <
    <
    As I mentioned in the e-mail the open GL package is disabled because the version of it which is available to users was not compiled with the appropriate "-fPIC" option, and the compilation hence fails. This configuration and subsequent compilation worked. These steps together with setting the usual $ROOTSYS variable in the .soft file, as described above, produced a working version of root. The source code is available in my directory /u/ac/sknaack/Installs/*, and the v5.20 .tar file is also already present on Peter's account if you are looking for v5.20
    >
    >
    As I mentioned in the e-mail the open GL package is disabled because the version of it which is available to users was not compiled with the appropriate "-fPIC" option, and the compilation hence fails. This configuration and subsequent compilation worked. These steps together with setting the usual $ROOTSYS variable in the .soft file, as described above, produced a working version of root. The source code is available in my directory /u/ac/sknaack/Installs/*, and the v5.20 .tar file is also already present on Peter's account if you are looking for v5.20
     

    Midas

    Midas is in /u/ac/sknaack/mucap/daq/midas/ and /u/ac/sknaack/Installs/midas-2.0.0 in my home directory.
    Line: 194 to 150
     Midas's relationship with mu can be tempestuous. It's not always immediate to compile and set up midas and get it working with our analysis. I have compiled midas with cc and g++ using our customize mana.c* file both with the version of midas in the daq CVS directory, and separately I compiled Midas 2.0 from the source files on the PSI midas website, midas.psi.ch. They both compile now with fairly little complaint or alteration to the Makefiles. To be sure you can always check how things are set up in these Makefiles if you find you have difficulty compiling Midas.

    Some pointers.

    Changed:
    <
    <
    • To include mysql and SSL in the midas compilation there is one place in the Makefile where "-lssl" should be added to be consistent such as at the following.
    >
    >
    • To include mysql and SSL in the midas compilation there is one place in the Makefile where "-lssl" should be added to be consistent such as at the following.
     ifdef NEED_MYSQL CFLAGS += -DHAVE_MYSQL LIBS += -lssl $(MYSQL_LIBS) -lz endif
    Changed:
    <
    <
    and
    >
    >
    and
     ifdef NEED_LIBROOTA ROOTLIBS := $(ROOTSYS)/lib/libRoot.a -lssl -ldl -lcrypt ROOTGLIBS := $(ROOTLIBS) -lfreetype endif
    Changed:
    <
    <
  • The usual variables need to be defined in the .soft file as usual, which is described above. Remember the library and binary files will be created in a directory "linux64", not linux
  • >
    >
    • The usual variables need to be defined in the .soft file as usual, which is described above. Remember the library and binary files will be created in a directory "linux64", not linux
     MIDAS_ROOT=/$HOME/daq/midas/) MIDAS_HOME = $MIDAS_ROOT LD_LIBRARY_PATH += $MIDAS_ROOT/linux64/lib PATH += $MIDAS_ROOT/linux64/bin
    Changed:
    <
    <
  • Make sure that the rmana.o file is created in the linux64/lib directory. This is important to set up mu.
  • The usual care should be taken in setting up the odb to ensure that the .*.SHM files are not corrupted and that all of the information in the master.odb is not corrupted. This is described above in some detail
  • >
    >
    • Make sure that the rmana.o file is created in the linux64/lib directory. This is important to set up mu.
    • The usual care should be taken in setting up the odb to ensure that the .*.SHM files are not corrupted and that all of the information in the master.odb is not corrupted. This is described above in some detail
     

    The Java Packages for MQL

    Changed:
    <
    <
    Two packages are necessary to compile MQL. They are both already present on Abe, and can be accessed by adding the following lines to your .soft file.
    >
    >
    Two packages are necessary to compile MQL. They are both already present on Abe, and can be accessed by adding the following lines to your .soft file.
     JAVA_HOME = /usr/local/j2sdk1.4.2_14/jre/bin/java PATH += $JAVA_HOME/bin

    JJTREE = /u/ac/pwinter/installation/javacc-4.0 PATH += $JJTREE/bin

    Changed:
    <
    <
    Yes, the second is in Peter's directory already. You can of course copy it and set it up analogously in your own directory, changing the .soft settings appropriately.
    >
    >
    Yes, the second is in Peter's directory already. You can of course copy it and set it up analogously in your own directory, changing the .soft settings appropriately.
     

    MD5

    Can be found in /u/ac/sknaack/Install/md5-lib. Also in Peter Winter's installations directory. It's important that this package is adequately identified in the mu make file in two places. In my Makefile look for the UNCOMPRESS_LIB and MIDAS_CFLAGS variables to make sure these are consistent with your exact setup.

    Compiling MQL and Compress

    Line: 244 to 186
     The analysis compression library is compiled in my directory with icc and icpc, and does the usual trick of making an error, but the needed libmucap_compress.a library does appear. This has been seen before.

    Compiling mu and libMucapEvent.so

    Mu Makefile changes in mu/work:
    Changed:
    <
    <
    • CC = icc and CXX = icpc in my current setup.
    • CXXFLAGS; add the -fPIC option, at the end
    • make sure the X11LIBS variable is set as
      X11LIBS = -L/usr/X11/lib64 -lX11
    >
    >
    • CC = icc and CXX = icpc in my current setup.
    • CXXFLAGS; add the -fPIC option, at the end
    • make sure the X11LIBS variable is set as
      X11LIBS = -L/usr/X11/lib64 -lX11
     This has thus far produced a working mu and libMucapEvent.so for me.

    Compiling mta

    Changed:
    <
    <
    • CC = icc and CXX = icpc in my current setup.
    • CXXFLAGS; add the -fPIC option, at the front.
    • make sure the X11LIBS variable is set as
      X11LIBS = -L/usr/X11/lib64 -lX11
    >
    >
    • CC = icc and CXX = icpc in my current setup.
    • CXXFLAGS; add the -fPIC option, at the front.
    • make sure the X11LIBS variable is set as
      X11LIBS = -L/usr/X11/lib64 -lX11
     I will warn that as of today the mta is not fully vetted. But it did compile and didn't segfault. The warning sign is that the size of the mta output file was about half of the expected size. (22 MB vs. 50 MB) \ No newline at end of file
     
    This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
    Ideas, requests, problems regarding TWiki? Send feedback