, view all tags
-- PeterWinter - 18 Dec 2007 This page will describe in the future all necessary steps, to setup the various analysis tools. -- BrendanKiburg - 18 Dec 2007

-- PeterWinter - 09 Jan 2008

Setting your environment variables

In the following sections, you will often have to set some environment variables and/or change the PATH. Depending on the system you have to do this in different locations:
  • On your own cluster, PC, laptop etc. you can set them in your ~/.bashrc (or equivalent like ~/.tcshrc). To effect the changes execute

    > source ~/.bashrc
    Please refer to your shell manual for details.
  • On NCSA, you edit the ~/.soft file and execute

    > resoft 
    Please visit the softEnv manual at NCSA for more details.

Have Root installed:

If your system has Root installed, go on to the next section. If not (like on tungsten.ncsa.uiuc.edu), then install it locally:
  1. Downloaded source tar for root versions 5.08 from root.cern.ch
  2. tar -xzvf root_v5.08.00.source.tar.gz
  3. cd root
  4. > ./configure linux --enable-shared --enable-thread --disable-ssl
    > make
    > make install
  5. Set $ROOTSYS to your current working version of Root (e.g. ROOTSYS = /u/ac/bkiburg/root)
  6. Add the Root libraries to LD_LIBRARY_PATH: LD_LIBRARY_PATH += $ROOTSYS/lib
  7. Add Root to your path: PATH += $ROOTSYS/bin

This can easily take several hours to build depending on your system, so go grab a coffee. If you are trying to install versions later than 5.08.00, you may have difficulty with the --enable-shared and --enable-thread options.

Java 2 SDK:

  1. Copy j2sdk-1_4_2_09-linux-i586.bin to home dir from /u/ac/bkiburg/tungsten/installs/
  2. Run j2sdk-1_4_2_09-linux-i586.bin file in the directory where you would like j2sdk installed.
  3. Set JAVA_HOME = $HOME/j2sdk.4.2_09 and add the java executables to your path: PATH += $JAVA_HOME/bin

On your own cluster, PC, laptop etc. install J2SDK:

  • You can download it from the web or ask your administrator to do so if you don't have permissions to install that.
  • Set the JAVA_HOME variable to the path where the J2SDK package was installed (e.g. JAVA_HOME = $HOME/j2sdk.4.2_09) and add the java executables to your path: PATH += $JAVA_HOME/bin

JavaCC (for mql compilation later):

  • If JavaCC is installed on your system, skip this and the next step and just set the variable in the last step. Otherwise, download it (try this java link). On NCSA you can also directly copy the javacc-4.0.tar.gz file from ~bkiburg/tungsten/installs/ or ~pwinter/installation.
  • Untar (tar -zxvf javacc-4.0.tar.gz) the file in the directory where you like to have JavaCC installed.
  • Set the JJTREE to where JavaCC is installed (e.g. JJTREE_HOME = $HOME/javacc-4.0) and add the binaries to your path: PATH += $JJTREE_HOME/bin

CVS software

  • For the MIDAS and analysis software installation, you will have to get access to the CVS to retrieve the source code. Please check the Instructions to get this access.
  • Once you have that you can browse the CVS repository to see what we have in CVS. Use your NPL username and password to login.


Even if you have MIDAS already installe on your system, you should probably recompile it with the mana.c from the mu/misc/ to avoid checksum errors (see one of the following steps).

  1. If you don't have the MIDAS sources yet you can either get them from the web or you check out the daq/ repository from CVS: > cvs co daq
  2. To compile MIDAS, ROOT must be installed first (and ROOTSYS defined). See section above to do that.
  3. Check out the mu/ repository from CVS: > cvs co mu
  4. NOTE mana.cpp is now up to date in the CVS verson of MIDAS. No extra copying is needed **OBSOLETE:Copy mana.c from mu/misc/mana.c to avoid checksum error when analyzing skimmed data.**OBSOLETE
  5. Check the Makefile in daq/midas/ to use cc as compiler (not icc)
  6. Run make in daq/midas/. Before, you might need to adapt the Makefile a bit to set some library paths correctly. Especially, you probably need to change the path MYSQL_LIBS for the libmysqlclient.a to match your system. If this library is not installed, you have to install it first (on my Ubuntu laptop, I had to install the package libmysqlclient15-dev for that).
  7. Set your MIDAS paths to point to a working version
    • MIDAS_ROOT has to link to the directory where you just compiled MIDAS (e.g. MIDAS_ROOT=/$HOME/daq/midas/)
    • LD_LIBRARY_PATH += $MIDAS_ROOT/linux/lib
    • PATH += $MIDAS_ROOT/linux/bin
  8. Set your MIDAS_DIR for a place you want associated MIDAS files (SHMS) to be located (e.g. MIDAS_DIR=$HOME/mucap/mu/work)

Compiling the tree generating analysis: ./mu

  1. You need the mu/ directory from CVS, as well as MIDAS installed
  2. First compile the mql part in the mu repository by:
    > cd mu/mql
    > mv JavaCharStream.java JavaCharStream.obsolete
    > make clean
    > make
  3. Compile the compress part in the mu repository by:
    > cd mu/compress
    > make clean
    > make
    This might actually end up with some error messages (depending on your system) but normally that's not a problem. It's just important that the libmucap_compress.a was generated.
  4. Although it's up to you, it's a usual habit that we compile the tree generating software ./mu in the mu/work directory. Here you will also have the Makefile to do so. To compile go to this directory (e.g. >cd $HOME/mu/work) and run
    > make
  5. If you're lucky, you'll end up without an error and the compiled executable ./mu as well as the library libMucapEvent.so. If you end up with error message, please have a look at the ideas at the end of this section.
  6. Add the path to the compiled libMucapEvent.so to your LD_LIBRARY_PATH (e.g. LD_LIBRARY_PATH+=$HOME/mu/work)
  7. If you want to add a new MIDAS module to the software, you have to add it to the mu/work/MODULES file. Only modules that are in this file will be compiled into ./mu and hence only they can be ran later on. Since you can later on switch modules individually on and off in the ODB (see below) it doesn't harm to have more modules than necessary in here. Any changes to this file require to recompile (i.e. > make clean; make) before it affects ./mu.

You were unlucky and ./mu didn't compile right away. Here are some ideas:

  • Check that the different include and library paths in the mu/work/Makefile are correct. Also check that the compiler is the one you have on your system (cc, gcc, g++, icc etc.)
  • Double check that all the environment variables in the above steps were set correctly
  • If you have a code error and can't figure out what's giong on, contact UIUC for help
  • ....
To be added:
  • decompressor.c needs md5 library: vpath needs ../src/uky and UNCOMPRESS_LIBRARY needs -L/home/winter/installation/md5-lib/ -lmd5 (on NPL the -L is needed)
  • Make sure to have the same compiler consistently in all used Makefiles (icc and icpc OR cc and gcc)
  • Adding a UIUC module

Compiling the tree processing analysis: ./mta

  • This part of the analysis comes with the checkout of the module mu/ from CVS. To compile the ./mta, you have to go to mu/src/uiuc/macros/EventTree
  • Before you compile, it's a good idea to check the Makefile and adapt certain variables for your working environment, such as:
    • $UIUCDIR must be set to the directory of the ./mu sources, e.g. $HOME/mu/src/uiuc
    • $LIB_DIR is the directory where you compiled the ./mu executable, e.g. $HOME/mu/work
  • You should check that your version will do and execute all subroutines in the /src directory which you want for the analysis:
    • main.cpp
    • files.make
  • Compile the ./mta in mu/src/uiuc/macros/EventTree:
    > make clean
    > make
  • If it compiles, you can run the analysis on a tree output root file from the ./mu. Refer to the section below how to do that. If not here are some hints what could have gone wrong:

Running the tree generating analysis: ./mu

  • The basic analysis should include the following input parameters:
    • -i input_filename.mid
    • -o output_filename.root (histograms from the raw analysis)
    • -T tree_output_filename.root (mucap tree for post-processing)
    • For example, from ~/mu/work/ , run : ./mu -i /data/run56555.mid -o /data/processed/ana/ana56555.root -T /data/processed/tree/tree56555.root -n 100
  • Additional common options
    • -h : for help
    • -n : 20 (do the first 20 blocks)
    • -n : 20 100 (do the first 20 midas blocks starting at block 100)

Running the tree processing analysis: ./mta

  • The basic analysis should include the following input parameters:
    • input_filename
    • output_filename
    • For example, from ~/mu/src/uiuc/macros/EventTree/ , run: ./mta /data/processed/tree/tree56555.root /data/processed/mta/mta56555.root

ODB settings

  • You should use the default odb for the Illinois based analysis. It contains several parameters that are critical for detector timings. Any additional timings/parameters that are determined to be run-dependent should be added to the /Uiuc_analysis/Parameters/RunN branch of the odb
  • To get an up-to-date odb, from /mu/odb/, change to this directory and type:
    • cvs update master.odb
    • Before you can load this master.odb file, you have to check that your ODB is large enough in size. Go to your ODB directory (usually defined by $MIDAS_DIR or in $MIDAS_EXPTAB) and execute:
      • > ls -ltar
      • You should see among other lines one like:

        -rw-r--r-- 1 winter npl 30135232 Feb  8 13:55 .ODB.SHM
      • If your .ODB.SHM has a size less than the one above, you should resize the ODB. To do that, first save your current ODB:
        > odbedit
        > save current.odb
      • Then, in the ODB directory, delete all .SHM files:
        > rm -f .*.SHM
      • Create new default ODB with size specification
        > odbedit -s 30000000
      • Go back to the mu/odb directory and follow the next step
    • Now you can load the master.odb from mu/odb. Before it might be good to save your current ODB:
      > odbedit
      > save current.odb
      > load master.odb
* You can then optionally overload the master.odb settings with your old ODB settings > load current.odb

Using NCSA

======== IN PROGRESS =========

Setting up the Analysis on Abe

======== IN PROGRESS ========= This is still fairly preliminary so proceed with caution.

Accessing Tungsten With FTP

There was an e-mail sent to the mulan and mucap collaborations yesterday which metioned there were problems accessing Tungsten from Abe via ftp. You can see that e-mail for details, but the long and short of it is that you may use two commands to get access to Tungsten via ftp, which works around the authentication problems which were reported to administrators.

myproxy-logon -s myproxy.ncsa.uiuc.edu
[when prompted enter you NCSA kerberos password]
(this is the same password you use to login to Abe) 
uberftp tund.ncsa.uiuc.edu


The second e-mail mentioned some matters concerning the various software packages we need to compile the mu and mta analysis programs. I currently have set up root v5.18 The key bit of information which is unique for Abe is that the configuration of the makefile to compile root should be set up at follows.
./configure linuxx8664icc --enable-shared --disable-ssl --disable-opengl --with-fftw3-libdir=$HOME/fftw-3.1.2/.libs/
As I mentioned in the e-mail the open GL package is disabled because the version of it which is available to users was not compiled with the appropriate "-fPIC" option, and the compilation hence fails. This configuration and subsequent compilation worked. These steps together with setting the usual $ROOTSYS variable in the .soft file, as described above, produced a working version of root. The source code is available in my directory /u/ac/sknaack/Installs/*, and the v5.20 .tar file is also already present on Peter's account if you are looking for v5.20


Midas is in /u/ac/sknaack/mucap/daq/midas/ and /u/ac/sknaack/Installs/midas-2.0.0 in my home directory.

Midas's relationship with mu can be tempestuous. It's not always immediate to compile and set up midas and get it working with our analysis. I have compiled midas with cc and g++ using our customize mana.c* file both with the version of midas in the daq CVS directory, and separately I compiled Midas 2.0 from the source files on the PSI midas website, midas.psi.ch. They both compile now with fairly little complaint or alteration to the Makefiles. To be sure you can always check how things are set up in these Makefiles if you find you have difficulty compiling Midas.

Some pointers.

  • To include mysql and SSL in the midas compilation there is one place in the Makefile where "-lssl" should be added to be consistent such as at the following.
    ifdef NEED_MYSQL
    LIBS       += -lssl $(MYSQL_LIBS) -lz
    ROOTLIBS    := $(ROOTSYS)/lib/libRoot.a -lssl -ldl -lcrypt
    ROOTGLIBS   := $(ROOTLIBS) -lfreetype
  • The usual variables need to be defined in the .soft file as usual, which is described above. Remember the library and binary files will be created in a directory "linux64", not linux
    LD_LIBRARY_PATH += $MIDAS_ROOT/linux64/lib
    PATH += $MIDAS_ROOT/linux64/bin
  • Make sure that the rmana.o file is created in the linux64/lib directory. This is important to set up mu.
  • The usual care should be taken in setting up the odb to ensure that the .*.SHM files are not corrupted and that all of the information in the master.odb is not corrupted. This is described above in some detail

The Java Packages for MQL

Two packages are necessary to compile MQL. They are both already present on Abe, and can be accessed by adding the following lines to your .soft file.
JAVA_HOME = /usr/local/j2sdk1.4.2_14/jre/bin/java

JJTREE = /u/ac/pwinter/installation/javacc-4.0
Yes, the second is in Peter's directory already. You can of course copy it and set it up analogously in your own directory, changing the .soft settings appropriately.


Can be found in /u/ac/sknaack/Install/md5-lib. Also in Peter Winter's installations directory. It's important that this package is adequately identified in the mu make file in two places. In my Makefile look for the UNCOMPRESS_LIB and MIDAS_CFLAGS variables to make sure these are consistent with your exact setup.

Compiling MQL and Compress

MQL compiles exactly as before, which is described above.

The analysis compression library is compiled in my directory with icc and icpc, and does the usual trick of making an error, but the needed libmucap_compress.a library does appear. This has been seen before.

Compiling mu and libMucapEvent.so

Mu Makefile changes in mu/work:
  • CC = icc and CXX = icpc in my current setup.
  • CXXFLAGS; add the -fPIC option, at the end
  • make sure the X11LIBS variable is set as
    X11LIBS = -L/usr/X11/lib64 -lX11
This has thus far produced a working mu and libMucapEvent.so for me.

Compiling mta

  • CC = icc and CXX = icpc in my current setup.
  • CXXFLAGS; add the -fPIC option, at the front.
  • make sure the X11LIBS variable is set as
    X11LIBS = -L/usr/X11/lib64 -lX11
I will warn that as of today the mta is not fully vetted. But it did compile and didn't segfault. The warning sign is that the size of the mta output file was about half of the expected size. (22 MB vs. 50 MB)
Edit | Attach | Watch | Print version | History: r25 < r24 < r23 < r22 < r21 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r23 - 2009-06-17 - BrendanKiburg
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback