Difference: DatabaseInformation (1 vs. 11)

Revision 112011-04-21 - PeterWinter

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
Changed:
<
<
-- BrendanKiburg - 28 Aug 2006

Database Location, Access and Structure

Primary database at dolphin

The mysql database is physically located on dolphin . The trendplots are generated by connecting to cheetah:80/mucap and running one of the scripts. The scripts are located /home/data/www/root/trend/ . The perl scripts that are used to query the database so that the root files can plot the data are in /home/data/www/cgi-bin .

Online Analysis

During the run, the UCB analysis ran on a small fraction of the data and generated statistics that were used in the trendplots. These root files were stored at The online analyzer stored the data on chetah in the directory /home/data/online-hists/ in files with the format his#####.root . This files can be opened with the online-display program which preselects some of the histograms to display.

UCB histograms

The results of the UCB online analysis are a set of histograms that contain the capture yield information (info provided by Tom):
  • "capture_search_mustop_precount" : number of mustops to be inspected
  • "capture_cand_count" : number of capture candidates with at least an EH pixel, from rightward search (time after the mustop)
  • "capture_control_cand_count" : number of capture control events with at least an EH pixel, from leftward search (time before the mustop)
  • "capture_cand_count_EVH" : number of capture candidates with an EVH pixel, from rightward search (time after the mustop)
  • "capture_control_cand_count_EVH" : number of capture control events with an EVH pixel, from leftward search (time before the mustop)

Yield is calculated : Y = (capture_cand_count_EVH - capture_control_cand_count_EVH) / (capture_search_mustop_precount)

Offline Analysis

As soon as MIDAS stopped a run and moved on to the next, it stopped the online-analyzer and skipped to the next (current) file. The data files were then transferred to the big cats (puma and lion) for offline analysis. Here they were analyzed by the UIUC analysis software.

The following directories are available On puma ( the 2nd disk became unstable, so /local/2/ was copied to /local/1/local2/):

  • /local/1/ak3_UIUC/
  • /local/1/muminus/
  • /local/1/muplus/
  • /local/1/local2/muminus/
  • /local/1/local2/muplus/
  • /local/1/local2/neutron/

On lion :

  • /local/1/ak3_UIUC/
  • /local/1/muminus/
  • /local/1/muplus/
  • /local/1/kicker_study_runs/

Within each of the directories are the subdirectories ana , mta and tree .

Copy of database at twiki.npl.uiuc.edu.

This copy was generated at dolphin on 9/11/2006 and imported to the uiuc server. Commands used:

  dolphin:
     mysqldump -u mucap -p  --databases mucap > peter/mucap.sql

  one:
    password change to xxx
    one,~>mysqladmin -h twiki.npl.uiuc.edu -u mucap -p password
    Enter password:
    one,~> mysql -h twiki.npl.uiuc.edu -u mucap -p mucap < /data/26/kammel/mucap.sql
    Enter password:
    one,~> mysql -h twiki.npl.uiuc.edu -u mucap -p
    Enter password:
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    Your MySQL connection id is 18 to server version: 4.1.11-Debian_4sarge7-log

    mysql> show databases;
+----------+
| Database |
+----------+
| mucap    |
+----------+
1 row in set (0.00 sec)

mysql> use mucap;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+---------------------+
| Tables_in_mucap     |
+---------------------+
| Beamline            |
| BeamlineElements    |
| cap_chup            |
|

Today I added the run8_chups database to this server on twiki. I have not yet imported it on dolphin, but that can be done trivially if necessary. -- BrendanKiburg - 28 Sep 2006

Scripts for Accessing & Searching the Database

I added a new table to the mucap database on dolphin that contains information on CHUPS during run10. Peter Kr. generated it with a spacing of 1 minute intervals. It was created by using the command: "CREATE TABLE run10_chups(Date DATETIME, PT1 FLOAT, HS FLOAT, MFC4Flow FLOAT, MFC5Flow FLOAT, AVGFlow FLOAT, TT5 FLOAT, TT6 FLOAT);" The relevant fields are as follows:
  • Date in the format yyyy-mm-dd hh:mm:ss
  • PT1 : pressure sensor
  • HS : humidity sensor
  • MFC[4-5]Flow : mass flow controller readbacks
  • AVGFlow : average flow in through the tpc/chups system
  • TT[5-6] : temperature sensors

I filled in the data as prescribed by Peter in the RunOverview Chups section.

General Database info

  • Steve's MySQL note

Accessing the Database via mulan method

Volodya upgraded the way that mulan accesses the mysql database and generates plots for the trendplot page. His information about how to do it is included here: 1. Connect to MySQL server:

TSQLServer *db = TSQLServer::Connect("mysql://be01/online","username","password");

2. Query:

TSQLResult *res = db->Query("select run,tau,N,B from Runlog where N/B>650");

3. Process query.

/*** number of rows ***/ int nrows = res->GetRowCount();

/*** number of fields, if you need it ***/ int nfields = res->GetFieldCount();

Normally, I declare a structure with relavant fields. I found it convinient, but you can do it in a different way, as you like. For example:

struct s_query { int run_nr; float tau; float N; float B; };

/*** allocate array which will contain results ***/ s_query *query = new s_query[nrows];

/*** iterate over rows ***/ for( int i=0; i<nrows; i++ ) {

TSQLRow *row = res->Next();

query[i].run_nr = atoi(row->GetField(0)); query[i].tau = atof(row->GetField(1)); query[i].N = atof(row->GetField(2)); query[i].B = atof(row->GetField(3));

/*** you should delete TSQLRow at the end ***/ delete row; }

Thus, "query" contains now data fetched from MySQL database and you can use it in your macro as you like.

4. Don't forget to delete at the end:

delete res; delete db; delete query;

Some working examples you can find on mulanbe:DAQ/www/online/trend_plots. The ouput you can see on our online page if you connect to mulanbe: http://mulanbe.psi.ch/online, look at "Trend Plots". I am sure you know how to setup ssh tunnel to connect to mulanbe.

Accessing the Trendplots [Old method -- stopped Sep 8th, 2006]

On cheetah:80/mucap , we have the trendplot page. I made the following changes to files in order to get the new trendplots to appear here as well:

  • Add link to /home/data/www/html/loi.html for "Brendan's Test Area" which calls the following script : /cgi-bin/mucap/show_trend_slow.cgi?impurity+all
  • Add option to /home/data/www/cgi-bin/show_trend_slow.cgi for there to be a plot called impurity
  • Added condition to only show run boxes for the "impurity" plots, not all show_trend_slow plots
  • Created new file /home/data/www/root/trend/impurity.cxx to be executed when "Brendan's Test Area" is clicked on. This is how I will generate a separate set of requests to the db. For now I have copied the stats.cxx and made the appropriate name changes (stats-->impurity)
  • Temporarily renamed the set "stats" --> "impurity" in the file /home/data/www/root/trend/make_dat_files.pl
>
>
Please go to https://muon.npl.washington.edu/twiki/bin/view/Main/DatabaseInformation
 \ No newline at end of file

Revision 102006-09-28 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Changed:
<
<
>
>
 

Database Location, Access and Structure

Primary database at dolphin

Line: 85 to 85
 
Added:
>
>
Today I added the run8_chups database to this server on twiki. I have not yet imported it on dolphin, but that can be done trivially if necessary. -- BrendanKiburg - 28 Sep 2006
 

Scripts for Accessing & Searching the Database

I added a new table to the mucap database on dolphin that contains information on CHUPS during run10. Peter Kr. generated it with a spacing of 1 minute intervals. It was created by using the command: "CREATE TABLE run10_chups(Date DATETIME, PT1 FLOAT, HS FLOAT, MFC4Flow FLOAT, MFC5Flow FLOAT, AVGFlow FLOAT, TT5 FLOAT, TT6 FLOAT);"

Revision 92006-09-12 - PeterKammel

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Line: 51 to 51
  mysqldump -u mucap -p --databases mucap > peter/mucap.sql

one:

Added:
>
>
password change to xxx one,~>mysqladmin -h twiki.npl.uiuc.edu -u mucap -p password Enter password:
  one,~> mysql -h twiki.npl.uiuc.edu -u mucap -p mucap < /data/26/kammel/mucap.sql Enter password: one,~> mysql -h twiki.npl.uiuc.edu -u mucap -p

Revision 82006-09-11 - PeterKammel

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006

Database Location, Access and Structure

Added:
>
>

Primary database at dolphin

 The mysql database is physically located on dolphin . The trendplots are generated by connecting to cheetah:80/mucap and running one of the scripts. The scripts are located /home/data/www/root/trend/ . The perl scripts that are used to query the database so that the root files can plot the data are in /home/data/www/cgi-bin .
Changed:
<
<

Online Analysis

>
>

Online Analysis

 During the run, the UCB analysis ran on a small fraction of the data and generated statistics that were used in the trendplots. These root files were stored at The online analyzer stored the data on chetah in the directory /home/data/online-hists/ in files with the format his#####.root . This files can be opened with the online-display program which preselects some of the histograms to display.
Changed:
<
<

UCB histograms

>
>

UCB histograms

  The results of the UCB online analysis are a set of histograms that contain the capture yield information (info provided by Tom):
  • "capture_search_mustop_precount" : number of mustops to be inspected
  • "capture_cand_count" : number of capture candidates with at least an EH pixel, from rightward search (time after the mustop)
Line: 20 to 22
  Yield is calculated : Y = (capture_cand_count_EVH - capture_control_cand_count_EVH) / (capture_search_mustop_precount)
Changed:
<
<

Offline Analysis

>
>

Offline Analysis

 As soon as MIDAS stopped a run and moved on to the next, it stopped the online-analyzer and skipped to the next (current) file. The data files were then transferred to the big cats (puma and lion) for offline analysis. Here they were analyzed by the UIUC analysis software.

The following directories are available

Line: 40 to 42
  Within each of the directories are the subdirectories ana , mta and tree .
Added:
>
>

Copy of database at twiki.npl.uiuc.edu.

This copy was generated at dolphin on 9/11/2006 and imported to the uiuc server. Commands used:

  dolphin:
     mysqldump -u mucap -p  --databases mucap > peter/mucap.sql

  one:
    one,~> mysql -h twiki.npl.uiuc.edu -u mucap -p mucap < /data/26/kammel/mucap.sql
    Enter password:
    one,~> mysql -h twiki.npl.uiuc.edu -u mucap -p
    Enter password:
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    Your MySQL connection id is 18 to server version: 4.1.11-Debian_4sarge7-log

    mysql> show databases;
+----------+
| Database |
+----------+
| mucap    |
+----------+
1 row in set (0.00 sec)

mysql> use mucap;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+---------------------+
| Tables_in_mucap     |
+---------------------+
| Beamline            |
| BeamlineElements    |
| cap_chup            |
|

 

Scripts for Accessing & Searching the Database

I added a new table to the mucap database on dolphin that contains information on CHUPS during run10. Peter Kr. generated it with a spacing of 1 minute intervals. It was created by using the command:

Revision 72006-09-08 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Line: 57 to 57
  General Database info
  • Steve's MySQL note
Changed:
<
<

Accessing the Trendplots

>
>

Accessing the Database via mulan method

Volodya upgraded the way that mulan accesses the mysql database and generates plots for the trendplot page. His information about how to do it is included here: 1. Connect to MySQL server:

TSQLServer *db = TSQLServer::Connect("mysql://be01/online","username","password");

2. Query:

TSQLResult *res = db->Query("select run,tau,N,B from Runlog where N/B>650");

3. Process query.

/*** number of rows ***/ int nrows = res->GetRowCount();

/*** number of fields, if you need it ***/ int nfields = res->GetFieldCount();

Normally, I declare a structure with relavant fields. I found it convinient, but you can do it in a different way, as you like. For example:

struct s_query { int run_nr; float tau; float N; float B; };

/*** allocate array which will contain results ***/ s_query *query = new s_query[nrows];

/*** iterate over rows ***/ for( int i=0; i<nrows; i++ ) {

TSQLRow *row = res->Next();

query[i].run_nr = atoi(row->GetField(0)); query[i].tau = atof(row->GetField(1)); query[i].N = atof(row->GetField(2)); query[i].B = atof(row->GetField(3));

/*** you should delete TSQLRow at the end ***/ delete row; }

Thus, "query" contains now data fetched from MySQL database and you can use it in your macro as you like.

4. Don't forget to delete at the end:

delete res; delete db; delete query;

Some working examples you can find on mulanbe:DAQ/www/online/trend_plots. The ouput you can see on our online page if you connect to mulanbe: http://mulanbe.psi.ch/online, look at "Trend Plots". I am sure you know how to setup ssh tunnel to connect to mulanbe.

Accessing the Trendplots [Old method -- stopped Sep 8th, 2006]

 On cheetah:80/mucap , we have the trendplot page. I made the following changes to files in order to get the new trendplots to appear here as well:

  • Add link to /home/data/www/html/loi.html for "Brendan's Test Area" which calls the following script : /cgi-bin/mucap/show_trend_slow.cgi?impurity+all

Revision 62006-09-07 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Line: 52 to 52
 
  • AVGFlow : average flow in through the tpc/chups system
  • TT[5-6] : temperature sensors

Added:
>
>
I filled in the data as prescribed by Peter in the RunOverview Chups section.

  General Database info
  • Steve's MySQL note
Changed:
<
<

>
>

Accessing the Trendplots

On cheetah:80/mucap , we have the trendplot page. I made the following changes to files in order to get the new trendplots to appear here as well:

  • Add link to /home/data/www/html/loi.html for "Brendan's Test Area" which calls the following script : /cgi-bin/mucap/show_trend_slow.cgi?impurity+all
  • Add option to /home/data/www/cgi-bin/show_trend_slow.cgi for there to be a plot called impurity
  • Added condition to only show run boxes for the "impurity" plots, not all show_trend_slow plots
  • Created new file /home/data/www/root/trend/impurity.cxx to be executed when "Brendan's Test Area" is clicked on. This is how I will generate a separate set of requests to the db. For now I have copied the stats.cxx and made the appropriate name changes (stats-->impurity)
  • Temporarily renamed the set "stats" --> "impurity" in the file /home/data/www/root/trend/make_dat_files.pl

Revision 52006-09-07 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Line: 42 to 42
 

Scripts for Accessing & Searching the Database

Added:
>
>
I added a new table to the mucap database on dolphin that contains information on CHUPS during run10. Peter Kr. generated it with a spacing of 1 minute intervals. It was created by using the command: "CREATE TABLE run10_chups(Date DATETIME, PT1 FLOAT, HS FLOAT, MFC4Flow FLOAT, MFC5Flow FLOAT, AVGFlow FLOAT, TT5 FLOAT, TT6 FLOAT);" The relevant fields are as follows:
  • Date in the format yyyy-mm-dd hh:mm:ss
  • PT1 : pressure sensor
  • HS : humidity sensor
  • MFC[4-5]Flow : mass flow controller readbacks
  • AVGFlow : average flow in through the tpc/chups system
  • TT[5-6] : temperature sensors
 
Added:
>
>
General Database info
 
  • Steve's MySQL note

Revision 42006-09-02 - PeterKammel

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Line: 43 to 43
 

Scripts for Accessing & Searching the Database

Added:
>
>
  • Steve's MySQL note
 

\ No newline at end of file

Revision 32006-08-30 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006
Line: 10 to 10
 During the run, the UCB analysis ran on a small fraction of the data and generated statistics that were used in the trendplots. These root files were stored at The online analyzer stored the data on chetah in the directory /home/data/online-hists/ in files with the format his#####.root . This files can be opened with the online-display program which preselects some of the histograms to display.
Added:
>
>

UCB histograms

The results of the UCB online analysis are a set of histograms that contain the capture yield information (info provided by Tom):
  • "capture_search_mustop_precount" : number of mustops to be inspected
  • "capture_cand_count" : number of capture candidates with at least an EH pixel, from rightward search (time after the mustop)
  • "capture_control_cand_count" : number of capture control events with at least an EH pixel, from leftward search (time before the mustop)
  • "capture_cand_count_EVH" : number of capture candidates with an EVH pixel, from rightward search (time after the mustop)
  • "capture_control_cand_count_EVH" : number of capture control events with an EVH pixel, from leftward search (time before the mustop)

Yield is calculated : Y = (capture_cand_count_EVH - capture_control_cand_count_EVH) / (capture_search_mustop_precount)

 

Offline Analysis

As soon as MIDAS stopped a run and moved on to the next, it stopped the online-analyzer and skipped to the next (current) file. The data files were then transferred to the big cats (puma and lion) for offline analysis. Here they were analyzed by the UIUC analysis software.

Revision 22006-08-28 - BrendanKiburg

Line: 1 to 1
 
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006

Database Location, Access and Structure

Added:
>
>
The mysql database is physically located on dolphin . The trendplots are generated by connecting to cheetah:80/mucap and running one of the scripts. The scripts are located /home/data/www/root/trend/ . The perl scripts that are used to query the database so that the root files can plot the data are in /home/data/www/cgi-bin .

Online Analysis

During the run, the UCB analysis ran on a small fraction of the data and generated statistics that were used in the trendplots. These root files were stored at The online analyzer stored the data on chetah in the directory /home/data/online-hists/ in files with the format his#####.root . This files can be opened with the online-display program which preselects some of the histograms to display.

Offline Analysis

As soon as MIDAS stopped a run and moved on to the next, it stopped the online-analyzer and skipped to the next (current) file. The data files were then transferred to the big cats (puma and lion) for offline analysis. Here they were analyzed by the UIUC analysis software.

The following directories are available On puma ( the 2nd disk became unstable, so /local/2/ was copied to /local/1/local2/):

  • /local/1/ak3_UIUC/
  • /local/1/muminus/
  • /local/1/muplus/
  • /local/1/local2/muminus/
  • /local/1/local2/muplus/
  • /local/1/local2/neutron/

On lion :

  • /local/1/ak3_UIUC/
  • /local/1/muminus/
  • /local/1/muplus/
  • /local/1/kicker_study_runs/

Within each of the directories are the subdirectories ana , mta and tree .

 

Scripts for Accessing & Searching the Database

\ No newline at end of file

Revision 12006-08-28 - BrendanKiburg

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="AnalysisInformation"
-- BrendanKiburg - 28 Aug 2006

Database Location, Access and Structure

Scripts for Accessing & Searching the Database

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback