Dr Guiyuan Lei
University of Newcastle
Newcastle upon Tyne, United Kingdom

Calibayes CISBAN

Home  Research  Resource  iNotes  Personal

iNotes

System Biology

Computer Tools

Misc

L^ATEX:

• Why use LaTeX?

LaTeX is a powerful tool for creating scientific papers, you can generate paper in pdf files and slides from LaTeX. You can also get html files from .tex file. To start use LaTeX , please refer Tobias Oetiker et al. The Not So Short Introduction to LaTeX2e , Andrew Roberts's Getting to grips with LaTeX and Tim Love's Text Processing using LaTeX .

• Bibtex

It is time-consuming task to manage references, especially when you use some common references in different papers with different bibliography appearances. Bibtex is a good choice to create your bibliography. For general information on Bibtex, please see Getting to Grips with LaTeX - Bibliography Management and Bibliographies with LaTeX.

What I want to emphases here is that the authors in .bib file should be connected by "and", not by comma. If you use comma between authors, the order of authors will be wrong. For example, use "Michel Goossens and Frank Mittelbach and Alexander Samarin" in the field of author. If you use "Michel Goossens, Frank Mittelbach and Alexander Samarin", the first author in the biobliography is not Michel Goossens any more.

Different journals demand difference reference styles. If the journal has specific citation style, like author-year style, it is good to use natbib package. Sometimes you need to use CUSTOM-BIB Package to generate specific bibliography styles, i.e., your own .bst files.

The following steps are for how to generate your own .bst for natbib and use it in WinEdt.

1. Download package custom-bib and generate your own .bst file. For example, I generate jcbnat.bst for the Journal of Computational Biology. Put the file jcbnat.bst in the directory ..\texmf\bibtex\bst\netbib\
2. Your should update the MiKtex in WinEdt so that the compiler can find this file. Go to the menu of "Accessories", select "MiKTex options", then in the coming up dialog, select tag "Root" and select the path "..\texmf" and click the button "Refresh FNDB", finally click "Apply" and "Ok" to finish the update.
3. Use the following sentence in the preamble of .tex file.
   \usepackage{natbib} % author-year citation style
   \bibpunct{(}{)}{;}{a}{,}{,}%Selecting citation punctuation
   \bibliographystyle{jcbnat}
4. Use \citep{} when you cite reference.

Now you will get the author-year citation style and the corresponding bibliography style which is defined in file jcbnat.bst.

Example: Latex.tex, bibtex.bib, jcbnat.bst.

• LaTeX2HTML

LaTeX2HTML is a convertor written in Perl that converts LaTeX documents to HTML. Please refer the LaTeX2HTML website.

• Pdfslide

Linux:

• Cluster computing condor

Condor is very useful for cluster computing. The system will distribution one job to many available computers within the cluster. You can define input file and output file for your program. You can also submit several condor jobs. Here is tip to submit 10 (or other numbers depends your computing facility) jobs automately and easily (don't need to write 10 classad files), quite useful expecially for MCMC method using different random seeds/sequences. The following example is for cisbclust cluster.

• Write a shell file "runmhd.sh". My execute file "metropolis" do metropolis halsting search for genetic network, it need input file "infile0" to provide the random seed, the data file etc. I aslo would like the executing of the metropolis output to a file, not screen. So the $1 and $2 are parameters which tell condor that the program "metropolis" has one input file and one output file. "cd" line tells condor where the program is.

  	#!/bin/sh
  	PATH=$PATH:~/bin
  	cd ~/MH-d/409genes/run1
  	date
  	./metropolis < $1 > $2
  	date
  

• Write submit file "mhd.classad", in this file you can define what is the name of input file and output file. Paramater "$(Process)" will count the rank/index according the parameter "Queue". Here I set "Queue" as 10. So I let condor to submit 10 jobs. The input file need to be "infile0", "infile1",..."infile9".
  

  Executable      = runmhd.sh
  Arguments       = infile$(Process) Fibroblast_networks$(Process).txt  
  Universe        = vanilla
  Requirements    = OpSys == "LINUX" && Arch =="X86_64"
  
  Error   = mhd$(Process)_condor.err
  Output  = mhd$(Process)_condor.out
  Log     = mhd$(Process)_condor.log
  
  Queue 10 
  

• Creats 10 input file "infile0", "infile1",..."infile9" which have different random seeds.
• Then submit the jobs by type the following command

  condor_submit mhd.classad
  

After the jobs are finished, I will get 10 different results "Fibroblast_networks0", "Fibroblast_networks1",..."Fibroblast_networks9" using 10 different random sequences.

• Condor checkpoint

But when you run long time jobs, the job may be evicted because of the priority issue. So it is necessary to use condor checkpoint which records current image of job when the program is evicted and resume from the break point automately. First the cluster should support "standard" (not only "vanilla") for "universe" parameter setting. Secondly need compile the code using "condor_compile". After submitting the condor job, you can use "condor_checkpoint [the host name of the computer which executing the job]" to force the checkpoint, or the job will be checkpointed automately for example every 3 hours (depend on systems).

Two examples of mhd.classd file suppot checkpoing from my work. First one for the previous MH-d program (metropolis) on
quantum:/data/ngl9/98genes/run1

Executable      = metropolis
Arguments       = infile$(Process) Yeast_networks$(Process).txt
Universe        = standard
Requirements    = OpSys == "LINUX"

Error   = mhd$(Process)_condor.err
Output  = mhd$(Process)_condor.out
Log     = mhd$(Process)_condor.log
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
Queue 10

Second example for my VAR code on
fruitfly:/data/BayesianNetworks/src/var1_rj/Yeast/183genes/run4
please note this program read in input and output file names from argv[1] and argv[2], also has another file to read in data. So we have "transfer_input_files = timecourses.dat".

Executable      = VARrj
Arguments       = infile$(Process) mcmc$(Process).tab
Universe        = standard
Requirements    = OpSys == "LINUX"

Input   = infile$(Process)
Output  = mhd$(Process)_condor.out
Error   = mhd$(Process)_condor.err
Log     = mhd$(Process)_condor.log
transfer_input_files = timecourses.dat
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
Queue 10

It seems a bit complex for different cluster systems. Those two don't need shell file runmhd.sh as the one which uses "vanilla" (cisbclust cluster does not support "standard" condor jobs). Also need be careful about different settings for transferred files.

• Subversion

Subversion is good for several to share files. The history of modification is kept. Some useful commands are as following:

1. svn checkout https://metagenome.ncl.ac.uk/subversion/cisban/BayesianNetworks/ BayesianNetworks
2. svn update
3. svn add
4. svn commit -m "message"

• ssh

For running MPICH, ssh should be configured to allow communication between nodes/processes without password. In order to run MPICH on a single computer, ssh should also be able to access localhost so that communicaton is allowed between virtual multiple processes.

• Configue ssh for MPICH
  1. ssh-keygen -t dsa -b 1024 -f ~/.ssh/gl_ssh
     Then enter passphrase. Two files will be generated.
  2. cd ~/.ssh
     cat gl_ssh.pub >> authorized_keys
  3. ssh-add ~/.ssh/gl_ssh
  4. run MPICH complied program
  5. Deactivate the ssh agent by using ssh-add -d ~/.ssh/gl_ssh . After deactivate, if you want to run ssh without password, you should run the command ssh-add ~/.ssh/gl_ssh again.
• Allow ssh localhost
  Add line: sshd:127.0.0.1 in the file /etc/hosts.allow

• Print files

For .ps or .pdf files, use:

 lpr -P [printer name] [file name] 

For .txt files, the files should be converted to ps first, you can use the following command, there are no .ps files generated, print the .txt files directly to the specific printer.

 a2ps -o- [txt file name] | lpr -P [printer name] 

• Copy files between different computers

Copy file from local (compg) to remote machine (basis)

 scp gibbsreg.cpp ngl9@basis.ncl.ac.uk:~/CISBAN/HdBCS/STEP1/ 

Copy file from remote (basis) to local machine (compg)

 scp ngl9@basis.ncl.ac.uk:~/CISBAN/HdBCS/STEP1/gibbsreg.cpp ~/CISBAN/HdBCS/

: is followed by directory

• View usb-disk

 cd /media/usb0/ 

• lamboot with specific nodes

When lamboot runs, it uses a file called bhost to know what processors are available. Sometimes, some nodes will fail and you may need to use your own, modified, bhost file to get lamboot to avoid the failed processor. Here's how to do this.

1. Make a copy of the /etc/lam/bhost (or /etc/lam/bhost.def) file into a directory that you own.
2. Comment out the line for the failed node in your copy of bhost.
3. Run lamboot with your bhost file as a command-line argument. For example, you can type the following to get lamboot to use the bhost file in the current directory rather than the one under /etc/lam.
   

    lamboot bhost

4. To stop LAM
   

    lamhalt bhost

• Compile C++ program which using fortran package in lammpi

If the code is written in C++, but using public fortran package such lapack,blas and blitz++, you should link the object file with library lammpi++. The following is one example:

g++ -c gibbserg.cpp -I/usr/lib/lam/include 

mpif77 -llammpi++ -lstdc++ gibbrseg.o -lblas -llapack -lg2c -lm

Otherwise the compiling fails. It is also important to keep in mind that you use lammpi compile one program and run the program in lammpi environment, using lamboot. If you compile one program using mpich, but run with lammpi, you would get error message like "One of the proceses started by mpirun has existed with nonzero exit code. ..."

 

Last modified:
16 July, 2008