Condor Boot Camp at Purdue
Lecture Materials
- Using Condor (Powerpoint) (PDF)
- Administrating Condor (Powerpoint) (PDF) (Handout)
- Condor Tutorial
Other Materials
Implementing an Industrial-Strength Academic Cyberinfrastructure at Purdue University
Implementing a Central Quill Database in a Large Condor Installation (Condor Week 2008)
BoilerGrid for cyro-EM image processing (Condor Week 2008)
This site is currently under construction. Please check back frequently for updates.
Condor submit description file examples
Example 1 Here is first the simplest possible submit description file. It will put one copy of the program hello (which has first been created by condor_compile) in queue for execution by Condor. There has been no definition of platform, so Condor will just use its default, which is to run the job on a machine which has the same architecture and operating system as the machine from which it was submitted.No input, output, and error commands are given in the submit description file, so the files stdin, stdout, and stderr will all refer to /dev/null. The program may produce output by explicitly opening a file and writing to it. A log file, hello.log, will also be produced. This log-file will contain events the job had during its lifetime inside of Condor, such as any possible errors. When the job finishes, its exit conditions will also be noted in the log file. It is recommended that you always have a log file so you know what happened to your jobs.
If your program only returns output to the screen (like the hello.c program below does), then you should include Output = hello.out or something like it, somewhere before Queue. Otherwise you will not see the output.
####################
#
# Example 1
# Simple condor job description file
#
####################
Executable = hello
Log = hello.log
Queue
Example 2
In this example (from the Condor manual, we queue two copies of the program mathematica. The first copy will run in directory run_1, and the second will run in directory run_2. For both queued copies, stdin will be test.data, stdout will be loop.out, and stderr will be loop.error. There will be two sets of files written, as the files are each written to their own directories. This is a convenient way to organize data if you have a large group of Condor jobs to run. The example file shows program submission of mathematica as a vanilla universe job. This may be necessary if the source and/or object code to program mathematica is not available.
####################
#
# Example 2: demonstrate use of multiple
# directories for data organization.
#
####################
Executable = mathematica
Universe = vanilla
input = test.data
output = loop.out
error = loop.error
Log = loop.log
Initialdir = run_1
Queue
Initialdir = run_2
Queue
Example 3
In this example (also from the Condor manual, the submit description file queues 150 runs of program foo which has been compiled and linked for Silicon Graphics workstations running IRIX 6.5. This job requires Condor to run the program on machines which have greater than 32 megabytes of physical memory, and expresses a preference to run the program on machines with more than 64 megabytes, if such machines are available. It also advises Condor that it will use up to 28 megabytes of memory when running. Each of the 150 runs of the program is given its own process number, starting with process number 0. So, files stdin, stdout, and stderr will refer to in.0, out.0, and err.0 for the first run of the program, in.1, out.1, and err.1 for the second run of the program, and so forth. A log file containing entries about when and where Condor runs, checkpoints, and migrates processes for the 150 queued programs will be written into file foo.log.
#################### # # Example 3: Show off some fancy features including # use of pre-defined macros and logging. # #################### Executable = foo Requirements = Memory >= 32 && OpSys == "IRIX65" && Arch =="SGI" Rank = Memory >= 64 Image_Size = 28 Meg Error = err.$(Process) Input = in.$(Process) Output = out.$(Process) Log = foo.log Queue 150
