Running R Jobs via LSF on Lewis
LSF, the Load Sharing Facility, is a subsystem for submitting, monitoring, and controlling a workload of batch jobs across compute servers in a cluster. With LSF, jobs can be scheduled for execution across the lewis compute nodes according to scheduling policies that attempt to fully utilize system resources without over committing those resources, while being fair to all users. For more information about the LSF batch system, see the online manual page, which can be viewed by executing the command:man lsfbatchJobs are submitted to be run under LSF via the bsub command. For general information about how to submit batch jobs to LSF on lewis, see Submitting Jobs via LSF on Lewis. For complete details on using bsub, see the online manual page, which can be viewed by executing the command:
man bsubThe bsub command has lots of options, and can be used in several different ways, but a simple way to define a batch job for submitting an R job via bsub is to create a command file which contains the R command which should be run, along with some options for the bsub command. A sample LSF job command file for an R job might contain lines like the following:
#BSUB -J jobnamewhere jobname is the name that you want to give to the job, and inputfile is the input file for R. The job's standard output will be placed in a file in your directory named jobname.onnnn, where nnnn is the job number (indicated by specifying the characters %J in the file name) that is assigned when the job is submitted. The error output will be placed in a file named jobname.ennnn.
#BSUB -oo jobname.o%J
#BSUB -eo jobname.e%J
R -save < inputfile
If you prefer to have the job output sent to you via email, you can specify an email address instead of specifying output and error file names:
#BSUB -J jobnameYou can specify -no-save instead of -save, and other options as required on the R command. You can also specify any other bsub command options that you want to include. The bsub command options can be included on the bsub command itself, but it is a good idea to put them in the command file.
#BSUB -u userid@missouri.edu
R -save < inputfile
Once the job command file has been created, it can be submitted for execution via the bsub command, as follows:
bsub < commandfileThe bsub command will respond with a line like the following:
Job <nnnn> is submitted to default queuewhere nnnn is the job number assigned to the job. The job number can by used to identify your job with other LSF commands, like bjobs, bhist, and bkill. See the man pages for those commands for complete details. The job number is also used to identify output files for your job if you set up your command file like the first example above..
Use the bjobs command to list your jobs that are queued and executing on lewis. For example:
bjobs -aSee the man page for bjobs for complete details.