The purpose of this vignette is to demonstrate the construction of a simple ecflow suite from within R which makes us of some template job scripts which are kept within the packages extdata directory.
library(ecFlowR)
#>
#> Attaching package: 'ecFlowR'
#> The following object is masked from 'package:stats':
#>
#> family
#> The following object is masked from 'package:utils':
#>
#> editStart by specifying the name of the suite:
In this example the suite definition will be stored in the suite_path, the scripts for the task in script_path and the generated job files in job_path. These paths should be either the full path name or relative to the home path of an already running server. In this case they are defined as:
suite_path <- tempdir()
script_path <- file.path(suite_path,'scripts')
job_path <- file.path(suite_path,'jobs')which can be created using the following commands:
The test suite consists to jobs designed to highlight different ways of handling failures, either by returning a failure to the ecflow server or by catching it and returning a success.
The suite is generated using functions which return vectors of text strings which define the elements of the suite definition file which is written into suite_path.
## create the suite
tmp <- suite("ecFlowR_test",
edit("ECF_MICRO","%"),
edit("ECF_JOB_CMD","Rscript %ECF_JOB% > %ECF_JOBOUT% 2>&1"),
edit("ECF_KILL_CMD","kill -15 %ECF_RID%"),
edit("ECF_CHECK_CMD","ps --pid %ECF_RID% -f"),
edit("ECF_NODE","%ECF_HOST%"),
edit("ECF_STATUS_CMD","ps --pid %ECF_RID% -f > %ECF_JOB%.stat 2>&1"),
edit("ECF_URL_CMD","${BROWSER:=firefox} -remote 'openURL(%ECF_URL_BASE%/%ECF_URL%)'"),
edit("ECF_HOME",job_path),
edit("ECF_FILES",script_path),
edit("ECF_EXTN",".R"),
task("shouldRun",
cron("00:00","23:59","00:01"),
label("lastRun"),
label("lastSucess")),
task("shouldFail",
cron("00:00","23:59","00:01"),
label("lastRun"),
label("lastSucess")),
task("catchFail",
cron("00:00","23:59","00:01"),
label("lastRun"),
label("lastSucess"))
)
## write out the suite
suite_def <- file.path(suite_path,paste0(suite_name,".def"))
writeLines(tmp,suite_def) Some important aspects of the definition are: - The setting of ECF_EXTN to the standard extension of R files - The setting of ECF_JOB_CMD to interpret the ecf scripts using the Rscript engine.
The three tasks are set to execute every minute and update the labels lastRun and lastSuccess with the last time the script was run and the last time it completed sucessfully respectively.
A set of example job scripts can be found in
For use in the suite they can be extracted to the script_path
Looking at the scripts we see that they contain standard R code.
writeLines(readLines(file.path(script_path,"catchFail.R")))
#> ## source the ecflowR functions
#> library(ecFlowR)
#>
#> ## set environment variables
#> Sys.setenv(ECF_PORT = %ECF_PORT%,
#> ECF_NAME = "%ECF_NAME%",
#> ECF_PASS = "%ECF_PASS%",
#> ECF_TRYNO = %ECF_TRYNO%,
#> ECF_RID = Sys.getpid())
#>
#> ## tell ecflow_server we have started
#> ecflow_init()
#>
#> ## get current time
#> curTime <- format(Sys.time(),tz='GMT')
#>
#> ## update the last run label
#> ecflow_label("lastRun",paste(curTime))
#>
#> ## wrap code in try catch
#> tryCatch({
#> ## wait a few seconds
#> Sys.sleep(5)
#> ## update the label
#> stop("Nasty R Error - catch it!!")
#> ## update the lastSucess label
#> ecflow_label("lastSucess",paste(curTime))
#> },
#> error=function(e){
#> print(paste("Error:",e$message))
#> })
#>
#> ## exit nicely
#> ecflow_complete()The key aspects that make this suitable for an ecflow task script are: - the setting of the environment variables for communication with the ecflow_server (Note the use of the ECF_MACRO character to identify variables subsituted by the ecflow_server during job generation) - calls to ecflow_init and ecflow_complete for telling the server when jobs have started and completed - The catching of the error (stop) in R so that the lastSuccess label is never updated the but the code returns a completed message to the server.