Tool.py

Auxiliary tools for syqada. Identify errors, reset or purge a batchroot directory, validate a protocol, task, or template, determine whether syqada should be invoked in the current directory, prepare a skeleton for a new instance of a workflow, display man pages.

The error stuff can be invoked either as syqada tools, or as syqada manage. The validation, skeleton-prep, and man pages are invoked as

>>> syqada validate filename
>>> syqada begin [--help]
>>> syqada tutorial [--help]
>>> syqada manual [page-prefix]

Error reporting invocation looks something like:

>>> syqada [manage | tools] batchrootdirectory [--errors [options] | --reset | --purge ]

Options:

‘–errors [classify | first | last | all ] [LL NN] [stderr] [stdout]’

Provide detailed listing of errors. first, last, or all shows the length of stderr and stdout as well as the name of the task shellscript for the first, last, or all errors.

Adding stderr and/or stdout can also show the standard error or output of a particular error.

LL is the number of lines of stderr or stdout to show (default shows the first and last 4 lines). NN is the number of error exemplars to show (default 2)

The classify option identifies classes of errors by assuming that standard error outputs with different numbers of lines usually indicate differing kinds of error. This is the default, because it’s usually all you’ll need to resolve a problem, and also shows stderr from one of each error class by default. Also invocable as syqada errors.

‘–reset’

Remove the contents of the batch directory except for METADATA so that syqada batch can recreate jobs and re-execute. Also invocable as syqada reset.

‘–purge’

Remove the entire batch directory to force syqada auto to recreate the METADATA and re-execute.

Also invocable as syqada purge.

class Tool.Stringency(strictness)

Set degrees of adherence to reproducible standards. Currently, only that we are going to clean up environment variables before running subprocesses.

class Tool.Tool(batchroot, args, batch_tool=None, stdin=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>, stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>)

Do useful things commonly needed. First version only simplifies error checking and batch cleanup.

characterize_one_error(id, errordict, ignoregoods=False)

Characterize the error output of a job by its linecount, putting the (id, output, error-evaluation) triple into errordict keyed by linecount

classify_errors(thismanylines, thismanyexemplars=3, anyway=False)

return a list of error indices that are exemplars of different errors.

find_exemplar(theclass, errordata)

Look at all errors of a given type (currently, same # lines) and pick a likely one or two. Return a triple: (id, a description of the error type, syqada evaluation)

print_outputs(id, desired, evaluation, thismanylines=2)

Given a sample id, show standard information, plus thismanylines of stderr or stdout if requested

desired can be one or both of stderr or stdout

purge()

Remove a batchroot directory completely, so that syqada auto can recreate the METADATA from scratch.

Check for successful outputs and prompt for confirmation if any exist.

reset()

Restore a batchroot directory to its uninitialized state, that is to say, containing only the METADATA file, so that syqada batch can recreate the jobs from scratch.

Check for successful outputs and prompt for confirmation if any exist.

show_error(requested=[], anyway=False)

Driver for all –error options.

Tool.common_parsing(parser)

Arguments that should be shared with any object (the BatchRunner)

Tool.initialize_begin_and_tutorial(args, interactive=True, stdin=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>, stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>, debug=False)

See if user wishes to initialize.

if args.begin, then see if we can find a prototype protocol to use

if tutorial, find it and args.begin it.

Tool.process_task(taskfile, should_fix, stdout, stderr, **configuration)

Validate a task and report any parameters missing from its spec, or simply list and fix them if should_fix is True

Tool.tool_parser(_input=None, parser=None, stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>)

Add terms to parser, creating the parser if necessary.

Tool.validate_file(args, stdout, stderr)

Depending on suffix of file, validate as a template, task, or protocol.

For tasks, validate the template they specifiy, for protocols, validate the tasks they specify (and recursively, the templates).

Report all missing or obsolete terms (obsolete as identified in Protocol) to stdout. Informative remarks and labels not appropriate to a config file go to stderr so that redirecting stdout will produce a config file skeleton.

Use config file if given to reduce the number of missing terms reported.