syqada errors¶
>>> syqada errors 02-demonstrate-failure-handling
classifies the stderr outputs of failed jobs and displays one or more. It is probably just a bit too verbose, but it will identify classes of failures that have exactly the same error output if they differ by nothing more than the sample name.
It is equivalent to:
>>> syqada manage 02-demonstrate-failure-handling --errors classify
As usual with SyQADA, this is merely syntactic sugar on tasks you could accomplish with command line tools in the sub-directories of your workflow. In the case of error examination, however, the syntactic sugar is not only sweet but high in protein. See How syqada manage Works to get some idea of how much.
>>> syqada errors task-directory [NN]
This command typically provides all the information you need to debug problems with tasks that have been reported as failed by syqada manage. Because this option is so useful, it has become the default, and is invoked by syqada auto as its final action upon detecting a failed task. NN, if provided, is the number of lines of each stderr file to show (default is 4).
An example follows:
% syqada manage 0010-overlap/ --error classify
Checking control directories... ............................................................
Checking logs... ....................................................................................................
.....................
164 errors with 1 lines of stderr output
15 errors with 2 lines of stderr output
%%%%%%%%%%%%% All 164 1-line outputs differ only by the sample id. %%%%%%%%%%%%%
######################## One example of a 1-line output ########################
TCGA-2L-AAQI-N: 1 stderr, 0 stdout 0010-overlap/ERROR/build_overlap_sets-runner-TCGA-2L-AAQI-N.sh%26755
********************************************************************************
----------------------------------- stderr -----------------------------------
--------------------------------------------------------------------------------
Error: The requested bed file (0010-overlap/output/TCGA-2L-AAQI-N.haploh) could not be opened. Exiting!
--------------------------------------------------------------------------------
********************************************************************************
%%%%% There are 2 kinds of 2-line stderr file after extracting sample ids. %%%%%
################## Example 1 of a 2-line output for 15 samples #################
TCGA-H6-A45NN-T: 2 stderr, 0 stdout 0010-overlap/ERROR/build_overlap_sets-runner-TCGA-H6-A45NN-T.sh%31645
********************************************************************************
----------------------------------- stderr -----------------------------------
--------------------------------------------------------------------------------
tail: cannot open `06-annotate_cnvs/output/TCGA-H6-A45NN-T.txt' for reading: No such file or directory
Error: The requested bed file (0010-overlap/output/TCGA-H6-A45NN-T.haploh) could not be opened. Exiting!
--------------------------------------------------------------------------------
********************************************************************************
################## Example 2 of a 2-line output for 15 samples #################
TCGA-3A-A9IN-T: 2 stderr, 0 stdout 0010-overlap/ERROR/build_overlap_sets-runner-TCGA-3A-A9IN-T.sh%27056
********************************************************************************
----------------------------------- stderr -----------------------------------
--------------------------------------------------------------------------------
tail: cannot open `06-annotate_cnvs/output/TCGA-3A-A9IN-T.txt' for reading: No such file or directory
Error: The requested bed file (0010-overlap/output/TCGA-3A-A9IN-T.cnv) could not be opened. Exiting!
--------------------------------------------------------------------------------
********************************************************************************
The –errors option takes several other optional arguments:
--errors [first[N] | last[N] | all [stderr] [stdout] [NN]
--errors first displays information about the first error in the LOGS directory.
It shows the sample name, the length of the stderr and stdout files, and the task runner.
--errors first3 displays information about the first 3 errors (etc)
--errors last shows the same information about the last error.
--errors last4 shows the same information about the last 4 errors.
--errors all shows all errors
--errors stderr [ adds to the previous information the contents of the standard error out (elided
to the first two and last two lines of the .err file. To see more, add the number of lines you wish to
see.
--errors stdout performs the same thing for standard output. Both may be requested, but only one
numeric value for number of lines to be displayed will be recognized and used for both stderr and stdout.
An example from the tutorial follows. This is what you should see if you run the tutorial to the point that task two fails. It reports the sample that erred, the length of the stderr and stdout files, and the name of the runner script (Note that syqada manage –errors classify will determine that there is a single error and simply show the same output).:
> syqada manage 02-demonstrate-failure-handling/ --errors
Checking control directories... ...........
Checking logs... ......................
********************************************************************************
rxia: 15 stderr, 1 stdout 02-demonstrate-failure-handling/ERROR/demonstrate-failure-handling-runner-rxia.sh%39571
********************************************************************************
********************************************************************************
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
** syqada believes this is an intentional error for the tutorial.
** For more information, run:
** syqada errors --help EXAMPLE_MESSAGE
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
Automator terminated task 02-demonstrate-failure-handling with an error.
The same command, requesting that standard error be displayed for the first error.:
> syqada manage 02-demonstrate-failure-handling/ --errors stderr 8
Checking control directories... ...........
Checking logs... ......................
********************************************************************************
rxia: 15 stderr, 1 stdout 02-demonstrate-failure-handling/ERROR/demonstrate-failure-handling-runner-rxia.sh%39571
********************************************************************************
----------------------------------- stderr -----------------------------------
--------------------------------------------------------------------------------
This demonstrates a job failure that might occur due to a
system configuration issue. In this case, the data had
a length of 8 and there was no file called no-length-based-name-bias.
...(7)...
syqada batch 02-demonstrate-failure-handling --step run
will cause it to run without error.
syqada batch 02-demonstrate-failure-handling --step repend run
will accomplish the same thing in one step
--------------------------------------------------------------------------------
********************************************************************************
The parenthesized number in the ellipsis indicates the number of unprinted lines.
Error Interpretation¶
Upon recognizing a task failure, SyQADA attempts to recognize and interpret the error to save user effort. An example is found in the basic tutorial. When step 02 fails, syqada prints this:
////////////////////////////////** syqada believes this is an intentional error for the tutorial. ** For more information, run: ** syqada errors –help EXAMPLE_MESSAGE ////////////////////////////////
Automator terminated task 02-demonstrate-failure-handling with an error.
If you do as suggested, you will see
> syqada errors –help EXAMPLE_MESSAGE The tutorial has failed (so as to demonstrate how to find an error).
syqada shows you the truncated stderr for each class of failure (here there’s only 1).
- The simple way to see the whole stderr output is:
- syqada errors 02-demonstrate-failure-handling 15
The parameter 15 specifies the number of lines of stderr output to display without ellipses.