371 lines
8.1 KiB
Groff
371 lines
8.1 KiB
Groff
|
.\"
|
||
|
|
.\" This file and its contents are supplied under the terms of the
|
||
|
|
.\" Common Development and Distribution License ("CDDL"), version 1.0.
|
||
|
|
.\" You may only use this file in accordance with the terms of version
|
||
|
|
.\" 1.0 of the CDDL.
|
||
|
|
.\"
|
||
|
|
.\" A full copy of the text of the CDDL should have accompanied this
|
||
|
|
.\" source. A copy of the CDDL is also available via the Internet at
|
||
|
|
.\" http://www.illumos.org/license/CDDL.
|
||
|
|
.\"
|
||
|
|
.\"
|
||
|
|
.\" Copyright (c) 2012 by Delphix. All rights reserved.
|
||
|
|
.\"
|
||
|
|
.TH run 1 "23 Sep 2012"
|
||
|
|
.SH NAME
|
||
|
|
run \- find, execute, and log the results of tests
|
||
|
|
.SH SYNOPSIS
|
||
|
|
.LP
|
||
|
|
.nf
|
||
|
|
\fBrun\fR [\fB-dgq] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] [\fB-uxX\fR \fIusername\fR]
|
||
|
|
\fIpathname\fR ...
|
||
|
|
.fi
|
||
|
|
|
||
|
|
.LP
|
||
|
|
.nf
|
||
|
|
\fBrun\fR \fB-w\fR \fIrunfile\fR [\fB-gq\fR] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR]
|
||
|
|
[\fB-uxX\fR \fIusername\fR] \fIpathname\fR ...
|
||
|
|
.fi
|
||
|
|
|
||
|
|
.LP
|
||
|
|
.nf
|
||
|
|
\fBrun\fR \fB-c\fR \fIrunfile\fR [\fB-dq\fR]
|
||
|
|
.fi
|
||
|
|
|
||
|
|
.LP
|
||
|
|
.nf
|
||
|
|
\fBrun\fR [\fB-h\fR]
|
||
|
|
.fi
|
||
|
|
|
||
|
|
.SH DESCRIPTION
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
The \fBrun\fR command has three basic modes of operation. With neither the
|
||
|
|
\fB-c\fR nor the \fB-w\fR option, \fBrun\fR processes the arguments provided on
|
||
|
|
the command line, adding them to the list for this run. If a specified
|
||
|
|
\fIpathname\fR is an executable file, it is added as a test. If a specified
|
||
|
|
\fIpathname\fR is a directory, the behavior depends upon the \fB-g\fR option.
|
||
|
|
If \fB-g\fR is specified, the directory is treated as a test group. See the
|
||
|
|
section on "Test Groups" below. Without the \fB-g\fR option, \fBrun\fR simply
|
||
|
|
descends into the directory looking for executable files. The tests are then
|
||
|
|
executed, and the results are logged.
|
||
|
|
|
||
|
|
With the \fB-w\fR option, \fBrun\fR finds tests in the manner described above.
|
||
|
|
Rather than executing the tests and logging the results, the test configuration
|
||
|
|
is stored in a \fIrunfile\fR which can be used in future invocations, or edited
|
||
|
|
to modify which tests are executed and which options are applied. Options
|
||
|
|
included on the command line with \fB-w\fR become defaults in the
|
||
|
|
\fIrunfile\fR.
|
||
|
|
|
||
|
|
With the \fB-c\fR option, \fBrun\fR parses a \fIrunfile\fR, which can specify a
|
||
|
|
series of tests and test groups to be executed. The tests are then executed,
|
||
|
|
and the results are logged.
|
||
|
|
.sp
|
||
|
|
.SS "Test Groups"
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
A test group is comprised of a set of executable files, all of which exist in
|
||
|
|
one directory. The options specified on the command line or in a \fIrunfile\fR
|
||
|
|
apply to individual tests in the group. The exception is options pertaining to
|
||
|
|
pre and post scripts, which act on all tests as a group. Rather than running
|
||
|
|
before and after each test, these scripts are run only once each at the start
|
||
|
|
and end of the test group.
|
||
|
|
.SS "Test Execution"
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
The specified tests run serially, and are typically assigned results according
|
||
|
|
to exit values. Tests that exit zero and non-zero are marked "PASS" and "FAIL"
|
||
|
|
respectively. When a pre script fails for a test group, only the post script is
|
||
|
|
executed, and the remaining tests are marked "SKIPPED." Any test that exceeds
|
||
|
|
its \fItimeout\fR is terminated, and marked "KILLED."
|
||
|
|
|
||
|
|
By default, tests are executed with the credentials of the \fBrun\fR script.
|
||
|
|
Executing tests with other credentials is done via \fBsudo\fR(1m), which must
|
||
|
|
be configured to allow execution without prompting for a password. Environment
|
||
|
|
variables from the calling shell are available to individual tests. During test
|
||
|
|
execution, the working directory is changed to \fIoutputdir\fR.
|
||
|
|
.SS "Output Logging"
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
By default, \fBrun\fR will print one line on standard output at the conclusion
|
||
|
|
of each test indicating the test name, result and elapsed time. Additionally,
|
||
|
|
for each invocation of \fBrun\fR, a directory is created using the ISO 8601
|
||
|
|
date format. Within this directory is a file named \fIlog\fR containing all the
|
||
|
|
test output with timestamps, and a directory for each test. Within the test
|
||
|
|
directories, there is one file each for standard output, standard error and
|
||
|
|
merged output. The default location for the \fIoutputdir\fR is
|
||
|
|
\fI/var/tmp/test_results\fR.
|
||
|
|
.SS "Runfiles"
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
The \fIrunfile\fR is an ini style configuration file that describes a test run.
|
||
|
|
The file has one section named "DEFAULT," which contains configuration option
|
||
|
|
names and their values in "name = value" format. The values in this section
|
||
|
|
apply to all the subsequent sections, unless they are also specified there, in
|
||
|
|
which case the default is overridden. The remaining section names are the
|
||
|
|
absolute pathnames of files and direcotries, describing tests and test groups
|
||
|
|
respectively. The legal option names are:
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBoutputdir\fR = \fIpathname\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
The name of the directory that holds test logs.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBpre\fR = \fIscript\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Run \fIscript\fR prior to the test or test group.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBpre_user\fR = \fIusername\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Execute the pre script as \fIusername\fR.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBpost\fR = \fIscript\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Run \fIscript\fR after the test or test group.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBpost_user\fR = \fIusername\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Execute the post script as \fIusername\fR.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBquiet\fR = [\fITrue\fR|\fIFalse\fR]
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
If set to True, only the results summary is printed to standard out.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBtests\fR = [\fI'filename'\fR [,...]]
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Specify a list of \fIfilenames\fR for this test group. Only the basename of the
|
||
|
|
absolute path is required. This option is only valid for test groups, and each
|
||
|
|
\fIfilename\fR must be single quoted.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBtimeout\fR = \fIn\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
A timeout value of \fIn\fR seconds.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fBuser\fR = \fIusername\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Execute the test or test group as \fIusername\fR.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.SH OPTIONS
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
The following options are available for the \fBrun\fR command.
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-c\fR \fIrunfile\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Specify a \fIrunfile\fR to be consumed by the run command.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-d\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Dry run mode. Execute no tests, but print a description of each test that would
|
||
|
|
have been run.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-g\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Create test groups from any directories found while searching for tests.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-o\fR \fIoutputdir\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Specify the directory in which to write test results.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-p\fR \fIscript\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Run \fIscript\fR prior to any test or test group.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-P\fR \fIscript\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Run \fIscript\fR after any test or test group.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-q\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Print only the results sumary to the standard output.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-t\fR \fIn\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Specify a timeout value of \fIn\fR seconds per test.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-u\fR \fIusername\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Execute tests or test groups as \fIusername\fR.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-w\fR \fIrunfile\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Specify the name of the \fIrunfile\fR to create.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-x\fR \fIusername\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Execute the pre script as \fIusername\fR.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB-X\fR \fIusername\fR
|
||
|
|
.ad
|
||
|
|
.RS 6n
|
||
|
|
Execute the post script as \fIusername\fR.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.SH EXAMPLES
|
||
|
|
.LP
|
||
|
|
\fBExample 1\fR Running ad-hoc tests.
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
This example demonstrates the simplest invocation of \fBrun\fR.
|
||
|
|
|
||
|
|
.sp
|
||
|
|
.in +2
|
||
|
|
.nf
|
||
|
|
% \fBrun my-tests\fR
|
||
|
|
Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS]
|
||
|
|
Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS]
|
||
|
|
Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS]
|
||
|
|
|
||
|
|
Results Summary
|
||
|
|
PASS 3
|
||
|
|
|
||
|
|
Running Time: 00:00:07
|
||
|
|
Percent passed: 100.0%
|
||
|
|
Log directory: /var/tmp/test_results/20120923T180654
|
||
|
|
.fi
|
||
|
|
.in -2
|
||
|
|
|
||
|
|
.LP
|
||
|
|
\fBExample 2\fR Creating a \fIrunfile\fR for future use.
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
This example demonstrates creating a \fIrunfile\fR with non default options.
|
||
|
|
|
||
|
|
.sp
|
||
|
|
.in +2
|
||
|
|
.nf
|
||
|
|
% \fBrun -p setup -x root -g -w new-tests.run new-tests\fR
|
||
|
|
% \fBcat new-tests.run\fR
|
||
|
|
[DEFAULT]
|
||
|
|
pre = setup
|
||
|
|
post_user =
|
||
|
|
quiet = False
|
||
|
|
user =
|
||
|
|
timeout = 60
|
||
|
|
post =
|
||
|
|
pre_user = root
|
||
|
|
outputdir = /var/tmp/test_results
|
||
|
|
|
||
|
|
[/home/jkennedy/new-tests]
|
||
|
|
tests = ['test-01', 'test-02', 'test-03']
|
||
|
|
.fi
|
||
|
|
.in -2
|
||
|
|
|
||
|
|
.SH EXIT STATUS
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
The following exit values are returned:
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB\fB0\fR\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
Successful completion.
|
||
|
|
.RE
|
||
|
|
.sp
|
||
|
|
.ne 2
|
||
|
|
.na
|
||
|
|
\fB\fB1\fR\fR
|
||
|
|
.ad
|
||
|
|
.sp .6
|
||
|
|
.RS 4n
|
||
|
|
An error occurred.
|
||
|
|
.RE
|
||
|
|
|
||
|
|
.SH SEE ALSO
|
||
|
|
.sp
|
||
|
|
.LP
|
||
|
|
\fBsudo\fR(1m)
|