source: git/Tst/README @ c1d386

Last change on this file since c1d386 was c1d386, checked in by Hans Schönemann <hannes@…>, 16 years ago
*hannes: path updated, examples git-svn-id: file:///usr/local/Singular/svn/trunk@9785 2c84dea3-7e68-4137-9b89-c4e89433aadc
  • Property mode set to 100644
File size: 6.1 KB
1README file for Singular test-suite
4This is the test-suite for Singular. This directory contains
6regress.cmd     -- perl script which drives tests
7Short           -- directory containing "short" tests
8Long            -- directory containing "long" tests
9Old             -- directory containing "old" (TST) tests
12  perl ./regress.cmd -h
13for a summary of the comaand-line options of regress.cmd.
15To test(s) correctness of Singular:
171.) Put the Singular binary in this directory
18    (prefered solution: ln -s <path_to_Singular_binary> Singular)
192.) Run
20     perl ./regress.cmd [*.tst] [*.lst]
21    with the desired tst (i.e., Singular test scripts) or lst (list
22    of tst files -- see Old/universal.lst for an example) file(s).
23    (prefered test scenario:
24    perl ./regress.cmd Old/universal.lst Short/ok_s.lst Long/ok_l.lst
25    )
263.) If no differences between the new results and the
27    previously obtained results are found, regress.cmd exits with
28    status 0.
294.) If differences between the new results and the
30    previously obtained results are found, regress.cmd exits with
31    status != 0, and, for each file xx.tst which lead to a
32    difference, the coresponding files, xx.res, and xx.diff
33    are kept, and differences are written to stdout.
345.) The Singular executable which is used for testing can
35    alternatively be specified using the -s <Singular> option.
38To test memory/timing performance of Singular:
401.) Put the Singular binary in this directory
412.) Run
42     perl ./regress.cmd -e [*.tst] [*.lst]
43    with the desired tst (i.e., Singular test scripts) or lst (list
44    of tst files -- see Old/universal.lst for an example) file(s).
453.) If no result differences occur, and if timing/memory performance
46    differences are less than $error_val (currently set to 5)
47    per-cent, then regress.cmd exits with status 0.
484.) Otherwise, exit status is != 0, and the corresponding files
49, xx.res, xx.diff, and xx.stat.diff (which logs the
50    differences of timing/memory performance) are kept, and
51    differences are written to stdout.
525.) Using the -r instead of the -e option only reports about
53    memory/timings differences, i.e., it does not cause an exit with
54    status != 0 if timing/memory performance is above specified
55    per-cent.
566.) The timing/memory performance differenes which trigger an error
57    (or, report) can be also be set: Either use
58      ./regress.cmd -[e,r] all%<val>
59    which sets tolerance of all criteria to <val> per-cent, or, use
60      ./regress.cmd -[e,r] <crit>%<val>
61    which sets tolerance of all <crit> to <val> per-cent.
63To generate new result file(s):
66    perl ./regress.cmd -g [*.tst] [*.lst]
67with the desired tst (i.e., Singular test scripts) or lst (list
68of tst files).
69NOTE: Running regress.cmd with the -g option re-generates the result
70and stat files, i.e., all previous results and statisitics are
71overwritten and lost!
73To include timing/memory statistics for a machine:
76    perl ./regress.cmd -m [*.tst] [*.lst]
77with the desired tst (i.e., Singular test scripts) or lst (list
78of tst files).
79NOTE:  Running regress.cmd with the -m option merges the
80timings/memory results of the current machine into the respective
81xx.stat file and overwrites previous timings/memory results of the
82current machine.
85To add a new test for a library/commando to the test-suite:
87* The following files need to be provided:
88        short/xx_s.tst : Singular code for short and basic tests
89        long/xx_l.tst : Singular code for long and extended tests
90  or, alternatively:
91        short/xx.tst: Singular code for short tests, only
92        short/xx.res : Result output of xx.tst
94* xx_s.tst should test the essential functionality of the
95  library/commando in a relatively short time (say, in no more than
96  30s).
98* xx_l.tst should test the functionality of the
99  library/commando in detail so that, if possible, all relevant
100  cases/results are tested. Nevertheless, such a test should not run
101  longer than, say, 10 minutes.
103* If useful tests do generally execute in a short time, have xx.tst,
104  only.
106* Rules for providing tst files:
107  1.) tst files always start with the following three commandos as
108  preamble:
109         LIB "tst.lib";
110         tst_init();
111         tst_ignore("CVS ID $Id:$"); // or, write own version number here
112  tst_init() writes some general info to stdout (like date,
113  uname, hostname, version, etc.). The library tst.lib (contained in
114  the Singular distribution) provides, among others, the routines
115  tst_init() and tst_ignore().
117  2.) tst files should end with the following statements:
118         tst_status(1); $
119  which enables (automatic) checks of the timing/memory performance of
120  Singular.
122  3.) All system-dependent output (like run-times, memory usages,
123  pathnames, dates, etc.) should generally be avoided.
125  4.) After time/memory critical sections of the tst files, the
126  command
127        tst_status();
128  should be inserted. This enables (automatic) checks of the
129  timing/memory performance of Singular since the last call to
130  tst_status (resp. since the start-up of Singular).
132  5.) If system-dependent output can not be avodied, the routine
133  tst_ignore(...) should be used:
134  tst_ignore(val [, keyword]): 'val' can have arbitrary type for which
135                               a string conversion exists;
136                               if present, keyword must be one of the
137                               following strings: "time", "memory"
138  tst_ignore() outputs 'val' by prepending the following prefix:
139  no keyword --       // tst_ignore:
140  "time" keyword --   // tst_ignore: time:
141  "memory" keyword -- // tst_ignore: memory:
142  which causes automatic tests to ignore these lines when doing a diff
143  on result files.
145* The command for generating xx.res.gz.uu and xx.stat is:
146  cd <directopry where xx.tst resides>; perl ../regress.cmd -g xx.tst
148* You can download the newest version of tst.lib from
Note: See TracBrowser for help on using the repository browser.