Changeset e2114af in git for Tst/README


Ignore:
Timestamp:
Jul 13, 1998, 11:22:54 AM (26 years ago)
Author:
Olaf Bachmann <obachman@…>
Branches:
(u'fieker-DuVal', '117eb8c30fc9e991c4decca4832b1d19036c4c65')(u'spielwiese', 'b52fc4b2495505785981d640dcf7eb3e456778ef')
Children:
de80b3c31fb1416223e69bba22557229daaa6908
Parents:
19fbf0a3717469b69ac8af448e962da5745a4ef3
Message:
* incoporatred status checks


git-svn-id: file:///usr/local/Singular/svn/trunk@2317 2c84dea3-7e68-4137-9b89-c4e89433aadc
File:
1 edited

Legend:

Unmodified
Added
Removed
  • Tst/README

    r19fbf0 re2114af  
    44This is the test-suite for Singular. This directory contains
    55
    6 regress.cmd     -- perl script which drives tests
     6regress.cmd     -- perl script which drives tests 
    77Short           -- directory containing "short" tests
    88Long            -- directory containing "long" tests
    99Old             -- directory containing "old" (TST) tests
    1010
    11 To run Singular test(s):
    12 ------------------------
     11Run
     12  perl ./regress.cmd -h
     13for a summary of the comaand-line options of regress.cmd.
     14
     15To test(s) correctness of Singular:
     16-----------------------------------
     171.) Put the Singular binary in this directory.
     182.) Run
     19     perl ./regress.cmd [*.tst] [*.lst]
     20    with the desired tst (i.e., Singular test scripts) or lst (list
     21    of tst files -- see Old/universal.lst for an example) file(s).
     223.) If no differences between the new results and the
     23    previously obtained results are found, regress.cmd exits with
     24    status 0.
     254.) If differences between the new results and the
     26    previously obtained results are found, regress.cmd exits with
     27    status != 0, and, for each file xx.tst which lead to a
     28    difference, the coresponding files xx.new.res, xx.res, and xx.diff
     29    are kept, and differences are written to stdout.
     305.) The Singular executable which is used for testing can
     31    alternatively be specified using the -s <Singular> option.
     32
     33
     34To test memory/timing performance of Singular:
     35----------------------------------------------
    13361.) Put the Singular binary in this directory
    14 2.) cd to one of the directories above
    15 3.) run
    16      ../regress.cmd [*.tst] [*.lst]
     372.) Run
     38     perl ./regress.cmd -e [*.tst] [*.lst]
    1739    with the desired tst (i.e., Singular test scripts) or lst (list
    18     of tst files) file(s).
    19 4.) For each file xx.tst,
    20       ../Singular -tqr12345678 xx.tst > xx.new.tst
    21       diff xx.res xx.new.res | tee xx.diff
    22     is run.
    23 5.) If no diffs were found, xx.diff and xx.new.res are rm'ed and
    24     xx.res is gzip'ed and uuencoded. Otherwise (i.e., diffs were
    25     found), xx.diff and xx.new.res are kept, diffs are printed to
    26     stdout, and regress.cmd exits with exit status != 0.
     40    of tst files -- see Old/universal.lst for an example) file(s).
     413.) If no result differences occur, and if timing/memory performance
     42    differences are less than $error_val (currently set to 5)
     43    per-cent, then regress.cmd exits with status 0.
     444.) Otherwise, exit status is != 0, and the corresponding files
     45    xx.new.res, xx.res, xx.diff, and xx.stat.diff (which logs the
     46    differences of timing/memory performance) are kept, and
     47    differences are written to stdout.
     485.) Using the -r instead of the -e option only reports about
     49    memory/timings differences, i.e., it does not cause an exit with
     50    status != 0 if timing/memory performance is above specified
     51    per-cent.
     526.) The timing/memory performance differenes which trigger an error
     53    (or, report) can be also be set: Either use
     54      ./regress.cmd -[e,r] all%<val>
     55    which sets tolerance of all criteria to <val> per-cent, or, use
     56      ./regress.cmd -[e,r] <crit>%<val>
     57    which sets tolerance of all <crit> to <val> per-cent.
    2758
    28 Do also
    29    ../regress.cmd -h
    30 and see the source code of ../regress.cmd for some more details on
    31 regress.cmd.
     59To generate new result file(s):
     60-------------------------------
     61Run
     62    perl ./regress.cmd -g [*.tst] [*.lst]
     63with the desired tst (i.e., Singular test scripts) or lst (list
     64of tst files).
     65NOTE: Running regress.cmd with the -g option re-generates the result
     66and stat files, i.e., all previous results and statisitics are
     67overwritten and lost!
    3268
    33 Note:
    34 (1) You should use regress.cmd -g to generate a result file!
    35 (2) You need the (GNU) uuencode for regress.cmd to work
    36 
    37 To unpack a result file (say, xx)
    38 ----------------------------------
    39 uudecode -o xx.res.gz xx.res.gz.uu; gunzip -f xx.res.gz
    40 
    41 To pack a result file (say, xx)
    42 --------------------------------
    43 gzip -f xx.res; uuencode xx.res.gz xx.res.gz > xx.res.gz.uu; rm -rf xx.res.gz
     69To include timing/memory statistics for a machine:
     70---------------------------------------------------
     71Run
     72    perl ./regress.cmd -m [*.tst] [*.lst]
     73with the desired tst (i.e., Singular test scripts) or lst (list
     74of tst files).
     75NOTE:  Running regress.cmd with the -m option merges the
     76timings/memory results of the current machine into the respective
     77xx.stat file and overwrites previous timings/memory results of the
     78current machine.
    4479
    4580
     
    4984        short/xx_s.tst : Singular code for short and basic tests
    5085        long/xx_l.tst : Singular code for long and extended tests
    51         short/xx_s.res.gz.uu : Result output of xx_s.tst
    52         long/xx_l.resres.gz.uu : Result output of xx_l.tst
    5386  or, alternatively:
    5487        short/xx.tst: Singular code for short tests, only
     
    72105         LIB "tst.lib";
    73106         tst_init();
    74          tst_ignore("CVS ID $Id:$");
     107         tst_ignore("CVS ID $Id:$"); // or, write own version number here
    75108  tst_init() writes some general info to stdout (like date,
    76   uname, hostname, version, etc.). The library tst.lib (appended
    77   below) provides, among others, the routines tst_init() and
    78   tst_ignore().
     109  uname, hostname, version, etc.). The library tst.lib (contained in
     110  the Singular distribution) provides, among others, the routines
     111  tst_init() and tst_ignore().
    79112
    80   2.) All system-dependent output (like run-times, memory usages,
    81   pathnames, dates, etc.) must be output by means of the routine
    82   tst_ignore(...):
     113  2.) tst files should end with the following statements:
     114         tst_status(1); $
     115  which enables (automatic) checks of the timing/memory performance of
     116  Singular.
     117
     118  3.) All system-dependent output (like run-times, memory usages,
     119  pathnames, dates, etc.) should generally be avoided.
     120
     121  4.) After time/memory critical sections of the tst files, the
     122  command
     123        tst_status();
     124  should be inserted. This enables (automatic) checks of the
     125  timing/memory performance of Singular since the last call to
     126  tst_status (resp. since the start-up of Singular).
     127
     128  5.) If system-dependent output can not be avodied, the routine
     129  tst_ignore(...) should be used:
    83130  tst_ignore(val [, keyword]): 'val' can have arbitrary type for which
    84131                               a string conversion exists;
     
    86133                               following strings: "time", "memory"
    87134  tst_ignore() outputs 'val' by prepending the following prefix:
    88   no keyword --       // ignore:
    89   "time" keyword --   // ignore: time:
    90   "memory" keyword -- // ignore: memory:
     135  no keyword --       // tst_ignore:
     136  "time" keyword --   // tst_ignore: time:
     137  "memory" keyword -- // tst_ignore: memory:
    91138  which causes automatic tests to ignore these lines when doing a diff
    92139  on result files.
    93140 
    94   3.) Time and/or memory-intensive tests should be followed by
    95   (tst_ignore'd) time (resp. memory) output, so that we can keep track
    96   of the timing (resp. memory) behaviour of Singular. This could be
    97   accomplished as shown below:
    98   int mytime = timer;
    99   /* time/memory intesive computations */
    100   tst_ignore("time", timer - mytime);
    101   tst_ignore("memory", memory(0));
    102 
    103 * Command for generating *.res.gz.uu files:
    104   cd <directopry where xx.tst should reside>; ../regress.cmd -g xx.tst
     141* The command for generating xx.res.gz.uu and xx.stat is:
     142  cd <directopry where xx.tst resides>; perl ../regress.cmd -g xx.tst
    105143
    106144
Note: See TracChangeset for help on using the changeset viewer.