source: git/Tst/README @ 19fbf0

README file for Singular test-suite
===================================

This is the test-suite for Singular. This directory contains 

regress.cmd     -- perl script which drives tests
Short           -- directory containing "short" tests
Long            -- directory containing "long" tests
Old             -- directory containing "old" (TST) tests

To run Singular test(s):
------------------------
1.) Put the Singular binary in this directory
2.) cd to one of the directories above
3.) run 
     ../regress.cmd [*.tst] [*.lst]
    with the desired tst (i.e., Singular test scripts) or lst (list 
    of tst files) file(s).
4.) For each file xx.tst, 
      ../Singular -tqr12345678 xx.tst > xx.new.tst
      diff xx.res xx.new.res | tee xx.diff
    is run. 
5.) If no diffs were found, xx.diff and xx.new.res are rm'ed and
    xx.res is gzip'ed and uuencoded. Otherwise (i.e., diffs were
    found), xx.diff and xx.new.res are kept, diffs are printed to
    stdout, and regress.cmd exits with exit status != 0.

Do also 
   ../regress.cmd -h
and see the source code of ../regress.cmd for some more details on
regress.cmd.

Note: 
(1) You should use regress.cmd -g to generate a result file! 
(2) You need the (GNU) uuencode for regress.cmd to work

To unpack a result file (say, xx)
----------------------------------
uudecode -o xx.res.gz xx.res.gz.uu; gunzip -f xx.res.gz

To pack a result file (say, xx)
--------------------------------
gzip -f xx.res; uuencode xx.res.gz xx.res.gz > xx.res.gz.uu; rm -rf xx.res.gz


To add a new test for a library/commando to the test-suite:
-----------------------------------------------------------
* The following files need to be provided:
        short/xx_s.tst : Singular code for short and basic tests 
        long/xx_l.tst : Singular code for long and extended tests
        short/xx_s.res.gz.uu : Result output of xx_s.tst
        long/xx_l.resres.gz.uu : Result output of xx_l.tst
  or, alternatively:
        short/xx.tst: Singular code for short tests, only
        short/xx.res : Result output of xx.tst

* xx_s.tst should test the essential functionality of the
  library/commando in a relatively short time (say, in no more than
  30s).

* xx_l.tst should test the functionality of the
  library/commando in detail so that, if possible, all relevant
  cases/results are tested. Nevertheless, such a test should not run
  longer than, say, 10 minutes.

* If useful tests do generally execute in a short time, have xx.tst,
  only.

* Rules for providing tst files:
  1.) tst files always start with the following three commandos as
  preamble: 
         LIB "tst.lib";
         tst_init();
         tst_ignore("CVS ID $Id:$");
  tst_init() writes some general info to stdout (like date,
  uname, hostname, version, etc.). The library tst.lib (appended
  below) provides, among others, the routines tst_init() and
  tst_ignore(). 

  2.) All system-dependent output (like run-times, memory usages,
  pathnames, dates, etc.) must be output by means of the routine
  tst_ignore(...):
  tst_ignore(val [, keyword]): 'val' can have arbitrary type for which
                               a string conversion exists;
                               if present, keyword must be one of the
                               following strings: "time", "memory"
  tst_ignore() outputs 'val' by prepending the following prefix:
  no keyword --       // ignore: 
  "time" keyword --   // ignore: time:
  "memory" keyword -- // ignore: memory:
  which causes automatic tests to ignore these lines when doing a diff 
  on result files. 
  
  3.) Time and/or memory-intensive tests should be followed by
  (tst_ignore'd) time (resp. memory) output, so that we can keep track
  of the timing (resp. memory) behaviour of Singular. This could be
  accomplished as shown below: 
  int mytime = timer;
  /* time/memory intesive computations */
  tst_ignore("time", timer - mytime);
  tst_ignore("memory", memory(0));

* Command for generating *.res.gz.uu files: 
  cd <directopry where xx.tst should reside>; ../regress.cmd -g xx.tst