ease retracing of reference test result changes

[kroeker@…]

To provide practicable history retracing of reference result changes, I think it would be good

1. to checkin .res files into the code repository.
2. commits related to .tst and .res files should not be bundled: only one test should be changed per single commit with descriptive message.

################################################################### The simpler solution to achieve this seems to me is not to check in gz.uu files but only .res files:

-completely avoid storing .res.gz.uu and .res.gz files in (git) repo ( current test result files seems not really big ?) If these files are required for something e.g. for distribution targets generate them using 'make' targets in the BUILD directory.

=>regress.cmd should not delete '.res' files during a regular test by default (it does)

################################################################### Alternative approach:

the referenced output should be plaintext in regular case and stored in res.gz.uu only in exceptional cases (what are that cases?)

Therefore -regress.cmd should not delete .res files during a regular test (it does)

-if both files .res and .res.gz.uu exist, regress.cmd should complain in case the content is different.

-When generating new reference output,

keeping intemediate files should be the standard option -the res.gz.uu file should always be generated (that is the case) -at best generating ' res.gz.uu' and '.res' should be an atomic operation (either generate both of them or none of them)

Bad scenarios: res.gz.uu is not part of the repo but resides in folder, .res is part of repo and differs with extracted res.gz.uu Avoid by: -always checking both files in

not modifying .res* files by Makefile or by hand

###################################################################

Comments, alternative suggestions?

[hannes]

No:

• results of tests are usually really large (one gives all the details)
• results of (correctly written) tests are usually very stable (the mathematics does not change, does it?) (i.e. changes happen at least one order of magnitude less often then source files)
• if they change, the change is usually large (i.e. typically > 50 % of the file)

Therefore results files will be compressed if not in use. And if the test is correct, the content of these files is not needed and only uses space. If the test fails, the content is/may be interesting and is kept.

If one changed feature/implementation results in several changed results of test - why seperate them artificially?

[kroeker@…]

With your permission, I would appreciate a briefly discussion at the internal monthly Singular meeting before a final refusal, since I believe that the issue has major priority.

Replying to hannes:

• results of tests are usually really large (one gives all the details) and only uses space.

They aren't. 4 MB is not large today.

• results of (correctly written) tests are usually very stable (the mathematics does not change, does it?) (i.e. changes happen at least one order of magnitude less often then source files)

But they do change and this changes should be easily retraceable.

• if they change, the change is usually large (i.e. typically > 50 % of the file)

Then the test is not well-designed.

If the test fails, the content is/may be interesting and is kept.

My objections are not about debugging, but about retracing and reviewing changes. If this is not necessary (it is) , why use a versioning system for reference results at all?

If one changed feature/implementation results in several changed results of test - why seperate them artificially?

I agree, in that case it should not be done, only unrelated multiple changes should not be checked in by a single commit.

[kroeker@…]

good news, it is not necessary to check in plaintext '.res' files to easily retrace their changes.

Instead we could store '.gz' instead of '.gz.uu' files and use the following setting in

.git/config :

[diff "resgz"]
textconv = gunzip -c 

.gitattributes :

*.res.gz diff=resgz

to compare '.res.gz' files locally (using 'git diff')

It could be even possible to keep everything as is of someone manages to rewrite the rules for .res.gz.uu extension ( I failed )

I'm not sure if a similar hook is supported online at github (will ask on StackOverflow and the github support)

[Oleksandr]

ok, i will try to find something for gz.uu ...

[Oleksandr]

Ok, here is what you use in .git/config as [diff "gz.uu"]:

[diff "gz.uu"]                                                                                                                               
  textconv = "myf(){ A=`mktemp`;uudecode -o \"$A\" \"$@\" && zcat \"$A\" && rm \"$A\"; }; myf"                                               

.gitattributes should contain of course the following:

*.gz.uu diff=gz.uu

ps: zcat may be replaced with gunzip -c AFAIK