What needs to be fresh?

I have a couple of tools now which were designed to be used on a single compile unit (they are bundled with ABL2DB presently, but I will be documenting and making them separately in the not too distant future). This is helping with James' code base because, when he finds that Proparse has a problem with a particular compile unit, he can just cycle through commenting out blocks of code and running this tool until it parses successfully and thus isolate the problem code (FWIW, at the moment these seem to mostly do with the LIKE and bad table names). But, identifying the problem compile units is a bit tedious since these only show up in a run of ABL2DB and, while there are possible shortcuts, this takes a while. The big problem is that it will report issues with several compile units and then just silently fail altogether, so one has to keep doing it over and over again.

So, I thought I would write a little scanning routine that would run through specified directories and attempt to parse compile units and report back any which failed. I would like to optimize the chances of this making it through the full set of code in one pass so that we don't have to keep running it over and over.

With that goal in mind, should I be creating a new Environment and Schema for each compile unit. I wouldn't think these would get corrupted, but since I am creating a new ParseUnit for each compile unit, it seems like either ParseUnit is failing in such a way to bring the whole session down or the Environment or Schema are getting corrupted somehow. If it is the former, I may be stuck because I won't have a way to determine where the problem is.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
tamhas's picture

Preliminary results on James

Preliminary results on James code is that the scanner completes without Proparse errors. This means one of four things:

1. Fetching the Program_root is not an adequate test that the parse has completed.
2. The failure has to do with contamination which is avoided by the fresh restart on each compile unit.
3. I'm not testing what I think I am.
4. James has fixed all the things which were causing Proparse to hang.

He will be trying a run in the context of ABL2DB, but it would be handy to have input on whether I should be using a different test.

tamhas's picture

He is still getting null

He is still getting null pointer exceptions, so the question seems to be, do I need to access more of the AST in the scanner to find the problems or is it the fresh start on every compile unit which is avoiding them?

john's picture

Calling TreeParser01 is the

Calling TreeParser01 is the whole thing. That's all you should need to do in order to test that Proparse can cleanly parse the code.
I can't imagine why reloading the environment would make a difference.

tamhas's picture

So, my scan routine check

So, my scan routine check for the compilability of the proposed compile unit by actually compiling it and looking for errors and checking for the XREF file and then calls a separate class whose core is:

method public void Initialize():
/* Initialize for Proparse */
if chSchemaDirectory = "" then chSchemaDirectory = session:temp-directory.
if chSchemaDumpFile = "" then chSchemaDumpFile = "proparse.schema":U.
chSchemaDumpFile = chSchemaDirectory + "/" + chSchemaDumpFile.

obSetEnvironment = new SetEnvironment().

obSetSchemaFromFile = new SetSchemaFromFile().
obSetSchemaFromFile:chSchemaDirectory = chSchemaDirectory.
obSetSchemaFromFile:chLogicalDataBaseList = chLogicalDataBaseList.
obSetSchemaFromFile:chAliasList = chAliasList.
obSetSchemaFromFile:lgDeletePPSchemaFile = false.

end method.

method void DoParse():
define variable obJavaFile as java.io.File no-undo.
define variable obParseUnit as ParseUnit no-undo.
define variable minWhich as integer no-undo.

obJavaFile = new java.io.File(chCompileUnitToScan).
obParseUnit = new ParseUnit(obJavaFile).

if NodeTypes:getTypeName(obParseUnit:getTopNode():GetType()) <> "Program_root"
then log-manager:write-message(substitute("ScanOneCU: TopNode not Program_root in &1", chCompileUnitToScan)).

But, this process is producing no null pointer exceptions. But, we do get them in the normal processing. In the normal processing the compile is separated and the compilability is limited to the test for an XREF, although my understanding is that a failure to compile means no XREF. The other difference is that the environment and schema are established once and only the ParseUnit is unique to each CU.

This makes it sound like the Scanner test is valid and the suspect is a fresh copy of Schema and Environment for each parse attempt.

tamhas's picture

OK, I have now restructured

OK, I have now restructured the two parts of ABL2DB which use Proparse such that, for each compile unit to be processed, it creates a fresh class instance with Environment and Schema, does the parse, and then this object and the Proparse dependent objects are deleted. At least the first of these has now run to completion, suggesting that the null pointer exceptions we were getting were a result of the environment or schema getting corrupted. In the old approach, the environment and schema were created once for the whole session. Now, they are created fresh for each compile unit.

tamhas's picture

Apropos of which, is: if

Apropos of which, is:

if NodeTypes:getTypeName(obParseUnit:getTopNode():GetType()) = "Program_root".

A reasonable test that obParseUnit:treeParser01() has completed successfully?