Refactoring preprocessor directives

A parser accepts a stream of tokens, and then builds the tree. Proparse's token stream has some token types that get filtered and attached to nodes as hidden tokens.

com.joanju.proparse.DoParse.java:

filter.hide(NodeTypes.WS);
filter.hide(NodeTypes.COMMENT);
filter.hide(NodeTypes.AMPMESSAGE);
filter.hide(NodeTypes.AMPANALYZESUSPEND);
filter.hide(NodeTypes.AMPANALYZERESUME);
filter.hide(NodeTypes.AMPGLOBALDEFINE);
filter.hide(NodeTypes.AMPSCOPEDDEFINE);
filter.hide(NodeTypes.AMPUNDEFINE);

Those are the only 'hidden' tokens that get into the stream. (WS is WhiteSpace)

The other tokens, like &IF, are dropped and don't get into the stream because they themselves change what gets into the token stream.

&IF FALSE &THEN
  blah blah blah
&ELSE
  display 3.
&ENDIF

The "blah blah blah" doesn't get into the stream at all, not even as hidden tokens. Neither does the &IF, &ELSE, &ENDIF. The text is just dropped.

There is, however, another set of data entirely, and that is the "macro source graph", which contains a lot of the data you would go looking at a COMPILE..LIST for, including all the &IF, {&whatever}, and {myfile.i} references. The Java objects in the Macro Source Graph come from this:
http://www.joanju.com/dist/docs/api.html#ListingFile
and those Java objects are in the blob. You can see them in the UML I posted. We haven't started working on the API for those yet.


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

this gives me some problems

this gives me some problems : for example, in 90% of my .w files created in the appbuilder, I have

&IF DEFINED(UIB_IS_RUNNING) &THEN
DEF VAR ip_foo AS CHAR NO-UNDO.
&ELSE
DEF INPUT PARAMETER foo AS CHAR NO-UNDO.
&ENDIF

this allows me to run the .w directly from the appbuilder.

What I think that you are saying is that depending on where UIB_IS_RUNNING is set, either the DEF VAR _or_ the DEF INPUT would be used, and the other dropped.

If I was trying to refactor some other part of the code then I would lose this when writing out the new source ?


john's picture

We don't write out the new

We don't write out the new source directly from the syntax tree.

Our refactoring logic has two phases. The first phase analyzes the syntax tree, and makes a list of files, lines, columns that need changing.

In the second phase, we change the text of the source files, but only the portions of the lines of text that need to be changed. For example:

  1. slurp the entire source file into memory
  2. in the 'in-memory' copy of the text, insert 'NO-UNDO' at line 42, column 30
  3. write the modified text back out to disc