This book will teach you how you can customize Prolint.
Traditional situation:
Up until Prolint release 73, Prolint supports two groups of rules: standard rules and custom rules. Standard rules are installed as part of the Prolint distribution. Custom rules don't come from the Prolint website; a custom rule is created by the Prolint user and never committed to the Prolint website and hence not available to other users.
When someone wants to share their custom rules, they have to be accepted by project admin (=Jurjen) and then wait until the next Prolint release.
New: Contributed rules
In addition to standard rules and custom rules, a new layer "contrib rules" was added. Everyone can contribute their rules to the website and everyone can decide to download any number of contributed rules. Contributed rules are not installed as part of Prolint distribution, unless a contributed rule is "upgraded" to a standard rule.
This is more attractive for people who have custom rules to share, because these rules do not have to wait for acceptance and are published immediately. It is also attractive to Prolint users to be able to choose from a public collection of custom rules, and it is attractive for myself because I won't have the burden to accept/review/reject rules.
Subversion:
In te subversion repository for prolint, a path "/trunk/contribs/rules" is added and user "guest" is authorized to read/write in this path. The password for "guest" is no secret: "OEHive", so anyone can commit their rules to this path. The guest user is not authorized to commit anything outside the "/trunk/contribs/rules" path.
When you commit something in the "/trunk/contribs/rules" as "guest" user, please please please write your real name in the subversion commit note.
Example: suppose you have a rule named "acme.p" that you want to contribute, your own name is "Willy E. Coyote" and you have a subversion command-line client installed on your pc. The full command to upload your acme.p rule would be:
svn import c:\p4gl\tools\prolint\contribs\rules\acme.p svn://oehive.org/prolint/trunk/contribs/rules/acme.p --username guest --password OEHive -m "Dont walk through a painted image of a cave. Rule contributed by Willy E. Coyote"
Personally I hardly ever use the subversion command-line, but prefer the TortoiseSVN program that nicely integrates with Windows Explorer. You can download the command-line tools or TortoiseSVN from http://subversion.tigris.org
Definitions:
Properties for normal rules are defined in a texfile named "rules.d" but that is simply not practical for contributed rules, because it would mean that users would have to download a new rule and also merge a new line into their existing textfile. We have a better solution. The sourcefile of a contributed rule contains its own properties as a structured comment in the sourcefile. Please have a look at existing contributed rules to see working examples. One warning: only the first comment in the sourcefile is parsed.
Help:
Someone who contributes a rule should also add a help page, which is a child-page of http://www.oehive.org/prolint/rules
That help page should describe the purpose of the rule and should also identify the rule as a contributed rule that can be downladed from the subversion contribs path.
In addition, the help page should be attached to the "contributed rules" keyword, so all contributed rules can be easily listed.
Here is a mini-manual how to write a help-page: http://www.oehive.org/node/1227
Shop for rules:
Visitors will be able to discover contributed rules in several ways:
Search order:
Prolint will search rule definitions in the following order:
1. prolint/contribs/rules
2. extend and override the results from (1) by prolint/rules
3. extend and override the results from (2) by prolint/custom/rules
Execution order:
Prolint will search rule code in the following order:
1. prolint/custom/rules
2. when not found, search the rule in turbolint
3. when not found, search the rule in prolint/rules
4. when not found, search the rule in prolint/contribs/rules
How the "prolint/custom" directory is used by Prolint:
When Prolint executes a rule, it will search the rule in this order (regardless if the rule was listed in "prolint/custom/rules/rules.d" or not):
When the Prolint Results Window searches for help on a specific rule, it will first search in directory "prolint/custom/help/rules" for a HTML file and if it doesn't find it, then it will browse to The OpenEdge Hive for on-line help.
Each rule is a non-persistent procedure located in directory prolint/rules. A rule will be invoked by prolint.p if it is listed in file prolint/rules/rules.d.
An easy way to insert a new record in this file, is to run dialog
prolint/core/dnewrule.w.
This dialog can be invoked from the Prolint Desktop window.
The file contains the following fields per record:
Set as many logicals to "no" as possible, this saves performance when the end-user doesn't run many rules.
Each rule should have help.
For standard rules (rules that are deployed with Prolint) these are topics at oehive.org. Simply create a book page as child of oehive.org/prolint/rules and it will work.
For custom rules (that you keep privately) you can create a .htm file in directory "prolint/custom/help/rules", see custom help for more info.
Run prolint/launch/test.p often, to see if rules behave as expected.
See topic "regression testing" for more details.
It is often convenient to write test-cases in directory prolint/regrtest before starting to program your new rule, so you have immediate feedback while programming.
If you add test-cases please send them to me so they are included in the next Prolint release.
File "prolint/rules/_template.p is a template for a new rule. You can use it as a starting point, if you don't want to start from scratch.
Each rule needs a couple of input parameters, which are defined by {prolint/core/ruleparams.i}.
I hope these parameters don't need introduction, although parameter "pragma_number" may be a stranger:
Pragmas are used for suppressing warnings. If the sourcefile contains a directive like {&_proparse prolint-nowarn(rule_id)}, then every node in the statement following the
directive will get an attribute. The identifier of the attribute is an integer named "pragma_number", the value of the attribute is 1 (to indicate that the attribute is "set").
Each rule_id has its own unique pragma_number. The procedures for searching through proparse nodes will skip each node where this pragma-attribute is set.
When the rule finds a statement or a situation where it wants to raise a warning about, you should run procedure PublishResult.
Procedure PublishResult is implemented in prolint/core/lintsuper.p which is
a super-procedure to every rule.
You should not
"publish" the warning directly to the outputhandlers using PUBLISH "Prolint_AddResult", because if you do you miss the extra functionality that lintsuper.p adds to it.
Searching nodes in the parsed source tree.
There are basically two ways to query nodes in the tree created by proparse:
Both procedures have identical parameter signatures, so they can be exchanged at any time - just pick the fastest one. Alternatively you can run
searchNode (also in lintsuper.p) which will choose to run either searchNodeTree or searchNodeQueries for you.
Parameters for searchNode* procedures:
Prolint release 63 runs in Progress 8 and in Progress 9. Prolint release 64 (and up) does not run in Progress 8 anymore.
When you create a custom Prolint rule you may also want to provide a Help file.
When your custom rule has id "xyz" then simply drop a file "xyz.htm" in directory "prolint/custom/help/rules".
When no custom help file is found, Prolint will try to open topic "xyz" at oehive.org.
This snippet from prolint/outputhandlers/logwin.w explains why:
FILE-INFO:FILE-NAME = "prolint/custom/help/rules/":U + pContext + ".htm":U.
IF FILE-INFO:FULL-PATHNAME<>? THEN
fullpath = file-info:FULL-PATHNAME.
ELSE
fullpath = "http://oehive.org/prolint/rules/":U + pContext.
RUN prolint/core/openhtml.p(fullpath).
I think this is a simple yet effective method, but differs greatly from Prolint release 65 and earlier.
In Prolint <66 all help files were deployed as static htm files in directory "prolint/help", and for custom help you would write ".htxt" files in the "prolint/custom/help" directory and use the "Rebuild help" button to merge those files into the static htm files.
The reason why I prefer to open a live page at oehive.org, is because of its "open" spirit: you and everyone else can easily add comments to the help topics for everyone's benefit and improve the existing help.
Filters let you modify or delete warnings.
Directory "prolint/filters" contains 4gl procedure files, each .p file in this directory is a filter.
Each time when any rule creates a warning, the warning will first go through each filter before it is published to each outputhandler.
A filter can modify the description or severity fields, or it can mark the warning as 'filtered' in which case the outputhandler will probably ignore the warning.
Each filter has the same API which makes it easy to add your own custom filters. You can use the filters/_template.pp file as a boilerplate for creating a new filter.
The Prolint distribution already contains a couple of standard filters:
Directory prolint/settings or each of its subdirectories can contain a nowarn.lst file. In this text-file you can specify
a list of warnings you want to suppress.
This is convenient for sourcefiles (especially includefiles) you can't modify to add _proparse_ directives, like third-party includefiles
or src/adm2, for example.
For more details about the nowarn.lst file, see example file
Each profile directory (prolint/settings/profilename) can also contain an exclude.lst file.
This file works similar, but slightly different from nowarn.lst
exclude.lst works with file patterns with wildcards, both for filenames and rulenames. For example you can specify that all rules except rule "backslash" must be ignored in all files matching "third-party/*.i"
For more details about the exclude.lst file, see example file
Some rules must ignore warnings caused by code in AppBuilder-generated sections. Filter "ignoreab" figures out if a line is inside an ab-generated section.
An outputhandler must be located in directory prolint/outputhandlers, and has to be listed in
file prolint/outputhandlers/choices.d.
Columns in file prolint/outputhandlers/choices.d:
| "GUI" | this outputhandler will only run in a GUI session |
| "TTY" | this outputhandler will only run in a ChUI session |
| "*" | this outputhandler will run in any window-mode |
| "" | this outputhandler will only run in batch-mode |
Each outputhandler is loaded persistently by prolint.p, but prolint will never delete the procedure. The outputhandler has to delete itself when ready,
usually from within the event-procedure for event "Prolint_FinalizeResults".
The outputhandler subscribes to, and responds to (some) of the following published events:
PROCEDURE Prolint_InitializeResults : DEFINE INPUT PARAMETER pClearOutput AS LOGICAL NO-UNDO. END PROCEDURE.
Event Prolint_InitializeResults is published when prolint.p starts.
This is the right moment to create an output stream or XML-document or temp-table, or whatever kind of output you want to create.
pClearOutput=TRUE indicates that the existing output (logfile) needs to be emptied or that a new output stream (logfile) should be started.
pClearOutput=FALSE indicates that new lint-results should be appended to existing output (logfile).
PROCEDURE Prolint_List_Rules : DEFINE INPUT PARAMETER pRuleList AS CHARACTER NO-UNDO. /* comma-separated list of ruleid */ END PROCEDURE.Event Prolint_List_Rules is published to let you know which rules are selected in the current profile. This is a comma-separated list, not including the semi rule-id's used for system warnings like "prolint,proparse,compiler". This event is published once, after InitializeResults and before the first Status_Filestart.
PROCEDURE Prolint_Status_FileStart : DEFINE INPUT PARAMETER pSourceFile AS CHAR NO-UNDO. END PROCEDURE.Event Prolint_Status_FileStart is published to notify you when prolint starts to work on a new sourcefile, or actually on a new compilation-unit.
PROCEDURE Prolint_AddResult : DEFINE INPUT PARAMETER pCompilationUnit AS CHARACTER NO-UNDO. /* the sourcefile we're parsing */ DEFINE INPUT PARAMETER pSourcefile AS CHARACTER NO-UNDO. /* may be an includefile */ DEFINE INPUT PARAMETER pLineNumber AS INTEGER NO-UNDO. /* line number in pSourceFile */ DEFINE INPUT PARAMETER pDescription AS CHARACTER NO-UNDO. /* human-readable hint */ DEFINE INPUT PARAMETER pRuleID AS CHARACTER NO-UNDO. /* defines rule-program and maps to help */ DEFINE INPUT PARAMETER pSeverity AS INTEGER NO-UNDO. /* importance of this rule, scale 0-9 */ END PROCEDURE.Event Prolint_AddResult is published when prolint wants you to add something to the output (logfile).
PROCEDURE Prolint_Status_FileEnd : END PROCEDURE.Event Prolint_Status_FileEnd is published to notify you when prolint is done with the sourcefile that was earlier published in event Prolint_Status_FileStart.
PROCEDURE Prolint_FinalizeResults : END PROCEDURE.Event Prolint_FinalizeResults is published when prolint is ready.
PROCEDURE Prolint_Status_Action : DEFINE INPUT PARAMETER pAction AS CHAR NO-UNDO. END PROCEDURE.Event Prolint_Status_Action is probably only useful for logwin.w; parameter pAction is the text to show in the second panel of the statusbar.
PROCEDURE Prolint_Status_Profile : DEFINE INPUT PARAMETER pProfile AS CHAR NO-UNDO. END PROCEDURE.Event Prolint_Status_Profile sends you the name of the configuration profile that is used by Prolint.
PROCEDURE Prolint_Status_StopTimer : END PROCEDURE. PROCEDURE Prolint_Status_StartTimer : END PROCEDURE.Events Prolint_Status_StopTimer and Prolint_Status_StartTimer are used for finding out how long it takes to run rules: event Prolint_Status_StopTimer is published before compilation and parsing begins, Prolint_Status_StartTimer is published when compilation and parsing are ready which is also the moment that the loop "for each rule: run value(rule)..." begins.
If you have created a rule you should also create an help page for it. Your rule will probably warn progammers that some code statement is not quite good, and then those programmers will probably wonder why you think it is not good and how they can solve it.
These are the steps how to create a help page:
1. go to http://www.oehive.org/node/11
This page, simply titled "Rules" is the parent page of all help pages for rules. The page contains not much but an alphabetic list of rules, and at the bottom there is a link "Add child page". That's the link we need: click it. You will now get to see an input form for a new page.
2. in field "title" you should enter the rule-id. That is, when your rule is "memoryleak.p" then the rule-id is "memoryleak" so the title of the new page is also "memoryleak". All lowercase.
3. field "navigation" : do not select anything please.
4. field "keywords" : you dont need to enter anything except when the rule is a so-called "contributed rule". In that case please enter "contributed rules"
5. field "Body": here you enter anything you want to explain the rule to Prolint users. What I usually do is: explain what the rule is trying to detect, explain why the detected code is considered bad, do some suggestions how to fix it.
6. field "audience" (the list of toggle-boxes for groups):
please please please select the "Prolint" group!!
Unfortunately you can only select the "Prolint" group when you sourself are a member of that Prolint group, so you might have to become a member first.
7. field "URL path settings"
Here is some magic.... enter "prolint/rules/" + rule-id.
So for example: if the rule-id is "memoryleak" then enter "prolint/rules/memoryleak"
This ensures that the "Help" buttons in Prolint can browse to your help page.
8. Press the "Submit" button.
That's all, you have now created a help page for a rule!
All tests are performed by "rules", these are programs in directory prolint/rules.
This is where the actual knowledge about Progress source code review is implemented.
Each test is implemented in one rule, one rule implements one test.
All output is handled by outputhandlers, these are programs in directory prolint/outputhandlers.
An output handler receives results from rules and writes them to something else: for example to a window, to a
textfile in arbitrary format, to a pipe, to a database, whatever.
Customizing prolint will most likely mean: adding new rules and/or outputhandlers. If you do, please submit your customizations to
this Prolint Open Source project.
The invocation of the rules is coordinated by prolint/core/prolint.p.
You could basically summarize prolint.p to something like:
for each outputhandler:
run value(outputhandler) persistent.
/* each outputhandler subscribes to prolint messages */
end.
for each sourcefile:
for each rule:
run value(rule) (input sourcefile).
/* each rule publishes prolint messages */
end.
end.
prolint.p is designed to run silently: it does not ask questions, it does not display anything.
It gets all its information from input parameters and configuration files, it sends all its output to outputhandlers using publish/subscribe.
Input parameters for prolint/core/prolint.p are gathered by prolint/launch/start.p, which has no input parameter by itself.
prolint/launch/start.p invokes dialog prolint/core/selectfiles.w where an end-user can select files in a GUI dialog.
If you press button "Lint files" in the Results window, you actually
just run prolint/launch/start.p. If you would run the dialog directly, the dialog
would stay visible during the execution of prolint/core/prolint.p.
Variations of prolint/launch/start.p are no-brainers, see for example
prolint/launch/test.p which tells prolint.p to run a regression test on itself.
An outputhandler is a persistent procedure, launched by prolint.p, that receives information from prolint.p or from rules, because it is subscribed to
messages that are published by prolint.p and by rules. An outputhandler typically writes messages to a logfile. Different outputhandlers
can be used to write different file formats, even simultaneously. The Prolint results window (prolint/outputhandlers/logwin.w) is really just an outputhandler.
Because most rules do basically the same thing in different
variations, they can all use a set of ip's in prolint/core/lintsuper.p.
Lintsuper.p is attached to each rule as a super-procedure, although it is not constructed such that lintsuper.p is
a base ancestor rule-class where rules are inherited classes - it's better to regard lintsuper.p as just a regular support library where the
'super' option is used for programming pleasure.
When you add of change something to any Prolint source, you can do a regression test to see if you broke something.
Before you can run the regression tests, you will first need to create a database named "prolintest.db" in directory "prolint/regrtest/db". A schema definition file for this database is provided: "prolint/regrtest/db/prolintest.df". Simply create a new database from empty and import this df file. This database does not need any data, areas are not important, it just needs to exist so the test code can compile. Prolint will automatically connect to this database in single user mode, and will automatically disconnect when the regression test is over.
Now, when the prolintest database is created, you can launch the regression test using:
RUN prolint/launch/test.p
In the end you should see a window containing the following text:
Comparing files prolint.log and EXPECT.LOG FC: no differences encountered
This means that everything is still working as expected.
How it works:
prolint/launch/test.p launches a Prolint session that checks every sourcefile in directory "prolint/regrtest" and, when you are using OpenEdge 10, also directory "prolint/regrtest-oo" which contains OO classes.
The warnings from this Prolint session are logged to a text file (prolint.log) and this text file is then compared to the expected warnings in file "prolint/regrtest/expect.log" and "prolint/regrtest-oo/expect.log".
These files are simply proliint.log files from earlier sessions. Simple, but effective.
The split into two directories, regrtest and regrtest-oo, is to make sure that Progress 9 users can also run the regresssion test.
If you add a new rule, it would be great if you also add test-cases for this rule to directory "prolint/regrtest" (or "prolint/regrtest-oo" if it is object oriented code).
run prolint/core/prolint.p(...) analyzes one or more sourcefiles, it is the entrypoint of all parsing.
Because prolint.p requires a couple of input parameters you will probably prefer to run a "wrapper"-procedure that first assigns those parameters and then passes them to prolint.p.
Let's first look at an example "wrapper", the parameters are explained later in this page.
procedure prolint/launch/start.p is an example of a "wrapper", it first runs a dialog where you can choose values for all parameters and then it calls prolint.p.
start.p should work in Progress 9 and OpenEdge 10, GUI or ChUI (and Progress 8 if you have Prolint release 63)
Here is a simple example with no user interface: it scans the contents of directory "d:\myproject" and runs prolint
DEFINE VARIABLE fname AS CHARACTER NO-UNDO.
DEFINE VARIABLE fullpath AS CHARACTER NO-UNDO.
DEFINE VARIABLE attribs AS CHARACTER NO-UNDO.
DEFINE TEMP-TABLE tt_files NO-UNDO
FIELD SourceFile AS CHARACTER.
/* find all sourcefiles in directory */
INPUT FROM OS-DIR ("d:\myproject").
REPEAT:
IMPORT fname fullpath attribs.
IF NOT (attribs MATCHES "*D*") THEN
IF (fname MATCHES "*~~.p") OR (fname MATCHES "*~~.w") THEN DO:
CREATE tt_files.
ASSIGN tt_files.SourceFile = fullpath.
END.
END.
INPUT CLOSE.
/* now lint all of them, send output to prolint.log (=profile "batchrun") */
IF CAN-FIND(FIRST tt_files) THEN
RUN prolint/core/prolint.p ("",
THIS-PROCEDURE:HANDLE,
"batchrun",
TRUE).
/* You could simply pass the temp-table handle, but
* that doesn't work in Progress version 8:
*
* IF CAN-FIND(FIRST tt_files) THEN
* RUN prolint/core/prolint.p ("",
* TEMP-TABLE tt_files:HANDLE,
* "batchrun",
* TRUE).
*/
RETURN.
PROCEDURE GetFirstLintSource :
/* purpose: prolint.p calls this ip to ask for the first sourcefile to analyze.
return ? if you don't have any sourcefiles. */
DEFINE OUTPUT PARAMETER pSourceFile AS CHARACTER NO-UNDO.
FIND FIRST tt_sourcefiles NO-ERROR.
IF AVAILABLE tt_sourcefiles THEN
pSourceFile = tt_sourcefiles.SourceFile.
ELSE
pSourceFile = ?.
END PROCEDURE.
PROCEDURE GetNextLintSource :
/* purpose: prolint.p calls this ip to ask for the next sourcefile to analyze.
return ? to stop */
DEFINE OUTPUT PARAMETER pSourceFile AS CHARACTER NO-UNDO.
FIND NEXT tt_sourcefiles NO-ERROR.
IF AVAILABLE tt_sourcefiles THEN
pSourceFile = tt_sourcefiles.SourceFile.
ELSE
pSourceFile = ?.
END PROCEDURE.
|
Note that the above example is just an example, an easier way to get almost the same result is:
RUN prolint/core/prolint.p ("d:\myproject",
?,
"batchrun",
TRUE).
|
Two differences with the first example:
1. Prolint will scan directory d:\myproject and all its subdirectories
2. Prolint will not only look for .p and .w extensions, but all extensions specified in prolint.properties.p
input pSourcefile (as character)
The filename of a sourcefile you want to lint. This parameter is convenient if you want to lint just one single sourcefile.
If you want to lint multiple sourcefiles it is recommended to set SourceFile="" and use hSourcefileList instead.
pSourcefile can also specify a directory. In that case Prolint will search every compilation-unit in that directory and all its subdirectories recursive. Compilation-units are recognized by file extension; the list of file extensions is "*.p,*.pp,*.w,*.cls" by default but you can change this using the prolint.properties.p file.
pSourcefile can also specify a comma-separated list of sourcefiles and/or directories.
input hSourcefileList (as handle)
The handle to a procedure or to a temp-table, or the unknown value. Set this parameter if you want prolint to analyze multiple sourcefiles in a single run.
If it is a PROCEDURE:HANDLE, prolint requires that this procedure contains two internal procedures: procedure GetFirstLintSource(output pSourceFile) and procedure GetNextLintSource(output pSourceFile). Prolint runs these internal procedures in hSourcefileList until one of them returns the unknown value.
If it is a TEMP-TABLE:HANDLE, then prolint assumes this temp-table contains one or more records, where each record contains the name of one sourcefile in a field with fieldname="SourceFile".
input pCustomProfile (as character)
The name of a profile, specifying which rules and outputhandlers to run.
input pClearOutput (as logical)
Specifies if you want to append warnings from Prolint to existing output, or replace existing output by new output.
if pClearOutput=TRUE, the outputhandlers will overwrite existing output.
if pClearOutput=FALSE, the outputhandlers will append to existing output.