This example demonstrates an advanced use of QGEN as a program generator to build QTAB specs from a template. The specs shown were adapted from a real project designed for a media research firm conducting a bimonthly tracking study for a major service magazine. The entire project was far more complex than the sample shown here and both text and data have been changed to conceal the subject matter and proprietary rating methodologies used in the original.
One set of measurements compared ratings for each article in an issue against norms consisting of average rankings over the previous 6 months for all of the articles in each of one or more matching categories. While the general structure of the output remained the same, the changing nature of the contents for each period meant that a large number of table definitions had to be rewritten each time.
The solution illustrated here was a QGEN program that generated a QTAB spec file containing those definitions that changed by inserting the unique information for each new period into templates. The modular specification file capabilities of QUIP software allowed a small file to be included into a larger QGEN spec file, generating output that in turn could be included into the final QTAB spec file. While a substantial amount of labor went into the initial setup and testing, each successive period could then be run from a batch file after modifying a few dozen lines in one small QGEN spec file.
In the original project, there were hundreds of tables produced for each period requiring several QGEN runs to normalize the data along different dimensions and to apply weights, along with different QTAB runs on each of the generated data files. But once the initial spec and batch files had been set up, the total amount of work required to process all tables for each new period was about the same as for the single table shown here.
This project had typically taken 3-4 weeks to run each period using our client's previous cross tabulation package. After the initial setup, automating the process with QUIP software cut the turnaround time to under two days for each period thereafter.
Note that this project predated some of the automation features now included in QUIP which allow writing generic specifications that can be changed by modifying entries in a substitution dictionary. While the dictionary approach would allow the project to be entirely run with QTAB alone, it can be relatively difficult to implement the variable layout of tables seen here. Generating table stub specs with QGEN may be more work to set up initially, but it is easier for an end-user to run on a continuing basis.
The listing that follows contains the entire file of information unique to the current period. This is all that needs to be modified in order to tabulate each new period.
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * RTSG1-25.QGS - 04/18/01 - REV. 2 - Codes and article titles for period 25 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * The following 5 instructions must be updated for each new period 261 mov 'January 1991 Issue' Current issue 301c2@'25' all Current period 303c2@'20' all Beginning period for norms 305c2@'25' all Ending period for norms 307c2@'09' all Number pages shown this issue * Page/Category/Title table - This changes for each period - Period 91-25 * ------------------------------------------------------------------------- * Argument = 2 digit number | Function = Page # (3 chars) * | 2 spaces (for ratings in GEN5) * | JW category codes (3 x 2 chars) * | Article title (42 chars) *-------------------------------------------------------------------------- TABLE jwcats A=2 F=53 E=20 01 028 16 Do It Yourself - A New Kitchen 02 040 0513 Vitamins That Help You Stay Thin 03 060 2935 Kitchen Primer: Rice That Stays Fluffy 04 070 1358 He's a Crown Prince and he Cooks Too! 05 078 3035 Mediterranean Fish Soup in 30 Minutes 06 090 040613The New Cooking New Guide to Europe 07 102 3135 Quick and Easy Winter Squash Soup 08 105 5813 The First Lady's Favorite Meals 09 114 0613 New England Vacation Spots END
The file shown above is called as an include by QGEN spec file rtssgen1.qgs.
The output data from this QGEN run is, in turn, called as an include by QTAB spec file rtsstab1.qts, generating the final sample rating table for this period.
Note that the period code (25) is passed on the command line and is used to identify both the include files for a specific period and the names of all spec listing and output files. This allows the entire job to be run from a two line batch file:
qgend rtssgen1.qgs /# PERIOD 25 /# qtabd rtsstab1.qgs /# PERIOD 25 /#
The specs, data, listings and output from this example are in rtssdemo.zip.
Here are the other files and listings from this run for those who prefer to view them online:
rtssgen1.qgs shows the calling QGEN spec file.
rtssg-25.lst is the full spec listing from this QGEN run.
rtst2-25.qts is the output data from the QGEN run. It contains all of the QTAB specifications unique to the current period needed to run the sample table.
rtsstab1.qts is the calling QTAB spec file. It does not change between periods.
rtsst-25.sl1 is the full QTAB spec listing from running the above.
rtsst-25.tb1 is the sample table produced by this whole exercise.