Data Management to the Rescue?

Kathy Lang checks out a flexible new CP/M package.

Regular readers will know that many of the packages I’ve reviewed in this series have particular areas of strength that make them well suited to certain areas of data management. This month’s offering, a British package called Rescue, which comes from Microcomputer Business Systems and runs under CP/M, is a general-purpose, menu driven data management package which has much in common with others in this field. But it has unusually flexible provision for different types of information, and its data validation is among the best I’ve seen.

Rescue comes in three parts: the first deals with configuring the system for your computer, and is not needed again unless you make major changes. The second part covers the creation and amendment of the data files, and of the screen and report formats, while the third permits record amendment and display. This separation makes it easy to set up a system in which most users have access to the information in the files, but cannot change the format of those files or interfere with any provision for protecting parts of the data for security reasons.

Data is stored in fixed-length records in Rescue, but some ingenious methods are used to keep data storage to a minimum – I’ll say more about that later. Once you’ve set up a record format, you can still add fields to the end of the records, but you can’t change the sizes of existing fields unless you’ve made provision for that in advance. (MBS is apparently about to release an option to permit more radical changes to existing files, but it isn’t available yet). You can access the records in two ways. Individual fields may be used as keys, and any one of them used to access a particular record for display and/or editing. You can also select subsets of the data by setting up a set of selection rules, which are used to extract a set of records for browsing on the screen or for printing. You can set up as many screen and report definitions as you please for any set of data; these definitions need describe only a few fields in a record if necessary, and any or all of these descriptions may be password protected.

Rescue is used through menus, but users can set up their own menus through quite simple procedures. Thus you can set up a series of operations to be activated by one menu option. You can’t at present access one file from another, so that the current version of Rescue does not have true database capabilities.

Constraints

Figure 1 shows the major constraints imposed by Rescue. The maximum record size of 1024 is the same as several others I’ve reviewed, but Rescue’s dictionary capability makes it more economical of data storage than many.

Some people will find the limitation of 60 characters in a field more serious. I haven’t included in the figure a full list of the field types allowed, as it is very lengthy. Virtually any kind of data format can be expressed with one of the field types provided. I’ll say more about them in the next section.

File creation

The process of file creation is shown in Figure 2, which is a ‘road map’ of all the menus associated with the data definition part of Rescue.

The first stage in file creation involves setting up a data description file, specifying the basic format of each record and the keys it will have. At this stage you must assign a data type to each field. There are four main groups of data alphanumeric, numeric, date, and dictionary. There are several forms of data type in each group; for instance, character data may be just that and contain any valid ASCII character, or they may be alphanumeric, in which case they may only contain letters or digits and any attempt to enter invalid data will be rejected by the system. There is quite a variety of numeric fields, too, including money (sterling). You can specify that a field is to conform to a mask, to ensure that such items as account references, which often have prescribed formats, are entered in a valid form.

Probably the most unusual type of data is the dictionary field, which permits the person entering data to include only certain values. There are two kinds of dictionary field; a short form, which permits up to 29 characters in total to be used for each field, and a long form, which allows up to 255 entries, each of up to 60 characters. The latter are shared among all the fields in the file, so supposing one has a series of questions each with the same range of answers – for example, answers ranging from Poor to Excellent in a market research survey – you only need one dictionary entry for all the fields to refer to. Each response takes up only one character in the record in the data file for either type of dictionary, so the method is really a way of combining coding with captions for codes.

Every field within the record must also fall into one of four entry categories: mandatory (ie, the field must always have a value), optional (the field may be empty), calculated or display-only. Calculated fields are derived from calculations on constants or on other fields in the same record. Display-only fields are provided so that for certain modes of access fields can be shown but not altered – account numbers might for instance be protected in this way. Any field in a record may also be linked to others in a number of ways.

Direct linkage provides for situations where some fields only have values if another field – said to be the controlling field – has a certain value. For instance, details about a property might say if the property were freehold or leasehold but only if it were leasehold would it be sensible to ask for the life of the lease and the annual charge. This approach can also be used to deal with records with lists of information; you might want to store the names of all a person’s children where some people might have as many as six, without asking six questions about childless people. Most packages expect you at least to hit one key for each question when entering data from the keyboard but with the Rescue approach entry can be more finely tuned to stop
prompting for answers if they are not needed.

During file definition you must also specify the fields which are to be used as keys. Rescue treats the key field which is physically nearest to the beginning of the record as the main key, in that you have to ask specifically for other keys when you come to access the file; so it can save a little time to think about what order to store fields in the record. Up to 10 fields may be defined as key fields. Keys may be either unique or duplicate, and Rescue checks when supposedly unique key values are entered. All the key fields are referenced from a single index, which is automatically kept up to date when data is added or amended.

The next step is to define screen and print formats for the records; you can have as many of these as you wish, and each may describe only parts of the record – for instance, to prevent confidential information being seen by everyone. Next, you tell Rescue to set up an empty data file and structure the index file, and finally you construct any custom-defined menus you will need If you do specify more than one screen or report definition, then you will have to do some customisation of the menus in order to use the alternative formats, but this is quite a straightforward process.

Input and editing

The provisions for data validation given by the dictionary facilities, by the variety of data types and by the range checking which can also be set up at file definition time, are extremely powerful – it’s always possible to get the data wrong in a logical sense, but Rescue makes it quite hard to get it wrong in any other sense. That said I did find the mechanics of correcting data a bit clumsy; if you’ve made a mistake and go back to edit a record you can say where in the record you want the editing to begin but from there you must work sequentially through – you can’t work back up the screen either when entering or editing data. Since the program requires you to have a terminal which can move the cursor left and right, it seems a bit strange not to utilise cursor movement up as well, since no terminal is likely to have horizontal movement but not vertical…

When you retrieve records for amendment, you do so by specifying a particular key value; you can specify the use of any key, but you have to get the value of the first four or five characters exactly right (except that Rescue is ‘case-blind’ in this situation, so it will for instance match Smith and smith). Even when matching exactly on a key value you may retrieve more than one record, as duplicate keys are allowed. But searching for field values within ranges is only possible when you want to look at records, not when you want to change them.

Screen display

I said that you can have several definitions for a single file, so that records can be displayed on the screen in different ways for different users or applications. These screen definitions can be created by copying existing definitions and amending them, but I couldn’t find a way to see what definitions I already had except by going out to CP/M and using the Directory command. Screen layout is specified by giving row and column coordinates for each field you want to display, which I found much more difficult to use than the ‘paint-a-screen’ approach which has become fairly common. The coordinate approach also makes it more difficult to amend the layout, though Rescue does have one provision to make this a little easier by letting you specify a re-ordering of the display without changing the absolute coordinates.

The screen layouts are set up in the ‘definition’ part of Rescue. However, they are invoked from the main part of Rescue, through executing one of the options in the menus shown in Figure 3. Display can be of records specified either by matching one key, or by selection using the selection and extraction procedure which is described later.

Reporting

Rescue uses the same mechanism for printed reports as for screen display, so both are strictly record based. The only provision for aggregated information is totalling of numeric fields. It is possible to force page-breaks when values of particular fields change, but subtotalling is not provided. There is, however, a very flexible facility to interface with Wordstar and Mail/Merge, so it is easy to use them in combination with Rescue to write circular letters and concoct sets of standard paragraphs.

Selection

Rescue provides the ability to select parts of the data file for browsing, printing or further selection. The main method of doing this is to set up a set of selection rules in a file, and then to apply these to the data file to produce another file containing the selected records. The selection rules are very flexible: you have all the usual comparison operators (less than/greater than/equal to/not equal to) and data values can be compared with constants or with the values of other fields in the same record. Rules can be combined to provide ANDing and ORing within and between fields, and these combination facilities together with the NOT operator make it possible to select virtually any combination of values you could need. However, personally I don’t like the need to set up rules in a file, as it is rather cumbersome in practice; if you are using the standard facilities menus you must go to the ‘Maintain Rules’ menu (at the third level of menus), create the rules, then go back to the first level of menus and down to the third level ‘Extract and Sort’ menu to actually extract the records you need. Finally (from the same Extract menu) you can display or print the records that have been found. This provides a sharp contrast to the command language approach, in which one command will extract your records and a second at the same level will display them. However, you could tune the menus in Rescue to avoid some of this ponderousness, so it’s better in that sense than menu systems which you can’t adapt.

While actually comparing fields, upper and lower case letters are regarded as equivalent. You can use wild codes: ? will match any one character, * will match one or more characters. For dictionary fields, the order for comparison purposes is the order in the dictionary, so if you have a set of answers with Poor as the first and Excellent as the last Poor will be regarded as ‘less than’ Excellent even though P comes after E in the alphabet. This is usually what you want and with much coded data would be a very valuable feature.

Sorting

Rescue can sort a data file on up to five fields in one operation; the process is similar to selection and you can also combine selection and sorting to give a sorted extract file. Sorting is either in ascending or descending order, as with selection dictionary fields sort in their dictionary order (Poor before Excellent) rather than in alphabetical or numeric order. In addition ordinary character fields can be given a sort value which is different from their simple alphabetical order. This could be particularly useful where you had fields such as book titles which often have prefix words such as A or The, which you want to ignore for sorting purposes but wish to include as part of the field for printing (In most packages these prefix words must occupy a separate field, which will be empty for titles without a prefix word.)

Calculations

The calculation facilities in Rescue are quite powerful in the input phase, and practically non-existent after that. When you set up a data definition file, you can specify that a field is to be calculated from constants, or from combinations of other fields (including dictionary fields) in the same record. All the usual arithmetic operators are available. After input the only calculation you can request is totalling on printed reports; this is activated by requesting totalling of a field when a description file is set up. Up to 10 fields in any one description file may be set to be totalled.

Security

Protection in Rescue is of two kinds. It is possible to take the programs used in the Define stage off the run-time disk, so that the ordinary user can use file definitions and screen and report formats, but not amend them. At a more detailed level, password protection can be provided for particular data files, for individual description files (so that a user can be given access only to part of the data in a file) or for particular menu items in custom built menus (so that some users may have access to some functions but not others, while other users have greater facilities, but all within one menu). This is a flexible and powerful scheme, and should provide for most needs.

Stability and reliability

I didn’t have any problems over reliability with my use of Rescue. As to stability, new versions of Rescue, which are ‘cost options’, are intended to be compatible with existing versions. New features in the pipeline include a version for MS-DOS and a multi-user version.

Tailoring

As usual, the first task is to tailor Rescue for your particular terminal. This appeared quite straightforward (although, as is the common bad practice, you can’t be sure the tailoring has worked until you actually run the main Rescue suite). However, I had one misunderstanding which I never managed to sort out; this resulted in repeated prompts being printed on the same line as the error messages, which were thereby overlaid so that I couldn’t read the error message. I wasn’t able to discover whether this was an error in the software, the documentation or my interpretation of them and my Sirius manual, but it hasn’t happened to me before. While tailoring for the terminal, you can tell Rescue about cursor movement left and right but not about which keys move the cursor up and down, so much potential editing flexibility is lost.

Once into Rescue, the main tailoring facility is the ability to set up sequences of activities on custom-defined menus. This gets round some of the inflexibilities associated with menu-driven systems, and I found the approach quite easy to use.

Relations with outside

Rescue can write files in standard ASCII characters, using the ‘comma delimited’ format required by many other packages including specifically Wordstar’s Mail-Merge option. Thus you can set up files of information which you want included in circular letters or standard paragraphs, and then fire them off to Wordstar or another similar package.

Within Rescue you can include on a menu the ability to run another program, so it would be possible to tailor a menu to carry out a selection/printing sequence of this kind, called by Rescue ‘record processing’, without the user having to go back to CP/M. You can’t at the moment read external files of ASCII records into Rescue, though there is a menu option to do this already shown, which I’m told will be implemented in the very near future.

User image: software

Once again, your overall reaction to Rescue will be governed by whether you like menu-driven packages or not. I found the ability to tailor menus to provide facilities oriented to particular requirements a big help in mitigating the inflexibilities of menus. However, most users are likely to follow the well-established principle of ‘satisficing (a word coined by Herbert Simon the psycho-economist to describe the tendency to accept adequate or satisfactory results rather than go for the best possible) and only set up extra menus when they absolutely have to, for instance to access alternative screen layouts. So I suspect that mostly people will use the rather cumbersome standard menu facilities. I also had a rather mixed reaction to the complete separation of description of and access to the data files. Within an organisation which has a database administrator (who might simply be the boss in a small business) this could be a useful separation for security reasons, but it would be less helpful where the same person organises the data files and puts information into them, perhaps in a small office, one person business, etc.

Within the package itself, I as usual found some goodies and some nasties. The progress through the menus was orderly and logical and was made straightforward by the provision of the two ‘road maps’ which I show as Figures 2 and 3. The process of prompting was easy to understand. It would have been even easier if, when a question has a default response, this was displayed before the question is posed – in many cases the default is not shown even after you’ve accepted it unless you go back and edit the record concerned. Allowing the use of identifiable abbreviations, both for field names and for data values, is sensible.

I didn’t like the use of row and column coordinates when formatting screen displays and printed reports, especially as there is no default format so you always have to supply one. The ‘paint-a-screen’ approach is much easier in general than coordinate specification and if this is not supplied then there should at least be a default format with records displayed one field per line starting at the left of the screen or paper. I also found the inability to move back within a record when editing a real nuisance.

Documentation

The manual is basically a reference document but written in so much detail that it could be used to teach yourself about the package if you were reasonably familiar with data management terminology. However, the amount of detail makes it rather difficult to find your way around. Two goodies help a little in this: the use of emphasis within the text to call the readers attention to the most important parts of each section, and the printing of chapter headings right-aligned on each page (a real help to browsing at a general level). But the chapter names didn’t always make it easy to guess where a particular feature would be described, and since there was neither a detailed table of contents relating to each chapter nor an index, it was very hard to get from ‘now I’ve seen something about that feature somewhere’ to the exact part of the manual in question. Part of the remedy is close at hand, since if the ‘road maps’ (which perform most of the functions of a reference card) were annotated with the numbers of the sections documenting each menu item, readers would find it very much easier to locate the particular piece of information they need fast (As this article went to press, MBS issued an index for the manual, which should help.)

The other problem I had was that while each feature is documented in detail with examples of the particular feature, there are no examples of the display or use of groups of features. For instance, all the features of data entry are described in turn, but there is no figure showing how data definitions are displayed on the screen. Nothing bolsters a user’s confidence like some complete examples shown in real screen pictures!

I can’t resist ending this section by awarding MBS second prize so far in this year’s contest for manual typo errors, with ‘Data Validification’.

Costs and overheads

Rescue costs £295, and is available from MBS. To be realistic, you would need a disk system with the regular double-sided, double-density capacity of 370 Kbytes per drive on a two-drive floppy disk system, to enable you to have all the Rescue software on one disk drive and use the other for data I found the system very slow in loading individual program modules, which seemed to happen whenever I changed from one sub-menu to another. I was told that this was specific to the Sirius-Z80 card method of disk access, but I haven’t noticed the problem with other packages I’ve used. The times for actually running the Benchtests are shown in Figure 4. (Details of the tests were given in PCW December 1982.)

Conclusions

Rescue provides data management facilities through individual files. Data description facilities are very powerful. Rescue provides a variety of data types and validation features more extensive than any I have found before. These features also help to make Rescue much more economical on data storage than is usual in programs which use fixed length records. You can select and sort the data to provide pretty well any required subset but the process is rather cumbersome. Screen and report formats can be varied according to the needs of particular users, which makes it straightforward to protect particular data items; you can also permit users access only to certain Rescue features. Screen and report formats are described in a rather rigid way, and there are no default formats for easy initial use.

On the other hand, the ability to send data to and run Wordstars Mail-Merge option from within Rescue could be very valuable in some environments. Apart from the calculation features on data entry, the only calculating power within the package is the ability to total particular fields. The system is menu-driven, which can be ponderous in use, but you can if you wish design your own menus to mitigate this disadvantage to some extent. Rescue is in the main a single-file system – you cannot reference one file through data values in another. Provided this limitation is not a problem, you would find Rescue worth investigating, particularly if the variety of data types and the extensive data validation would be beneficial in your application.

Fig.1. Constraints  
Max no. files in one menu structure 20
Max file size CP/M limit or disk size, whichever is smaller
Max no. records 32760
Max size record 1024 characters (but good data compression methods)
Max no. fields 100
Max field size 60 characters, 14 digits
Max no. keyfields 10
Field types See text – several varieties of character, numeric, date (day/month/year), monetary (sterling), dictionary

Figure_002

Fig.2. ‘Roadmap’ of menus

Figure_003

Fig.3. Menu options

Fig.4. Benchmark times
BM1 Time to add 1 new field to each of 1000 records Setup time
BM2 Time to add 50 records interactively Scrolling time
BM3 Time to add 50 records “in a batch” NA
BM4 Time to access 50 records from 1000 sequentially on 25-character field 1 min 20 secs
BM5 Time to access 50 records from 1000 by index on 25-character field NA* (1-3 secs)
BM6 Time to index 1000 records on 25-character field 12 mins
BM7 Time to sort 1000 records on 5-character field 4 mins 10 secs
BM8 Time to calculate on 1 field per record and store result in record NA
BM9 Time to total 3 fields over 1000 records NA yet
BM10 Time to import a file of 1000 records NA yet
Note: NA=Not available. NA*=Not available as tested – key must match exactly.

 First published in Personal Computer magazine, April 1983

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s