Simplicty and flexibility!

Memo - from DFD to DG3

There have always been two things that have been especially troublesome in DataEase. Sequence numbers and Memo/Long text fields. The need for a text field longer than 255 presented itself very early on but because of the 8 bit data structure of DataEase, it had to be a "hack".

One of the things that makes one love DataEase is its ability to "take a beating". When you look at the crazy crossing of data access people attempt in DataEase - successfully we might add - you sometimes wonder how it could ever work. The reason is the record structure in DataEase. All the space for each field is pre-allocated so every record in a table is always the same size. This is very efficient when it comes to navigating the structure, but at the same time it is very "wasteful" as a lot of disk space might be allocated that is never used. This trade off was acceptable in an 8 bit world since the biggest field one could address anyway was 255. In the early days the solution was simple: let's just make a "hack" that allow us to merge several text fields in the interface when at the same time they are stored as several individual text files in the data file. This was a "brilliant" solution for slightly larger text fields and it was more than adequate in DFD where the screen was only 80x25 characters and a field would need to be the same size as the content. Back then a computer system was more about transactions and less about storing vast amounts of unstructured information. When we moved on to Windows this all changed. The need to store larger amounts of unstructured data increased, so the long text field from DFD was improved upon visually, but it was always the same. Several text fields were merged in the GUI. The benefit of this approach was that all the fields could be manipulated in the program - that is also limited by the 8 bit referencing - but each "LargeText" field would occupy up to 16 text fields, and suddenly we were hitting another limit - the 255 fields in one record. When 7.x was on the drawing board, the Memo field was again on the agenda. This time one wanted to make it a proper LargeText object. Instead of storing the field itself in the data record, a reference to the Memo was stored, and the memo itself was saved in an external file. This is obviously how one should store large unstructured amounts of data but the introduction of Memo to replace Long was a disaster. There are several reasons for this but the main one was the same as for most other "disasters" in DataEase such as migration and replacement of functionality - arrogance. If you are reading this we assume you are a programmer of sorts because you use DataEase. One of the things you have learnt over the years when moving forward with your application is defensiveness. You take backups, you do one thing at a time and test it etc. You never assume that everything will just work like magic. The cornerstone of DG3 for instance is the assumption that we don't know what you are going to do with the functionality. We cannot predict what you will try to do, and we don't want to tell you or limit you, so we need to take a defensive line when moving the product forward. We will never replace functionality but create new when at the same time leaving the old one "in situ" for compatibility etc. One of the reasons for this is of course that we too are learning. In DFW the approach was to replace one functionality with a new one, remove the old functionality and then attempt to migrate. It has never worked simply because apples are not pears. A memo field is brilliant for storing large amounts of text but you cannot search or index it and since it is stored externally it will cause poor performance if used as a "slightly" larger text field. The Memo field is most efficient if you store a lot of data in it when you use it, and if used sporadically in a table. A long: field is brilliant at being a "slightly" larger text field but does not work very well if you want to store larger amounts of text. A long: field will always use its allocated space so is most effective when sized correctly according to the demand, and is always used. Another benefit is that you can reference it fully in programming and each individual part of it can be indexed. The mistake with Memo fields is now obvious. It should never have replaced the long: field but been an added feature that came with a warning. By automatically migrating long: to Memo we caused a disaster while at the same time giving this great new addition to DataEase a bad name. Memo is necessary, and one of the most useful fields in DataEase, but it is simply not very good at being a long: field. With Memo we could have shone but instead we ruined its usefulness because we wanted to cover several bases. Since the Memo was the new long: field, editing it needed to be done as traditional DataEase. It needed to fit into our neat "tab order" and it could only store text. But what about formatted text? There is no reason why you can't store whatever you like in a Memo field, and in 7.5 and DG3 we have taken this step. We have re-introduced long: but now we call the SpanFields. We have introduced a manipulation class for Memo, so you finally can use it in programming, and we have made a RT control to span the Memo fields. Memo has finally come into its own.

Published: 25/10/11 - 10:03:02 (Ulrik Jacob Høegh - Krohn)

Last changed: 21/12/11 - 12:14:02 (Katia Robinson)

Related Articles

Memo - from DFD to DG3