Is there anything specific that I'm missing to ensure LE9 DQL runs at least as fast as DE4.53 or even DE8.5.

DE 8.5 does have it's performance issues but nothing compared to LE9.

Yeah I know the standard answer is migration has it's issues and I've had personal experience of the frustrations over the past few months.

Fact is that I never thought that an application built in DFW 5 would still be running 28 years later. It is now a massive system with 100s of users.

In the 2000's moved on to pure SQL based systems and occasionally did some upgrades on the old DE system. The client tried their luck and millions with 3rd parties to move the app to other more mainstream technologies, but DE is still standing.

Problem now is that DE6.53 does not run on the latest servers and they need it upgraded as their IT department has to replace the old servers.

While functionally the migration was a success, unfortunately DQL performance sucks.

The core of the system is expert systems that analyse tens of thousands of records of raw data and generate results.

One of the processes takes less than 3min to run on DE6.53 and about 6min on DE8.5.

On LE9 it takes at least 45min with one user and less records and is on a much newer and higher end server.

I thought that deleting and re-creating the DQL's to ensure there is no old migration baggage would help.

Did that - No joy.

The only other option is to re-create the forms/tables from scratch. That would be a mammoth task which I definitely do not want to do if it has no benefit in the end. Complex forms with multiple embedded subforms are also slow at retrieving data. The same forms have no such issue on 6.53.

Is there any possibility that re-creating the forms/tables from scratch is the solution or is LE9 simply just slow?

Dear Sam, it is a very pity but DQL for DFD 4.12U wasn't as good as dBase for hundreds of thousands records of clients data of local state savings bank department modification due to the  hyperinflation in Ukraine in 1992. and the procedure in dBase was very simple and clear ... while DQL was huge and complex ;) You simply have to verify if the value of an field is the substring of an long text string of conditions... You know in DQL it looks like a monstrous  construction with a lot of "if's ... so is the story. but try ..anything can happen even in this windows dark epoch ;)

Good luck!

Hi Ihor
4.53 That was a typo - old system is DE6.53.
I did have experience many moons ago with DEDOS and Dbase but that was a different era :)

Hi, Sam

when I first faced  that DFW monster I decided to change my business from software to hardware ... assembling comps from parts but DFD approach helped me in that for sure :) really my clients got their comps very fast like dfd apps ;) as to modern times don't forget about IBM mainframes experience in gates childhood )))))))))) that is why Your task needs a crowd of sysops & sysadmins and etc...

If you have specifc examples where you can show in ex. 8.5 and LE9 that the performance is much less in LE9 - not a full app but a simple sample, we can run it through debug and find the problem.

Performance over all is much better in LE9 than in previous products but there is a lot of factors that come into play. If a process is iterative any small problem can escalate and give very bad results.

Something like you illustrate here seems almost guaranteed to be related to memory swap to disk, that is the big killer of speed. With newer versions libraries grow and requirements for each task increase even if its the same job so you can get this problem with a new version even if an older version didn't have it.

One of the last projects on LE9 after full release is a collaborative project with DE9 to radically increase speed, up buffers etc. to make PRISM (impliedb now) much faster and more robust. We are talking 100 if not 1000 times of performance improvements.

However, I assume you run your DQL's like in the olden days. DQL was never properly implemented in DFW and this has been a problem from day one. The main performance problems with DQL is linked to GUI which is why we created ExecDQL which is headless DQL.

ExecDQL can't be compared to "PeteDQL" in performance. If you don't generate traditional output, but just process data its a no-brainer to run it as ExecDQL instead. 

We are currently working with an old DFW system for a big client ourselves and how complicated DQL's had to be written to work in early DFW is mindboggling. 

We had one control procedure with 3 export and re-imports and 28 procedures re-written into one procedure, and the process time went from 6 minutes to less than 6 seconds.

We will freely admit that LE9 might not be good at being PRE DE8 DataEase in the way that the way things had to be done that is backwards and complicated, while after DE8 the focus was on simplifying and making the standard functionality work without all these crazy workarounds.

If you import the ExcedDQLstore from the template that come with LE9, you will be able to run a DQL like this.

ExecDQL("@Dqlname", data-entry field1...field4,"","@Dqlname")
ExecDQL(*#00004",etc,"'#00004") 

you can transfer 4 data entry values via the function and use them in the dql as data-entry field1 etc. you can also format the output and the function will return the formated text.

list records
"Sam Bird need to learn the modern ways..." .

Will return that string as part of the function without you formatting it.

A format is like dos just with tags.

for Customers ;
List records
CustomerNr ;
CustomerName.

.items.
<h1>[{CustomerNr}] </h1>
<p>This is the customers name: [{CustomerName}]
.end

Tags are case sensitive so if you list CustomerNAMe the tag need to be [{CustomerNAMe}]

to be true LE9 is the best now. as to an example it is CRM app from ex. 8.5 ... I need more time to copy and paste data from/to external browser now. In ex. 8.5 slow times I had no stimuli to use any browser in CRM that is why the app mentioned worked faster than in LE9 now. I am not confident about ExecDQL chance  to load external program inside LE9 app but DFD had that option already. 

Glory to DataEase team nevertheless!

Things have moved along a lot since then.

I think its time we start thinking of DataEase as a product from the 1980s and consider what it is today.

Obviously one will never get the benefit if one don't use the new features.

ExecDQL is not about loading external apps, but obviously there is no more limittations inside ExecDQL than anywhere else in LE9. The fact is that you can do things in ExecDQL you can only dream about in other products and also earlier versions of DataEase. 

ExecDQL is DQL as a function and it will return whatever you process as the return value of that function. You can create and execute a DQL on the fly and you can do the same with the formatting. There is absolutely nothing you can't do with ExecDQL. ExecDQL alone is more powerful than any previous version of DataEase alone but again its down to the imagination of the user to fully exploit it.

Will look at exec DQL going forward.
For now makes no sense to re-invent the wheel if the wheel is already working...... and the wheel is running magnitudes faster now.

The DQL performance Achilles heel in LE9 is multiple field Relationships and queries eg:....with Field1 = AA AND that Field2 = BB

If a Relationship is defined with a single foreign key field or a query on only one field then it is fast.

If 2 fields are related then it slows down exponentially where the foreign table has thousands of records.

This occurs with or without indexed fields. It's almost as if LE9 stops using the indexes as soon as their is more than one field to query.

While is definitely ideal to rather use a single field for queries there was never such an extreme difference in any previous DE version i am aware of.

With SQL there is a similar issue which is resolved by creating indexes where the columns are defined in the same order as the where clause.

In DE/LE we can't create multi field indexes so the solution was to create an additional field in the queried table with concatenation of the multiple field values and set that field as indexed.
Now that same query which was running 45min is done in less than 3min!

Have not tested yet, but i suspect it's the same issue with DE8.5

This may possibly be related to another issue found with DE8.5 with For... queries including modify records to a SQL DB.

If the DE8.5 DQL does not refer directly to the Key in the SQL table then updates very often do not occur. 

It was frustrating as it was not consistent and there were no errors - just did nothing.
The same DQL works 100% every time in DE6.53 regardless of whether the queried SQL table Unique Key column is included or not.
DE6.53 & DE8.5 are linked to the same SQL DB so it is unlikely due to some issue on the SQL DB.

as a software & hardware tester with more than 40 years of experience with different architectures I propose to check that SQL DB with reliable native in the case  SQL tools(as I did with .dbf using dBase in 1992)... and due to the latest info I will search how to run LE9 inside of Google Chrome ;) don't worry be happy

Hi Ihor
Been there, got the T-Shirt. Native SQL DML has no issues and DE6.53 has no issues.
Only found it with DE8.5. Have not tested that specific issue with LE9.
It may also be a difference in the OLEDB on MS2012 vs MS2008 OLEDB ?

who knows ... I use the only method .. M: make one clear change and see what will be then think ...go to M )))) (loop until success)

as to me all recommendations are not to use Chromium OS or Chrome extensions to execute ms dos or windows binary files ...in general ;) 

Any comment from Dataease support regarding LE9 issues with multiple field Relationships and queries?

Hi, Sam

as I've said already about ms crowd of sysops & sysadmins ask the doors of law about their comments on the issue mentioned by You...

Also I've demonstrated the frustrating situation with using web browser in LE9 caused by mustdie too.

But please read this and alike from the origins Microsoft OLE DB Driver for SQL Server ... nevertheless ;)

Our journey started with DataEase 7.1 and the job has been to improve and fix issues with that as a baseline.

There might be a lot of things that works better or mostly just different in DFW 6 compared to DFW 7 onwards but that would be neither here nor there.

Our focus will at any given time be on the customers that has followed the versions forward and in fixing their challenges and problems and helping them move forward. We are well aware that there was a lot of problem for people moving from PRE DFW 7 to DFW 7 and beyond but that is water under the bridge.

To develop successful application in DFW 5 and DFW 6 users had to learn a lot of hacks and workarounds, used CDFs extensively and in many ways make very cumbersome applications. 

In DE8 and DE9 we have worked on making it easier again and that things should work out of the box. 

Yes, we have seen that complicated DFW apps with control procedures and a lot of temporary tables etc etc. might struggle and not work the same way in DE9 as it does in DE6 etc, but as that way of building applications is outdated and obsolete we are rather using our resources on making DataEase more flexible and offer modern ways of doing things than focusing on getting obsolete code to run.

We simply don't have the time or the will to constantly investigate allegations of something used to work like this or this work like that in version this.

Further to this we will not investigate "hearsay". If there is something in LE9/DE9 that doesn't work as it should we will need a simple sample that showcase what isn't working so we can investigate.