Hi DataEase

we moved from a physical server window 2012 to a virtual server with these characteristics:

The VM sits in a clustered VMware environment DRS enabled.

Vcenter version 7

ESXI version: VMware ESXi, 7.0.3n

Hosts: Dell PowerEdge R760

Dataease server OS: Windows server 2019

CPU Model Intel(R) Xeon(R) Gold 6430

Processor speed 2.1 GHz (Old Server 3.2 GHz)

Processor sockets 2

Processor cores per socket 32

Logical processors 128

Hyperthreading Active

Expose hardware assisted virtualization to the guest OS

Dataease server reserved RAM : 64 GB Reserve all guest memory (All locked)

Disk type : Thick Provision Eager Zeroed

SCSI Controller for V & W: VMware Paravirtual

SCSI Controller for C & D: LSI Logic SAS

The above are provisioned via attached LUN storage from Netapp AF800

Please take in mind the Drive W:\ , V:\ are the drive where the Dabase are saved.

The DataBases size is 100 GB in AVG

The problem is this:

More the procedure is heavy in term of records and fields to update and more the performance decreased than the old server.

The running time is dramatically increased on the new server

Could you please tell me if the settings are correct or there is something to be improved?

DataEase version 8.5.1.2674

The key is in the VIRTUAL bit. 

There is nothing you can do on the DataEase side of this, it is all on the VMWARE side of things. Obviously virtualizing a server means that you add a layer of processing on top of the hardware and hence loose performance.

We have a program going in DataEase called "ImplieDB" which is replacing PRISM as the database layer in all DataEase products. One of the goal with this is to increase performance dramatically and also to completely remove the need for running a DataEase database on a "server" as we will handle all locking etc, directly rather than use the server provided locking modules that has been sadly lacking in performance since Novell. 

IT departments invariable make decissions on behalf of the users they are supposed to server to make things easier for themselves. Virtual Servers is one of these things and whatever they say it delivers reduced performance compared to running things directly on the hardware. The only thing that will compensate for this reduction in performance is better hardware on all levels.

Modern Virtual Servers and Virtual Storage (LUN) etc... is all based on "slow" storage and document providing. It is not designed to run file intensive system like a DataEase database.

Your setup is basically the worst you can imagine for running a DataEase 8.5 system.

In short - an IT departments wet idealist dream is your worst nightmare. Add Virus software scanning all file access on your Database and you can say goodbye to performance forever.

So at least make sure they have excluded your App directory from virus scanning, that on its own will "kill" your performance. 

DataEase write to files all the time, data, indexes, temp files, sort cache etc....

IT departments ..those were mentioned in the earlier discussion by Sam Bird about of LE9 productivity 

as the ... crowd of sysops & sysadmins from ancient IBM360 times ... where is windows nightmares were seeded from.

as to Novell ... of course Oracle 7 server  on it too ... the huge enterprise used DE4.53 SQL version for CRM and accounting successfully until new non DataEase solution was implemented. we have to avoid the usage of "1c" software due to the war laws here now. 

May be DataEase again ? 

Hi and thank you for the feedback . 

Are there any recommendations running Dataease on a Virtual server? reserving cpu and memory , exposing the VM to the physical host hardware as much as possible? 

I can understand the key is in the virtual part but there must be others having Dataease running in a virtual environment.

When you run DataEase on a shared file server the "challenge" is file access and throughput i.e I/O.
 
So the main problem is disk access, not memory etc.

So quick disks, controllers etc. is what is the crux of the matter.

it is common to see disk access of 4-10 times longer in virtualized environment compared to direct access. So if your new HD is twice as fast as the old one it is still at least twice as slow after virtualization.

The fact that 8.5 runs slower in a virtualized environment has nothing to do with DataEase and everything to do with the virtualized environment so the request should target VMWare support on how to increase disk access speed.

The only true way to figure out where your bottleneck is to do disk access test with different settings etc.

Virtualization in our opinion is brilliant for IT departments but not so much for users and at the end of the line any savings achieved by splitting hardware this way is lost in lack of performance and problems for users many times over.

So in many ways the problems here is suboptimum resource management where one try to fit a square peg in a round hole.

A quick Google on benefits of virtualization gave us this:

Virtualization has many benefits for organizations, including:

• Reduced costs

  Virtualization can lower operational costs, reduce the need for additional hardware, and save money on electricity and cooling.
  Improved efficiency

• Virtualization can improve the efficiency of physical servers and power usage. It can also increase application performance and server availability.
  Faster disaster recovery

• Virtualized environments can help businesses regain access to IT infrastructure more quickly after a disaster or cyberattack.
  Increased productivity

• IT teams can spend less time maintaining physical hardware and more time on other tasks.
  Improved security

• Virtualization isolates virtual machines (VMs) from each other, which can help contain security breaches.
  Testing environment

• Virtualization can provide a safe environment to test new software, patches, and server upgrades.
  Improved data mobility

• Virtualized data centers can have more efficient data workloads, which can reduce traffic obstacles and bandwidth bottlenecks. 
  
  What is missing here is any benefits to people using systems on a daily basis to achieve the actual goal of the business which I'm sure is not IT infrastructure related...
  
  100% benefit to IT Dept. at the expense of the users as normal

• Consult VMWare resources on how to increase speed on I/O.
• Check if there is Virus software monitoring file access on your Application Drive. If there is EXCLUDE it. DataEase files does not contain viruses and is not loaded so viruses can be loaded but they are accessed all the time so virus software is the main culprit on performance problems.
  DE8.5 and before create sort files etc in the app catalogue which can slow down processing, so in LE9/DE9 this is moved to temp area on the device where DataEase is run from, this increases sorting a lot.
  LE9/DE9 is currently undergoing a big improvement program in performance where we completely remove need for locking etc, and make disk access much faster. We also completely rewrite indexing etc. This will increase speeds on average between 5 and 25 times, which will compensate for slower disk speed in virtualized environments.

Hi All! 

For sure, no VMware,  no windows, only DataEase OS, shortly DOS. 

No disks, no files and etc.

enough is enough! 

Only RAM ... a little ROM as a vintage device for museum purposes.

where are disks in your brain ? 

DataEase Team! 

Go ahead into new technologies of AI!

Good lick!

Ihor! 

We have talked about this before.

You are entitled to your opinions etc, but we don't need it as a comment on every genuine request for help or information. 

Please don't comment on posts about the good old days and AI. Neither is helpful and just create clutter.

agree, yup, this is not my personal opinion BUT! this is not old good days too. This is present day trend. SSD's productivities tends to RAM's ones.

Modern OSes are handling hardware equipped with wise chips and present day operating systems are wise too.

we have no choice ...reality is as is. LE9 is the fastest DE and inside it we feel like under an OS. 

this is shocking reply to IT departments and VMware producers:

   Quora  Hatchworks Glory to DataEase Team nevertheless! ps. we are using L9E& VM without problems.

The problems is always related to change. If you start of developing a system you will design it so it runs OK with the limitations you have at the time. Then when it goes slow, you will clean the system/delete unnecessary data, buy better hardware etc. and it works fine again.

The problem is when you "upgrade" and it then goes slower.

We had a client that moved from 4.53 on Novell 3.12 for over 100 users to LegEasy4DOS on Azure. Most things was much faster but a couple of critical routines that updated between countries went from taking 3 hours till 6 hours running overnight. 

They couldn't understand it as everything else was so much faster.

The problem was of course that the disk/network IO was emulated and hence was slower than Novell, while everything that was done in memory etc, was much faster.

When they came to us with it, we told them that this was to be expected but they shouldn't worry about intensive disk/network was 50% as fast, rather that the process already took 3 hours..

So we looked at the code rather than buy more hardware/memory etc. The table in question had 7 million records but the procedure was obviously designed i 1987 or something when they had 7000 records, so we simply rewrote it to use indexes etch efficiently and it went from 3 hours to 6 hours to 35 seconds.

So very often the problem is that one doesn't see the forest for all the trees.

das ist fantastisch! what about of reindexing time in the case then ? for sure, DataEase isn't a simple mechanism and the golden rule of mechanics is not about of 7000000 records with amount of fields more than 1... thx  a lot for the this amazing case!

VMware is excellent with LE9 on a new Apple processors ...non Intel for sure

Indexes is mostly "abused" in DataEase. People think the more the better, but they will only work if used right.

As a general rule indexes will make searching faster and saving slower. 

But...

DataEase will only use an index on searching sorting if the first field in the search is indexed and only on the first field. There is no combined indexes so column two onwards in any search will be sequential. That is fine if the first column will narrow it down substantially. However if one have 3 million records and put the index on a yes/no field where half is yes and half is no, you will get a return cursor of 1.5 million that will be search manually for ex. date=current date.

So obviously if one search for current date first and then yes/no you get a much better cursor.

If you sort in reverse on an index field it will do a sequential search again as the index is chronological and it can't be used in reverse.

When analyzing "troubled" DataEase applications where the developer/user say "it worked before" (yes it di when they had 50k records and not 5 million) the problem is very often that they over-index and don't use the indexes at all. I've seen tables with 250 fields and 70 indexes that "used to work". Maybe they shouldn't have.

When you look at the files some of the index files are bigger than the data file...

So obviously when 70+ indexes is to be updated and maintained every time  you save a record it will be slow...and meaningless as they indexes is not used anyway.

We frequently get "complaints" about the limit of 255 fields in a table, but to be honest. If one think having 255 fields in a table is optimum one should revise ones viewpoint and look up data-modelling on the internet. There might be something to learn.

So in general.

1. Only index fields that you know you will use the index on.
2. Keep the number of columns in a table at a minimum and the index fields the same.
3. Don't put virtual fields in tables and avoid to reference virtual fields from other tables. Calculate them when you need them where you need them. In Forms you can simply create virtual fields as part of the form rather than as part of the table. A good rule is never to use the form that defines table other than for defining the saved columns, then all virtual fields in forms will be a property of the form. 
4. In DQL you should never reference a virtual field but simply recreate it in the DQL as a temp variable.

thx. that was tested with dfd even in 1990s ... 

 blackouts due to the war impose low power consumption & high productivity of software here in Ukraine now...

ps. 1990s practice with field indexes was described in volume 2 of de 4.* manual devoted to dba & applications development. limitations were up to 8 indexed fields on a single form. only necessary fields were to be inedexed. new recommendations here are close to truth. the truth is in that no indexes at all. indexing makes database application slow and memory & disk storage consumption are huge. for sure in some cases indexing helps nevertheless.