DBAs are in for a rude awakening.
A database runs most efficiently when all of the data is held in RAM.
causes some data to be sent to a disk drive for later retrieval.
This process, called 'paging' can have a huge performance impact.
This can be shown numerically by comparing the time to retrieve data from
disk (about 10,000,000 nanoseconds) to the access time for RAM (about 20 ns).
Databases are the backbone of most Internet services. If a database
does not perform well, no amount of improvement of the web servers or
application servers will achieve good performance of the overall service.
That explains the large amount of effort that is invested in tuning
database software and database design.
These tasks are complicated by the
difficulty of scaling a single database to many systems in the way that web
servers and app servers can be replicated. Because of those challenges,
most databases are implemented on one computer. But that single system must
have enough RAM for the database to perform well.
Over the years, DBAs have come to expect systems to have lots of memory,
either enough to hold the entire database or at least enough for all commonly
accessed data. When implementing a database, the DBA is asked "how much
memory does it need?" The answer is often padded to allow room for growth.
That number is then increased to allow room for the operating system,
monitoring tools, and other infrastructure software.
And everyone was happy.
But then server virtualization was (re-)invented to enable workload
Server virtualization is largely about workload isolation - preventing
the actions and requirements of one workload from affecting the others. This
includes constraining the amount of resources consumed by each workload.
Without such constraints, one workload could consume all of the resources
of the system, preventing other workloads from functioning effectively.
Most virtualization technologies include features to do this - to schedule
time using the CPU(s), to limit use of network bandwidth... and to cap
the amount of RAM a workload can use.
That's where DBAs get nervous.
I have participated in several virtualization architecture conversations which
Me: "...and you'll want to cap the amount of RAM that each workload
DBA: "No, we can't limit database RAM."
Taken out of context, that statement sounds like "the database needs
infinite RAM." (That's where the CFO gets nervous...)
I understand what the DBA is trying to say:
DBA: "If the database doesn't have sufficient RAM, its performance will
be horrible, and so will the performance of the web and app servers that
depend on it."
I completely agree with that statement.
The misunderstanding is that the database is not expected to use less
memory than before. The "rude awakening" is modifying one's mind set
to accept the notion that a RAM cap on a virtualized workload is
the same as having a finite amount of RAM - just like a real server.
This also means that system architects must understand and respect the DBA's
point of view, and that a virtual server must have available to it the same
amount of RAM that it would need in a dedicated system. If a non-consolidated
database needed 8GB of RAM to run well in a dedicated system, it will still
need 8GB of RAM to run well in a consolidated environment.
If each workload has enough resources available to it, the system and all
of its workloads will perform well.
And they all computed happily ever after.
P.S. Memory needs of consolidated systems require that a system running
multiple workloads will need more memory than each of the
unconsolidated systems had - but less than the aggregate amount they had.
Considering that need, and the fact that most single-workload systems were
running at 10-15% CPU utilization, I advise people configuring virtual server
platforms to focus more effort on ensuring that the computer has enough memory
for all of its workloads, and less effort
on achieving sufficient CPU performance. If the system is 'short' on CPU power
by 10%, performance will be 10% less than expected. That rarely matters.
But if the system is 'short' on memory by 10%, excessive paging can cause
transaction times to increase by 10 times, 100 times, or more.