Friday Jun 06, 2008

Webinar: Understanding the Oracle Architecture - 2.5 Hours

I have a webinar to immerse attendees in the Oracle architecture, products, terminology and key features.  This fast paced presentation is designed to give someone with a database background, a detailed overview of the Oracle database server.   SQL Server, DB2, MySQL, Postgres, Ingres, SQLLite, etc. DBAs  and developers will get an understanding of how the Oracle database server works as well as key products and features.  At my blog   web.mac.com/george.trujillo, if you select the Oracle database tab, the key slides can be found with descriptions.  You will need to go into the archive to get  some of the podcasts.  As I get time, I will complete the audio podcasts for each slide. 

Topics include:

  • Oracle Architecture
  • Database versions
  • Database products
    • Real Application Clusters (High Availability with multiple servers)
    • Data Guard (Standby databases)
    • Streams (Data Sharing)
  •  Tools
    • Enterprise Manager (Grid Control and Database Control)
    • Data Pump
    • SQL\*Plus
  • Metadata
  • Cost-Based Optimizer
  • Backup and Recovery
    • Recovery Manager
    • Flashback
  • ETL
  • Schemas
  • Database Applications
    • PL/SQL
    • Java
    • XML
    • PHP
    • .NET
  • Fusion Middleware
    • J2EE
    • Web Services
    • Service Oriented Architecture
    • BPEL
    • XML
    • Application Servers
    • ADF
    • Frameworks

 

Wednesday May 28, 2008

MySQL versus Oracle Features/Functionality

I love working with MySQL and Oracle technologies.  Both of them are great solutions.  The point is to find the right environment to leverage what they do well for the best price/performance/scalability you are looking for.

  • MySQL is relatively light-weight, can be extremely fast when applications leverage architecture.  Lots of features stay free as the database servers grow such as replication and partitioning.
  • Oracle offers lots of feature/functionality for solving complex problems.  Supports large OLTP environments as well as VLDBs.

 Here are some of the feature comparisons from a DBA perspective.

Features/Functionality

MySQL

Oracle

Strengths

Price/Performance Great performance when applications leverage architecture.

Aircraft carrier database capable of running large OLTP and VLDBs.

Database Products

Enterprise ($) – supported, more stable.

Community (free) – more leading edge.

Enterprise ($$$$)

Standard ($$)

Standard One  ($)

Express (free) – up to 4GB

Application Perspective

Web applications often don’t leverage database server functionality. Web apps more concerned with fast reads.

More you do in the database the more you will love Oracle with compiled PL/SQL, XML, APEX, Java, etc.

Administration

Can be trivial to get it setup and running.  Large and advanced configurations can get complex.

Requires lots of in-depth knowledge and skill to manage large environments.  Can get extremely complex but also very powerful.

Popularity

Extremely popular with web companies, startups, small/medium businesses, small/medium projects.

Extremely popular in Fortune 100, medium/large enterprise business applications and medium/large data warehouses.

Application Domains

(most popular)

Web (MySQL excels)

Data Warehouse

Gaming

Small/medium OLTP environments

Medium/Large OLTP and enterprise applications.  Oracle excels in large business applications (EBS, Siebel, PeopleSoft, JD Edwards, Retek, ...)

Medium/Large data warehouse

Development Environments

(most common)

1) PHP

2) Java

3) Ruby on Rails

4) .NET

5) Perl

1) Java

2) .NET

3) APEX

4) Ruby on Rails

5) PHP

Note: Oracle focusing on Java for next generation business applications. 

Database Server

(Instance)

Database Instance stores global memory in mysqld background process.

 

User sessions are managed through threads.

Database instance has numerous background processes dependent on configuration.  System Global Area is shared memory for SMON, PMON, DBWR, LGWR, ARCH, RECO, etc.

 

Sessions are managed through server processes.

Database Server

(Physical Storage)

Made up of database schemas.

Each storage engine stores information differently.

Common storage engines:

MYISAM – stores data in .FRM, .MYD and .MYI files.

InnoDB – stores data in a common tablespace or individual tablespaces per table.

 

Binary logs are used for point-in-time recovery

Uses tablespaces for system metadata, user data and indexes.  Common tablespaces include:

SYSTEM

SYSAUX

USER DATA

USER INDEXES

TEMPORARY

UNDO

Redo and archive log files are used for point in time recovery.

Tables

Tables use storage engines.  Each storage engine provides different characteristics and behavior.

A few tables with tons of features.

Partitioning

Free, basic features

$$$ with lots of options

Replication

Free, relatively easy to setup and manage.  Basic features but works great.  Great horizontal scalability.

$$$, lots of features and options.  Much higher complexity with a lot of features.  Allows a lot of data filtering and manipulation.

Transactions

InnoDB and upcoming Falcon and Maria storage engines

Regular and Index only tables support transactions.

Backup/Recovery

No online backup built-in.

Replication

OS Snapshots

InnoDB Hot Backup

Recovery Manager (RMAN) supports hot backups and runs as a separate central repository for multiple Oracle database servers.

Export/Import

Easy, very basic.

More features.

Data Dictionary (catalog)

Information_schema and mysql database schemas offer basic metadata.

Data dictionary offers lots of detailed information for tuning.  Oracle starting to charge for use of new metadata structures.

Management/Monitoring

$, MySQL Enterprise Monitor offers basic functionality.

Additional open source solutions.

May also use admin scripts.

$$$$, Grid Control  offers lots of functionality.

Lots of 3rd party options such as BMC, Quest, Embarcadero and CA.

Storage

Each storage engine uses different storage. Varies from individual files to tablespaces.

Tables managed in tablespaces.  ASM offers striping and mirroring using cheap fast disks.

Stored Procedures

Very basic features, runs interpreted in session threads.  Limited scalability.

Advanced features, runs interpreted or compiled.  Lots of built in packages add significant functionality.  Extremely scalable.

 From my perspective, you almost can't compare Oracle and MySQL because they are two different type of products.

  • Oracle has tons of features from XML, user-defined types, lots of database management tools.  Oracle:
    • Can be used with Oracle Express, a database that has tons of functionality for the beginning project to running the largest business applications in the world. 
    • Has tons of features that usually minimize the need for 3rd party software.
    • Oracle is currently placing a major emphasis in two areas: 
      1. Enterprise business applications running Oracle Business Suite, Siebel, JD Edwards, PeopleSoft, Retek, etc.  Oracle's next generation Oracle Fusion Applications are going to have a major impact on business applications in the next few years.
      2. Oracle Fusion Middleware and the technology surrounding the middle-tier with application servers, J2EE, BPEL, SOA, XML, Web Services.  The ability to get disparate applications to work together are a key to Oracle Fusion applications.  Oracle's purchase of BEA shows the emphasis Oracle is placing on the middle-tier to run business applications.
  • MySQL excels when high speed reads can be used for web, gaming and small/medium data warehouses and OLTP systems.   MySQL:
    • Can also be used with startup companies and smaller projects because it can be simple to use at a low cost. 
    • Does not have a fraction of features of Oracle but for the companies using MySQL they are often building most of the functionality in the middle-tier and do not need lots of functionality in the database. 
    • Does not have a lot of the surrounding software to support a MySQL environment.  3rd party software and open software are often used to run MySQL in an enterprise environment.  Nagios and Memcache are used a lot in MySQL environments.
In this blog I am not trying to say MySQL or Oracle is better than the other.  They both have features where customers can benefit.   I enjoy working with both and would highly recommend both database servers.


Tuesday May 27, 2008

Top Things to Know if new to MySQL on Solaris

Here are ten things to know about MySQL if you are new to MySQL on Solaris:

  1. The default install is simple, easy, takes minimum resources but is not a configuration you should use for production environments.  If new to MySQL use the default install.
  2. To connect to a MySQL database server you need to know the host and the port.   The my.cnf (or my.ini) file is the startup configuration file.
  3. The mysqld background process is the database server that reads the startup configuration file.  The mysqld database server is thread based, so threads start up to manage sessions.
  4. MySQL has unique features just like other relational databases. MySQL database servers need to be configured for performance like any database server if running in a production environment.  You'd better understand these features if supporting MySQL.
  5. Before running MySQL in a production environment, you'd better understand storage engines, SQL Modes, MySQL security and the architecture of the MySQL database server.
  6. As of May 30, 2008  MySQL 5.0 is the GA release,  5.1 RC is about to go GA.  MySQL 6.0 is in alpha.  The goals of MySQL 6.x is adding a lot of online features, new important functionality with increased performance and throughout.  MySQL 6.0 will add significant scalability and functionality to the database server.
  7. Through version MySQL 5.1, there is no hot backup utility.  OS snapshots, replication, InnoDB Hot Backups are popular ways for backing up MySQL databases.  MySQL 6.x will have hot backup capability.
  8. MyISAM tables totally rock when high speed reads are very important.  Make sure you understand all characteristics or these tables before using them.
  9. InnoDB is the transactional storage engine to use through MySQL 5.1.
  10. New storage engines such as Maria, Falcon and enhancements to InnoDB are going to change the use of MySQL in the enterprise.
  11. Replication is one of the most popular features in MySQL.   It is free, relatively simple to set up and performs basic functionality of replication.  Does not contain advanced replication features but it also does not contain the complexity either.  That is what users like about MySQL replication.
  12. MySQL Cluster is a HA (high availability) environment.  MySQL Cluster works extremely well when applications leverage how MySQL Cluster works.  Make sure you understand the characteristics of MySQL Cluster before considering it for an application.
  13. Go to  http:/dev.mysql.com   and then click on Documentation.  Scroll down a little bit and find the Example Databases section.   The World, Sakila and Menagerie gzip and zip files contain SQL code for creating database schemas, building tables and storing them with data.  They provide sample data to play with.
  14. MySQL Enteprise Monitor is MySQL's management and monitoring software for managing MySQL database servers.  It contains lots of GUI interfaces and advisors.
  15. Popular software seen in MySQL environments include memcache, SQLYog, Navicat, Pentaho and Nagios.  

Why MOCA on Solaris?

Installing and configuring MySQL using the packages, RPMs and installers that can be downloaded from MySQL are easy to install. There are also numerous installers like YUM for Fedora that can install an entire LAMP or SAMP stack for you with a simple click. If all these installs are so easy why would you ever want to perform a manual install or use MOCA?

Well, database administration like life comes down to doing things easy and simple or doing things right. If you are going to perform a simple install or just want to learn MySQL, then definitely use one of the very easy install methods mentioned above. If you are concerned about any of the following perform a manual install using some set of guidelines like MOCA:
  • Installing multiple MySQL database servers on the same platform.
  • Installing multiple versions of MySQL software on the same platform.
  • Consistency of MySQL configurations across different operating systems and hardware.
  • Installations and configurations conforming to standards and guidelines.
  • Minimizing pain and anguish in your life as a DBA.
  • Avoiding evenings and weekend work.
  • Minimizing mistakes, downtime and adminstration costs.

  • If installing MySQL for development and in a small shop then simple installs are nice. That is part of the tremendous attraction of MySQL. For production DBAs installing MySQL across multiple platforms at the enterprise level, it is important to use guidelines and standards that your DBA team can control. When a lot younger I didn't mind working late hours and working on an hourly basis. Now, I prefer to minimize administration costs, late hours and minimizing pain. :)

    Installing MySQL 5.1 on Solaris 10 using MOCA

    Introduction

    The following instructions will lay out an installation of MySQL on Solaris using the MySQL Optimal Configuration Architecture (MOCA) for someone knowledgeable in MySQL/Solaris administration. MOCA is a set of best practices I put together to lay out a set of guidelines for installing and configuring a MySQL database server.  MOCA is designed for someone with experience with MySQL, it is not for someone brand new to MySQL.

    If you are new to MySQL or to Solaris, I recommend using the default package install for MySQL. The MySQL default install is recommended for someone new to MySQL or the operating system platform. If the default package install makes more sense for you, then you can stop reading.  This install uses MySQL 5.1.24 and it will work for any 5.1.x install (i.e. 5.1.42).

     Why Perform a Manual Install

    The default install with MySQL is great for users new to MySQL. It is simple, requires a few point and clicks and you are up and running. The problem with a default install is that it is designed to be a very simple install and take minimum resources. The default install also puts MySQL files in different locations on the filesystem dependent on the OS release and platform. The default install is not how an experienced DBA would want to set up a production database environment. It is much better to be able to control the layout and configuration of the database software for production database environments and for platforms where multiple MySQL servers may be installed in the future.

      This install assumes you have a fundamental understanding of Solaris and have an understanding of MySQL database administration fundamentals. Oracle DBAs will find this installation very similar to the concepts of the Optimal Flexible Architecture (OFA).
      For experienced MySQL DBAs a manual install is much better. For this purpose I created a best practices configuration and white paper called MOCA (MySQL Optimal Configuration Architecture). This is based on DBA best practices and should be very similar to Oracle, DB2 and SQL Server production DBAs. There are certain fundamental truths about how database servers should be installed, configured and managed. My MOCA whitepaper addresses these fundamental truths. This manual install will follow MOCA standards and conventions.

    Why MOCA? 

    Visit mysql-dba-journey.blogspot.com to get the details of the reasons behind MOCA and why it is based on best practices. There is also an example of installing MySQL on Mac OS and Linux.    In summary, MOCA focuses on:
    1. Separating database software from other software.
    2. Separating data and index files, log files for recovery, administration and backup files.
    3. Developing standard naming conventions.
    4. Defines a flexible configuration that can support multiple database servers on same platform.
    5. A consistent configuration for multiple servers and versions of MySQL database software.

    Installation Summary

    This installation looks more complex than it is.  I use this configuration for all  MySQL DBA classes.

    1. Remove old versions of MySQL if they exist.  Setup up operating system user called "mysql" and the environment for this user.
    2. Set up directories and directory permissions for all MySQL data files.
    3. Setup MySQL software and install MySQL software as mysql operating system user (not as root).  Configure the my.cnf configuration file.
    4. Create the mysql database (mysql_install_db) and setup the security environment (mysql_secure_installation). Start the mysql database server.
    5. Test the shutdown and startup of the database server. 

    Installation Environment

    The environment for this installation is Solaris 10 - Downloaded DVD iso image from www.sun.com website. I installed the Solaris 10 05/08 x86/x64 image for this demo (sol-10-u5-ga-x86-dvd.iso). MySQL 5.1 I downloaded from dev.mysql.com.

      Before installing MySQL on my platform, make sure there are no previous versions of MySQL preinstalled. Unless you want the older version of MySQL, your life will be much easier if you remove any previous releases that are not being used.
      Read through this installation a few times before starting.

    Look for existing MySQL software

    This install uses 5.1.24, these installation procedures can be used for any 5.1.x installation. Dependent on the version of Solaris, different packages may need to be installed or removed (old MySQL installations).  

    Check to see if MySQL is installed on your current system.
    # grep mysql /etc/passwd
    # find /usr/local -name '\*mysql\*' - print # look here for MacOS, Unix/Linux
    # find /var -name '\*mysql\*' - print # good place to start with Solaris
    # find / -name "\*mysql\*' - print # look everywhere for MySQL installations

     VM Fusion Choices for Installing Solaris 10

    My choices for installing Solaris 10 in a VM Fusion environment. During the installation you will be asked to hit F2 to continue. On a MAC that will be EscapeKey-2 or FN-F2. Solaris Interactive US-English Networked - DHCP IPv6 - No You may need to specify the amount of disk space to use. I allocated 10228 MB.

      You should now be able to log in as root. With Solaris choose the Java Desktop Environment or the Common Desktop Environment (CDE), this is a personal preference.

     Removing older versions of MySQL on Solaris

    Check for MySQL packages installed and remove them.
    # pkginfo | grep mysql

    The following packages SUNWmysqlr, SUNWmysqlt, SUNWmysqlu were found and removed.
    # pkgrm SUNWmysqlr
    # pkgrm SUNWmysqlt
    # pkgrm SUNWmysqlu

    Remove old MySQL files from common directories.
    # sudo rm /usr/local/mysql
    # sudo rm -rf /Library/StartupItems/MySQLCOM/

    Setup new mysql user if one does not exist.  If a mysql user does exist, set up a password, default shell, default directory, etc.
    No mysql user was found so I added one. Add the mysql group, mysql user, password and home directory.
    # groupadd -g 300 mysql
    # useradd -u 300 -g 300 -d /export/home/mysql -s /usr/bin/bash -c "MySQL DBA" mysql
    # passwd mysql

    Login and verify the mysql user setup

    # su - mysql (or exec login mysql)

    Then define a default profile file using your favorite text editor.

    --- .bash_profile file ------
    PS1='$PWD: '
    MYSQL_BASE=/opt/mysql
    MYSQL_HOME=/opt/mysql/5.1.24
    export MYSQL_BASE MYSQL_HOME
    PATH=$PATH:$MYSQL_HOME/bin
    --- end of .bash_profile file -------

    Set your environment by sourcing your profile file.

    $ cd $MYSQL_HOME
    $ .   ./.bash_profile

    Downloading MySQL 

    Go to http://dev.mysql.com and go to downloads. Find the distributions and choose the install release you want. I chose 5.1.24. I prefer a manual install so I choose the Solaris Tar Packages the Solaris 10 64-bit install. Select a mirror. On the Select a Mirror page, I choose No thanks, just take me to the downloads!

    MySQL Directory Organization 

    A good way to separate MySQL files and software:
    /opt/mysql/5.1.24 - Symbolic link to software directory location
    /db01/mysql/mysql01/data          - data directory
    /db02/mysql/mysql01/binlogs     - location of binary log files
    /db03/mysql/mysql01/admin       - main administration directory
    /db04/mysql/mysql01/backups    - location of backup files

    Create the following base (parent) directory to download the MySQL software into.
    # mkdir -p /opt/mysql
    # export MYSQL_NAME=mysql01

    The directory pattern of "mysql01" will be used to uniquely identify all physical files associated with this specific MySQL database server.
    Setup data directory structure
    # mkdir -p /db01/mysql/$MYSQL_NAME/data

    Setup mysql administration directory structure
    # mkdir -p /db03/mysql/$MYSQL_NAME
    # cd  /db03/mysql/$MYSQL_NAME/
    # mkdir logs errors sql startup run


    Setup binary log structure
    # mkdir -p /db02/mysql/$MYSQL_NAME/binlogs

    Setup backup directory structure for backups and exports.
    # mkdir -p /db04/mysql/$MYSQL_NAME
    # mkdir /db04/mysql/$MYSQL_NAME/backups
    # mkdir /db04/mysql/$MYSQL_NAME/exports

    Set permissions and ownership for MySQL file directories.
    # chmod -R 750                   /db\*/mysql/\*    /opt/mysql/\*
    # chown -R mysql:mysql    /db\*/mysql/\*    /opt/mysql/\*

    Before going further

    Double (triple) check all directory paths and permissions.   99.99% of issues with manual installs  are typos in the directory paths, typos in the file names or permission issues with directories.

    Double check all paths.   When you try to bring up database server, if it defaults to the default areas its because it can;t find a directory or doesn't have permission for directories specified so it will then try the default locations.

    Setup the MySQL software (as the MySQL OS user, not the root OS user)

    All following commands are run as the mysql OS user. In the /opt/mysql directory unzip and untar the MySQL software as the mysql OS user.
    $ cd /opt/mysql
    $ gunzip mysql-5.1.24-rc-solaris10-x86_64.tar.gz
    $ tar xvf mysql-5.1.24-rc-solaris10-x86_64.tar
    $ ln -s mysql-5.1.24-rc-solaris10-x86_64 5.1.24
    $ cp $MYSQL_HOME/support-files/my-small-cnf   $MYSQL_HOME/my.cnf

    Add the following entries to the my.cnf file to the [mysqld] group. This separates all your dynamic administration files, data files, and binary log files to different locations. A separate port is defined away from the default.

    [mysqld]
    log-error=/db03/mysql/mysql01/errors/mysql01.err
    pid-file=/db03/mysql/mysql01/mysql01.pid
    datadir=/db01/mysql/mysql01/data
    basedir=/opt/mysql/5.1.24
    log-bin=/db02/mysql/mysql01/mysql-bin

    Create the mysql database files for the MySQL instance. This will create the default database schemas and database files.

    $ cd $MYSQL_HOME 
    $ scripts/mysql_install_db --datadir=/db01/mysql/mysql01/data  --basedir=$MYSQL_HOME

    Verify data files and directories have been created in the datadir directory. 

    $ cd /db01/mysql/mysql01/data
    $ ls

    ib_logfile0  ib_logfile1  ibdata1      mysql        test 

    Start the MySQL database server pointing to the defined locations.
    $ cd /opt/mysql/5.1.24
    $ bin/mysqld_safe --defaults-file=$MYSQL_HOME/my.cnf &

    If there are socket errors:

    i.e.  MySQL client cannot start with the error "cannot connect to the MySQL server through socket <filename>

    Solution:

    Each MySQL needs to write to a unique socket file.  If you don't specify one, a default one is chosen which may not have the appropriate permissions.   Make sure the permissions are set properly (owned by mysql).  The socket needs to match for the mysqld and the client side.  Make sure the socket definitions match in the my.cnf file in the [mysqld] and [client] groups.


    Verify the mysqld background process is running as well as the mysqld_safe monitoring process. The mysqld background process should be up and running.

    $ ps -ef |grep mysql

    Clean up the database server by adding passwords and getting rid of anonymous users.  If there are problems with the mysql_secure_installation script, then set the password manually and get rid of the anonymous accounts and any accounts with no passwords.

    $ cd $MYSQL_HOME 
    $ bin/mysql_secure_installation 

    Shutdown the MySQL server to verify you can shutdown and startup the MySQL instance.

    $ mysqladmin --defaults-file= $MYSQL_HOME/my.cnf   shutdown -uroot -p
    $ cd $MYSQL_HOME

    $ bin/mysqld_safe --defaults-file= $MYSQL_HOME/my.cnf

    You're up and running have fun. Once you are confortable with this configuration layout, you can create a Unix shell script that will automate almost the entire process. With a shell script automation the install takes about ten minutes.

    Wednesday May 07, 2008

    Sun: Getting Started with MySQL

    A quick getting started guide to learning MySQL.[Read More]

    Wednesday Apr 30, 2008

    A DBA's Journey in the Sun

    I'm really excited about starting this blog. I've spent my entire career on the leading edge of technology. From Unix, Oracle, C, C+, Java, J2EE, SOA, BPEL, Oracle Business Rules, open source, MySQL and Virtual Reality. Since joining Sun I have been completely re-energized by seeing a lot of the vision of Sun and where it is heading. I have a lot of interest in multi-tiered technologies surrounding databases, application servers, SOA, BPEL, education, training and sharing knowledge.
    In this blog I want to share my thoughts and journeys living on the leading edge of technology and enjoying everything that makes life worth living.
      A summary of my background:
      • Oracle DBA - (since Version 5 beta): RAC, Data Guard, Streams, Tuning and B/R
      • Oracle Fusion Middleware (OAS, J2EE, BEA, BPEL, Business Rules, XML)
      • MySQL DBA
      • Oracle ACE
      • Independent Oracle Users Group - Board of Directors
      • International Oracle Users Council
      • Oracle Beta Leadership Council
      • IOUG - Masters Universities
      • Selected to Oracle of Oracles
      • Sun SL Mentor
        Additional Blogs:
        • http://mysql-dba-journey.blogspot.com
        • http://trubix.blogspot.com
        • http://web.mac.com/george.trujillo
          I hope you enjoy the blog.
          George Trujillo, Jr.
          About

          This blog is my perspective of Sun in relation to Oracle, MySQL, Oracle Fusion Middleware, Education and Virtual Reality platforms. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

          Search

          Categories
          Archives
          « April 2015
          SunMonTueWedThuFriSat
             
          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
            
                 
          Today