Tuesday Apr 14, 2009

SugarCRM Performance on Sun X4270 Server

Open Source Benchmark Kit for SugarCRM has been recently released on-line. This benchmark kit is a standardized benchmark kit that can be used to measure the performance of SugarCRM 4.5.1 and 5.2.0 releases. The source and instructions to use the benchmark kit can be downloaded from here: http://kenai.com/projects/erpbench

Using the Open Source Benchmark Kit for SugarCRM the performance of SugarCRM 4.5.1i has been measured on Sun X4270 Server. The following results were obtained on this server.

Final Result: 66.937 @CRM Ops/Sec, Total Users: 330


Server Configuration

Sun X4270

SSD Drives (2 x 32GB)

2 x Intel X5570 Processors

24GB RAM

Software Configuration

SugarCRM 4.5.1i

Apache 2.2.11

MySQL 5.0.67

PHP 5.2.9

Open Solaris Build 110



Throughput and Response Time Graphs for CRM Operations

The CRM Operations are divided into three drivers that call these operations during the benchmark. I have provided the throughput graphs for all the three including the Response Times.








Open Source Benchmark Kit for SugarCRM

This kit is based on the Faban Harness that is known to scale well for thousands of concurrent user simulations across distributed drivers. More information regarding Faban can be obtained from the project website here: http://faban.sunsource.net/

This benchmark is based on the typical functions that can be performed on a CRM application like SugarCRM. The following Business Objects are used during benchmark runs.

Module

List View, Detail View

Create, Delete

Login

-

-

Contacts

Yes

Yes

Meetings

Yes

Yes

Tasks

Yes

Yes

Accounts

Yes

Yes

Opportunities

Yes

Yes

Leads

Yes

Yes

Emails

Yes

Yes

Notes

Yes

Yes

Cases

Yes

Yes

Logout

-

-



Performance Tunings

The following tunings were applied to SugarCRM 4.5.1i for this benchmark:

#file name config_override.php

#Some of the following are available only in 5.x

$sugar_config['calculate_response_time'] = false;

$sugar_config['verify_client_ip'] = false;

$sugar_config['use_real_names'] = false;

$sugar_config['stack_trace_errors'] = false;

$sugar_config['developerMode'] = false;

$sugar_config['disable_count_query'] = true;

$sugar_config['hide_subpanels_on_login'] = true;

$sugar_config['disable_vcr'] = true


#file name config.php following lines are modified

'special_query_limit' => 9000000,

'default_limit' => 9000000,

The following tunings were applied to the MySQL database configuration:

#file name /etc/mysql/5.0/my.cnf

[mysqld]

port = 3306

socket = /tmp/mysql.sock

datadir = "/db_ssd/data"

back_log = 50

max_connections = 5000

max_connect_errors = 10

table_cache = 2048

max_allowed_packet = 2M

binlog_cache_size = 8M

max_heap_table_size = 128M

sort_buffer_size = 64M

join_buffer_size = 128M

thread_cache_size = 3000

thread_concurrency = 3000

record_buffer = 8M

query_cache_size = 256M

query_cache_type = 1

query_cache_limit = 4M

query_prealloc_size=65536

ft_min_word_len = 4

default-storage-engine = innodb

thread_stack = 192K

transaction_isolation = REPEATABLE-READ

tmp_table_size = 1024M

log_long_format


server-id = 1

key_buffer_size = 64M

read_buffer_size = 8M

read_rnd_buffer_size = 64M

bulk_insert_buffer_size = 128M


innodb_additional_mem_pool_size = 1024M

innodb_buffer_pool_size = 1G

innodb_data_file_path = ibdata1:2048M:autoextend

innodb_data_home_dir = /db_ssd/data

innodb_file_io_threads = 8

innodb_thread_concurrency = 0

innodb_flush_log_at_trx_commit = 1

innodb_log_buffer_size = 16M

innodb_log_file_size = 1G

innodb_log_files_in_group = 2

innodb_log_group_home_dir = /ZFS_DISK/db/logs

innodb_max_dirty_pages_pct = 90

innodb_lock_wait_timeout = 120


Tuesday Aug 19, 2008

SugarCRM FastStack Installers for OpenSolaris

In case you have not noticed SugarCRM has been available for Solaris both SPARC and Intel platforms as FastStack Installers for quite sometime now. These installers are so easy to install. In just few minutes you can start using SugarCRM without worrying about the details of web server and database.

These installers are primarily available for Solaris. If you want to use the same binaries to install on OpenSolaris you would have to make a few minor changes so that the installation goes smoothly.

Before we get started here's the link to the SugarCRM installers for Solaris: http://www.sugarcrm.com/crm/download/sugar-suite.html

Installing SugarCRM on OpenSolaris

I have installed on OpenSolaris 2008.05 that you can download fromhttp://www.opensolaris.com/get/index.html


-bash-3.2# uname -a
SunOS vayuport-os 5.11 snv_86 i86pc i386 i86pc

-bash-3.2# cat /etc/release
          OpenSolaris 2008.05 snv_86_rc3 X86
Copyright 2008 Sun Microsystems, Inc. All Rights Reserved.
           Use is subject to license terms.
                Assembled 26 April 2008

Check if you have have mysql and webservd users defined on your system. Most likely they should already there.

-bash-3.2# cat /etc/passwd | egrep "webservd|mysql"
mysql:x:70:70:MySQL Reserved UID:/:
webservd:x:80:80:WebServer Reserved UID:/:


-bash-3.2# cat /etc/group | egrep "webservd|mysql"
mysql::70:
webservd::80:


In case you do not see the above output run the following commands to create the users and the groups.


-bash-3.2# groupadd -g 70 mysql
-bash-3.2# groupadd -g 80 webservd

-bash-3.2# useradd -c 'MySQL Reserve UID' -g 70 -u 70 mysql
-bash-3.2# useradd -c 'WebServer Reserve UID' -g 80 -u 80 webservd


If you do see the users please go ahead and add the following two privileges as shown below:

  • MySQL Administration

  • Apache 22 Administration


-bash-3.2# usermod -u 70 -P "MySQL Administration" mysql
-bash-3.2# usermod -u 80 -P "Apache 22 Administration" webservd


With that you are now ready to install SugarCRM using the FastStack installer:


-bash-3.2# /usr/sfw/bin/wget
http://www.sugarforge.org/frs/download.php/4394/sugarcrm-CE-5.0.0g-solaris-intel-installer.bin

-bash-3.2# chmod a+x sugarcrm-CE-5.0.0g-solaris-intel-installer.bin
-bash-3.2# ./sugarcrm-CE-5.0.0g-solaris-intel-installer.bin


If you select the typical install option during the installation. You should see the following two screens toward the end.

Thursday Aug 07, 2008

Price to Performance with SugarCRM and Sun Web Server

Price to Performance with SugarCRM and Sun Web Server


Introduction

As promised in my earlier blog. I've completed the tests and finalized the tuning that gives the best price to performance on the Sun Solaris Operating System. SugarCRM is a Customer Relationship Management software written in PHP and needs the web stack (Web Server, PHP 5, and MySQL 5). We will be exploring options available for the optimal hardware and software combination with the best price to performance for Solaris Operating Environment.

Installation and Configuration

Please refer to my earlier blog http://blogs.sun.com/vanga/entry/scaling_sugarcrm_with_mysql_on on how to install and configure SugarCRM. For a Sun Web Server 7.0 installation along with configuration for PHP, please follow the instructions here http://developers.sun.com/webtier/reference/techart/php2.html. Additionally, I have also provided the configuration files that I used for Sun Web Server to make it work with the PHP installation that came as part of Cool Stack 1.2

Price to Performance Data



Software Stack

Maximum No. of Concurrent Users

% of Requests

( Response Time < 2 seconds)

\*Price/Performance

Apache2, MySQL5 and SugarCRM 4.5.1i

900

98.89

39.17 $/User

Sun Web Server 7.0, MySQL5 and SugarCRM 4.5.1i

1300

98.32

27.12 $/User

\* Price sheet is provided at the end

Benchmark Configuration

The benchmark configuration consists of Sun SPARC Enterprise T5120 server. I used single server to host both MySQL database and SugarCRM. A separate storage device, the Sun Storage J4200, is attached to the server for MySQL database data and log files. This storage is mounted as two volumes, one for table spaces (/db/data/5.0) and the other for log files (/db/logs/5.0).

 Sun SPARC Enterprise T5120Sun SPARC Enterprise T5220  Sun Storage J4200Sun StorageTek2540




Hardware Configuration

Sun SPARC Enterprise T5120 Server

  • 1 UltraSPARC-T2 Processor 1.2GHz (8 Cores, 8 h/w threads per core)

  • 16GB RAM

  • 2 x 146GB, 10K RPM Internal SAS Disks

  • 4 x 10/100/1000Mbps Ethernet, 2 PCI-X, 4 PCIe

Sun Storage J4200

  • 146GB (12 x 146GB SAS Disks)

  • Three 3Gb/s Port Connections

Preparing Database

We used Tidbit, a kit provided by SugarCRM to generate the seed data. Using the kit, the SugarCRM database is populated with 2000 users with a load factor of 2000, that breaks down to the following number of records for each module/business object.



Business Objects

Number of Records

Accounts

2000

Calls

48000

Emails

32000

Contacts

8000

Leads

8000

Opportunities

4000

Cases

8000

Bugs

6000

Meetings

16000

Tasks

8000

Notes

8000

Total Records\*

\*602,644


A total of 602,644 records were inserted to initialize the database for the tests. It includes the main objects and the relationship rows that link the data together.

Tuning Options Used

APC (Alternative PHP Cache) which caches the PHP opcodes that comes bundled with the Coolstack 1.2 is a good start. However, we compiled APC the latest version of APC 3.0.16 with the following options using Sun Studio compilers:

#cd APC-3.0.16

#phpize

#./configure --enable-apc --enable-apc-sem --enable-apc-mmap --with-apxs=/opt/coo

lstack/apache2/bin/apxs –with-php-config=/opt/coolstack/php5/bin/php-config

#make

#make install



For Sun Web Server 7.0, I've used the eAccelerator 0.9.5.2 for caching. The tuning for eAccelerator is also provided below:

Solaris 10 Operating System:

We have applied the following tuning options to the /etc/system file

set rlim_fd_max=32768
set rlim_fd_cur=32768
set ip_squeue_soft_ring=1
set ip:ip_soft_rings_cnt=8
set autoup=900

Also to support higher http requests the following ndd parameters were applied.

ndd -set /dev/tcp tcp_conn_req_max_q 16384
ndd -set /dev/tcp tcp_conn_req_max_q0 16384
ndd -set /dev/tcp tcp_naglim_def 1
ndd -set /dev/tcp tcp_smallest_anon_port 2048

Apache2 Configured and Optimized for SugarCRM

The default configuration of Apache httpd.conf loads several modules that are not needed to run SugarCRM. Depending on your web site requirements you can disable most of them. We have used the following minimal set of modules to successfully run SugarCRM.

#httpd.conf

ServerRoot "/opt/coolstack/apache2"

Listen 0.0.0.0:80

LoadModule authz_host_module modules/mod_authz_host.so
LoadModule cache_module modules/mod_cache.so
LoadModule mem_cache_module modules/mod_mem_cache.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule php5_module modules/libphp5.so

<IfModule !mpm_netware_module>
User webservd
Group webservd
</IfModule>

<IfModule mime_module>
PHPINIDir /opt/coolstack/php5/lib
AddType application/x-httpd-php .php .phtml
AddType application/x-httpd-php-source .phps
</IfModule>

EnableMMAP Off
EnableSendfile Off

Include conf/extra/httpd-mpm.conf

<IfModule mod_mem_cache.c>
CacheEnable mem /
MCacheSize 32768
MCacheMaxObjectCount 8192
MCacheMinObjectSize 1
MCacheMaxObjectSize 2048
</IfModule>


#httpd-mpm.conf

<IfModule mpm_prefork_module>
StartServers 1200
MinSpareServers 2000
MaxSpareServers 2000
MaxRequestsPerChild 0
ListenBacklog 8192
ServerLimit 2500
MaxClients 2500
</IfModule>

MySQL Tuning

Before creating the database, please make sure that the following file is saved in /etc/my.cnf in order to pick up the InnoDB configuration while creating the tablespaces and the logfiles. The settings applied to query cache, table cache and thread concurrency. This helps at higher loads and reduces the latency.

#/etc/my.cnf


[mysqld]
port = 3306
socket = /tmp/mysql.sock
datadir = "/db/data/5.0"
back_log = 50
max_connections = 5000
max_connect_errors = 10
table_cache = 2048
max_allowed_packet = 2M
binlog_cache_size = 8M
max_heap_table_size = 128M
sort_buffer_size = 64M
join_buffer_size = 128M
thread_cache_size = 3000
thread_concurrency = 3000
record_buffer = 8M
query_cache_size = 256M
query_cache_type = 1
query_cache_limit = 4M
query_prealloc_size=65536
ft_min_word_len = 4

default-storage-engine = innodb
thread_stack = 192K
transaction_isolation = REPEATABLE-READ
tmp_table_size = 1024M
log_long_format

server-id = 1
key_buffer_size = 64M
read_buffer_size = 8M
read_rnd_buffer_size = 64M
bulk_insert_buffer_size = 128M

innodb_additional_mem_pool_size = 1024M
innodb_buffer_pool_size = 2G
innodb_data_file_path = ibdata1:4096M:autoextend
innodb_data_home_dir = /db/data/5.0
innodb_file_io_threads = 8
innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 1
innodb_log_buffer_size = 16M
innodb_log_file_size = 1G
innodb_log_files_in_group = 2
innodb_log_group_home_dir = /db/logs/5.0
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 120

MPSS and libumem libraries were pre-loaded by making changes to the mysqld_safe script. More information regarding MySQL performance tuning can be found here: http://blogs.sun.com/yufei/entry/mysql_innodb_performance_tuning_on

#mysqld_safe file modified to pre-load MPSS and libumem libraries
MPSSHEAP=256M
export MPSSHEAP
LD_PRELOAD_64="/usr/lib/sparcv9/mpss.so.1 /usr/lib/sparcv9/libumem.so.1"
export LD_PRELOAD_64

PHP Tuning

Some of these settings are standard except however we did make changes to the apc default settings.

[PHP]
cgi.fix_pathinfo = 1
memory_limit = 320M ; Maximum amount of memory a script may consume (8MB)
default_socket_timeout = 1800
safe_mode = 0
post_max_size = 20M
upload_max_filesize = 20M

[Session]
session.use_cookies = 1
session.cookie_lifetime = 0
session.gc_probability = 1
session.gc_divisor = 5000
session.gc_maxlifetime = 0
session.entropy_file = "/dev/urandom"
;session.cache_expire = 300
;session.cache_limiter = nocache
session.save_path = "/tmp/sessions"

include_path=/opt/coolstack/php5/lib/php:.:
extension_dir=/opt/coolstack/php5/lib/php/extensions/no-debug-non-zts-20060613
extension="mysql.so"
extension="mysqli.so"
extension="curl.so"
extension="zlib.so"



APC Tuning:

Copy the following APC tuning options to the PHP.ini file.

#APC 3.0.16 tuning below

extension="apc.so"
apc.enabled=1
apc.shm_segments=1
apc.shm_size=128
apc.num_files_hint=2048
apc.user_entries_hint=4096
apc.gc_ttl=3600
apc.cache_by_default=1
apc.mmap_file_mask=/dev/zero
apc.slam_defense=0
apc.file_update_protection=2
apc.enable_cli=0
apc.max_file_size=1M
apc.stat=0
apc.write_lock=0
apc.report_autofilter=0
apc.include_once_override=1
apc.localcache=0

Sun Web Server 7 and eAccelerator 0.9.5.2 Tuning

#Sun Web Server magnus.conf

#
# Copyright 2006 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
Init fn="load-modules" shlib="libfastcgi.so"



#Sun Web Server obj.conf

#
# Copyright 2006 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# You can edit this file, but comments and formatting changes
# might be lost when you use the administration GUI or CLI.

<Object name="default">
NameTrans fn="assign-name" from="\*.php" name="fcgiPHP"
PathCheck fn="find-index" index-names="index.html,home.html,index.jsp,index.php"
ObjectType fn="type-by-extension"
ObjectType fn="force-type" type="text/plain"
Service method="(GET|HEAD|POST)" type="\*~magnus-internal/\*" fn="send-file"
AddLog fn="flex-log"
</Object>

<Object name="fcgiPHP">
Service fn="responder-fastcgi" app-path="/opt/coolstack/php5/bin/php-cgi" bind-path="localhost:3101" app-env="PHPRC=/opt/coolstack/php5" app-env="PHP_FCGI_CHILDREN=600" app-env="PHP_FCGI_MAX_REQUEST=20000000" app-env="FCGI_WEB_SERVER_ADDRS=127.0.0.1" req-retry="10" min-procs="1" max-procs="1" restart-interval="0" reuse-connection="true" resp-timeout="9999" listen-queue="8192"
</Object>

<Object name="j2ee">
Service fn="service-j2ee" method="\*"
</Object>

<Object name="es-internal">
</Object>

<Object name="cgi">
ObjectType fn="force-type" type="magnus-internal/cgi"
Service fn="send-cgi"
</Object>

<Object name="send-precompressed">
PathCheck fn="find-compressed"
</Object>

<Object name="compress-on-demand">
Output fn="insert-filter" filter="http-compression"
</Object>




#
# eAccelerator 0.9.5.2 Tuning
#

extension="eaccelerator-0952.so"
eaccelerator.shm_size="128"
eaccelerator.cache_dir="/tmp/eaccelerator"
eaccelerator.enable="1"
eaccelerator.optimizer="1"
eaccelerator.check_mtime="0"
eaccelerator.debug="0"
eaccelerator.filter=""
eaccelerator.shm_max="0"
eaccelerator.shm_ttl="0"
eaccelerator.shm_prune_period="0"
eaccelerator.shm_only="0"
eaccelerator.compress="0"
eaccelerator.compress_level="9"
eaccelerator.allowed_admin_path="/sun/webserver7/https-etg06/docs"

SugarCRM Tuning

I've applied the standard tuning recommended for SugarCRM along with some of the suggestions made here http://www.sugarcrm.com/wiki/index.php?title=Performance_Tweaks_for_Large_Systems. I've also disabled the session.gc_maxlifetime by setting to 0. Please note that if you have policy to expire the sessions it is recommended to use “cron” entry to clear the sessions files from time to time.

Conclusion

In this benchmark, I've been able to get the best price to performance out of the Sun Web Stack for SugarCRM on Sun SPARC Enterprise T5120 and Sun Storage J4200.



Note:

The following bill of material has been used to work out the price of the server and the storage system.

Product Name

Description

Part No.


Quantity

List Price

Sun SPARC Enterprise T5120 Server


Sun SPARC Enterprise T5120 Server, 8 Core 1.2GHz UltraSPARC T2 Processor, 16 GB FBDIMM Memory (8 x 2 GB), 2 x 146 GB 10000 rpm SAS Disks, 1 DVD+/-RW, 4 x 10/100/1000 Ethernet Ports

SECPCEF1Z


1

21,995.00


Power Cord

Power Cord Kit, North American/Asian, RoHS Compliant

X311L

4

0.00

Support

Sun SPARC Enterprise T5120 8-Core Warranty Upgrade to 3 Year Silver Support

IWU-T5120-8-3S

1

4,588.56


SUB TOTAL



26,583.56

Sun Storage J4200

Sun Storage J4200 Rack-Ready Chassis, No Drives, 2 Redundant AC Power Supplies and 2 Redundant Cooling Fans, Rail Kit, Single SAS I/O Module, 1 x .5M SAS Cable, RoHS-5 Compliant

XTA4200R00A10DISK

1

2,675.00

Sleds

Sun Storage J4200 and J4400 Array, 6 Air Management Sleds, RoHS-5 Compliant

XTA-JBOD-6AMS

2

0.00

Rail Kit

2U Universal Rack Rail Kit, RoHS-5 Compliant

XTA-4200-2URK-19U

1

125.00

Disk

1 Internal 146 GB 15000 rpm SAS Disk Drive, 3.5-Inch with Bracket, RoHS-6 Compliant

XTA-SS1NJ-146G15K

12

400.00

HBA

Sun StorageTek PCIe SAS Host Bus Adapter, 8 Port, RoHS-6 Compliant

SG-XPCIE8SAS-E-Z

1

550.00

Support

Sun Storage J4200 Array Warranty Upgrade to 3 Year Silver Support

IWU-STJ4200-3S

1

516.67


SUB TOTAL



8,666.67


TOTAL



35,250.23



Wednesday Jul 09, 2008

Using Sun Storage J4200/J4400 Arrays with MySQL and SugarCRM

Using Sun Storage J4200/J4400 Arrays with MySQL and SugarCRM


The Sun Storage J4200/J4400 Arrays are general purpose, high-availability, and cost-effective 2RU, 12-disk SCSI Enclosure Services (SES) disk tray system (J4200) and a 4RU, 24-disk SES disk tray system. Additional Information on the storage arrays can be obtained here http://docs.sun.com/source/820-3222-10/index.html.

We are working on a benchmark with the latest SugarCRM 4.5.1 with the intent to measure the price to performance of using Sun Storage J4200/J4400 Arrays. Please stay tuned for a more detailed blog on our findings. Other than the storage arrays replacing the ST2540 this benchmark is similar to the one we did recently http://blogs.sun.com/vanga/entry/scaling_sugarcrm_with_mysql_on

Monday Feb 25, 2008

Scaling SugarCRM with MySQL on Sun's Coolthreads server

Shows how to install and configure SugarCRM on Solaris Operating Sytem to achieve under 2 second response times for more than 90% of requests accessed by 900 concurrent users on Sun's Coolthreads Server. A detailed account of MySQL, Apache and PHP tuning is also provided to reproduce these results.
[Read More]

Sunday Feb 24, 2008

OpenOffice.org Extension for SugarCRM

Source code for OpenOffice extension for SugarCRM is available for developers. The plugin allows access to the SugarCRM modules using SOAP API.

[Read More]
About

Satish Vanga

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today