Wednesday Jun 08, 2016

How to configure IP over Infiniband (IPoIB) on Oracle Solaris and SPARC servers

Recently we worked with an ISV who wanted to certify their application with Oracle SuperCluster in order to support their customers who are using this SPARC and Solaris based engineered system. Their application has a kernel module which needed to be tested with the infiniband driver.

We connected 2 SPARC T5-2 servers using 2 IB cards and an IB switch for this project. This simple configuration can simulate a 2 node connection of an Oracle SuperCluster.

The following steps  re needed to make such an IPoIB configuration work. It is very important to first be sure that the switch is configured properly:

For the IB switch :

define it into DNS
then ssh root@switchIP
passwd is "changeme" ( usually this is the password from the manufacture)
enablesm ( to enable the master )
getmaster ( to check the master )

If the switch master is not working, the IB interface status will show as “down” on the server even if all the cables and cards are connected to the switch correctly.

On the Solaris server (T5 server in this case):

Check the physical network interfaces available on the server: dladm show-phys

LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown ixgbe1
net2 Ethernet unknown 0 unknown ixgbe2
net0 Ethernet up 1000 full ixgbe0
net3 Ethernet unknown 0 unknown ixgbe3
net6 Ethernet up 1000 full vsw0
net9 Infiniband down 0 unknown ibp1
net5 Infiniband down 0 unknown ibp0

net4 Ethernet up 10 full usbecm2
net10 Ethernet up 40000 unknown vsw2
net11 Ethernet up 40000 unknown vsw1

Check only the IB cards: dladm show-ib

LINK HCAGUID PORTGUID PORT STATE GWNAME GWPORT PKEYS
net9 10E000015A7460 10E000015A7462 2 down -- -- FFFF
net5 10E000015A7460 10E000015A7461 1 down -- -- FFFF

The following 2 commands are not mandatory, but will help you to easier identify the IB

dladm rename-link net9 ibp1

dladm rename-link net5 ibp0

Check renaming: dladm show-phys

LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown ixgbe1
net2 Ethernet unknown 0 unknown ixgbe2
net0 Ethernet up 1000 full ixgbe0
net3 Ethernet unknown 0 unknown ixgbe3
net6 Ethernet up 1000 full vsw0
ibp1 Infiniband down 0 unknown ibp1
ibp0 Infiniband down 0 unknown ibp0
net4 Ethernet up 10 full usbecm2
net10 Ethernet up 40000 unknown vsw2
net11 Ethernet up 40000 unknown vsw1

dladm show-ib

LINK HCAGUID PORTGUID PORT STATE GWNAME GWPORT PKEYS
ibp0 10E000015A7380 10E000015A7381 1 up -- -- FFFF
ibp1 10E000015A7380 10E000015A7382 2 down -- -- FFFF

Create IB default partition:

(the name is under GWPORT column resulted from previous command/)

dladm create-part -l ibp0 -P ffff ffff.ibp0

dladm show-part

LINK PKEY OVER STATE FLAGS
ffff.ibp0 FFFF ibp0 unknown ----

Create and assign the IP address :

ipadm create-ip ffff.ib0
ipadm create-addr -T static -a 10.1.10.11/24 ffff.ib0/v4


Important commands to check the status of the connection:
dladm show-ib
ibhosts
ibstat
ibswitches
iblinkinfo.pl –R

grep pciex15b3 /etc/path_to_inst


Should you need further help with your IPoIB configurations on SPARC servers, you can contact us at isvsupport_ww@oracle.com.


Monday Dec 21, 2015

Scaling Intellect MH on Oracle SPARC to 25,000 TPS


Intellect Design Arena Ltd, a Polaris Group company, is a global leader in Financial Technology for Banking, Insurance and other Financial Services.

A joint performance and scalability testing exercise was conducted by Intellect Design and Oracle Engineering teams to study the performance and scalability of Intellect MH on Oracle SPARC systems. The activity was aimed at scaling up the application load in terms of Transactions Per Second (TPS) with a workload that consisted of a mix of 10 OLTP scenarios with audit logging enabled, as well as some related batch scenarios. 

Intellect FT Message Hub (MH) is a lightweight Java based integration platform that facilitates seamless and transparent integration of business applications. It reduces the complexity of integrating disparate applications by leveraging the principles of Service Oriented Architecture (SOA).   

MH provides a function to exchange data online and in batch mode, and enables various interfaces, integration of customer access channels like PCs connected to the Internet and mobile phones, and connection with external financial intelligence institutions and settlement networks.

Message Hub serves as a pass-through station between business applications. It provides a common platform for the customer to do business transactions. The Listeners will be an entry point for front-end systems to perform straight through transaction processing. Transaction Rule Engine (TRE) communicates with the Communication Engine and the Message Engine for communication with the host and message formatting requirements respectively. These engines coordinate the operations based on configured rules. 

The following are the key features of Intellect FT Message Hub:

  • Routing
  • Message transformation
  • Message enhancement
  • Protocol transformation
  • Transaction workflow management
  • Synchronous/Asynchronous transaction
  • Pre/Post process transaction
  • Post dated/ scheduled transaction 
  • Fail-over
  • Support custom action 

MH supports all industry standard protocols including SOAP over HTTP, SOAP over JMS, RESTful, TCP/IP, MQ, JMS, HTTP/s, EJB, File, FTP, SFTP, SMTP, IMAP, POP3. The product also supports a wide range of messaging standards such as SWIFT, ISO 8583, XML, SOAP, JSON, Fixed Length, NVP, Delimited, EBCDIC, POJO, MAP.

The diagram below shows the technical architecture of Intellect FT Message Hub.


Test Details 

OLTP Tests:

The most common MH transactions were covered in the tests. Different transactions were tested with different listeners and communication engines. A mix of the following 10 OLTP Scenarios was tested:

Audit logging:

Audit records  that contain the request and message details are inserted into  the MH database, once at the point of receipt of the message in MH and a second time after the message has just been processed but before the transmission of the message to the external system using communication engines.

Batch Tests:

In batch processing, records are picked up by Intellect MH from a preconfigured location. The files are processed and records are submitted to external systems (stubs) using JMS communication engines.  Multiple files are processed by the managed servers in parallel. The following batch processes were tested:

Hardware Details:

The application was deployed on Oracle SPARC T5 systems, FS1 Flash storage and ZS3 storage. 

Software Details:

  • Oracle Solaris 11.2 
  • Oracle Database 12c RAC 
  • Oracle Weblogic 12c Cluster 
  • Oracle HTTP Server 12c 
  • Oracle JDK 8 
  • Apache JMeter  
  • IBM MQ 
  • Polaris FT Message Hub 15.1 


Test Results:

The systems near linearly scaled up to 25,000 TPS, with an average response time of 323 ms and about 52K concurrent users.  For the batch tests around 10 million records (1000 Files, each containing 10000 records) were processed in 21 minutes. 

These results are 6x better than results seen currently on typical large customer deployments.

More Information: 

For more information, details and system sizing help you can contact the team via isvsupport_ww@oracle.com.



Wednesday Jul 04, 2012

New Red Hat Linux to Solaris Porting Guide Published

The new Red Hat Linux to Solaris Porting Guide is now available. It covers all important scenarios to make Linux software (including other distributions like SUSe) available on Oracle Solaris 11.

About

Technical insights for and about ISV applications on Solaris and SPARC.
Application Availability, How-tos, Best Practices, Application Tuning and Results

Search

Archives
« July 2016
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
8
9
10
11
13
14
15
16
17
18
19
20
21
23
24
25
26
27
28
29
30
31
      
Today