Oracle Cloud Infrastructure Logging Analytics best practices series

February 28, 2024 | 3 minute read
Royce Fu
Principal Database and O&M Solution Architect
This is a syndicated post, view the original post
Text Size 100%:

Oracle Cloud Infrastructure (OCI) Logging Analytics provides advanced tools for log management and analysis, crucial for cost optimization and insightful decision-making. By adopting strategic practices in data retention and understanding the intricacies of storage pricing, businesses can significantly reduce operational expenses. The service's suite of features, including Log Parsers and Management Agents, enables the transformation of voluminous log data into valuable insights, aiding in application observability, system monitoring, security enhancement, and regulatory compliance. Leveraging these tools efficiently ensures organizations not only optimize their cloud expenses but also gain a competitive edge through data-driven strategies.

In the following blog series, you’ll be equipped with best practices that not only streamline your operations but also ensure that your use of OCI Logging Analytics is as cost-effective as possible. Let’s embark on this journey to maximize efficiency and optimize costs in the realm of cloud log management.
 

Part 1: Management Agent Tuning in log data analysis creates operational superiority

Embrace the power of Management Tuning in log data analysis to elevate your organization's efficiency and performance, paving the way for operational superiority.

  • Ingesting logs via Unified Monitoring Agent (FluentD) 
  • Ingesting logs via Management Agent
  • Essential facts about Management Agent configuration and Performance Tuning
  • Improve Logging Analytics ingestion performance from the OCI Logging Service and Service Connector Hub, by reducing log volume for logs
  • Alternate log collection - forward logs to the central syslog server

Part 2: Custom Log Sources and Parsers Tips to enhance proficiency and maximize value extracted from data

Use tips and best practices to enhance proficiency in creating and managing log parsers and log sources within Logging Analytics so that you can effectively process and analyze logs, ensuring that maximum value is extracted from data.

  • Customize Oracle-defined Log Source
  • Duplicate feature to clone Oracle-defined content 
  • Create a new Log Parser from scratch using the Guided Regex tool 
  • Use Data Filters in Log Source to mask sensitive data
  • Use Field Enrichment to integrate with Oracle Threat Intelligence Service
  • Use Labels to speed up Error Detection in log stream
  • Import knowledge content from Oracle Quickstart OCI Observability and Management Services GitHub community repository
Figure 1:  Logging Analytics Log Parsing and Processing Architecture

 

Part 3: OCI Logging Analytics cost optimization best practices for data retention

Explore the keys to optimizing expenses in OCI Logging Analytics in understanding and implementing best practices for data retention and comprehending the nuances of Logging Analytics storage pricing.

  • Logging Analytics storage essential facts:
  • Multiple and different log sources from OCI, on-premises, or third-party clouds:
  • Amongst the log data, you can categorize them into different types:
  • Best practices for managing data retention and storage of Logging Analytics:
  • Pricing tiers
  • Logging Analytics cost monitoring and alerts
Figure 2. Logging Analytics Active Storage and Archival Storage Diagram

Have more questions or want to engage with deep experts? 

Please check out our Oracle Cloud Customer Connect Observability and Management Community. Ask questions, connect with experts, and share your successes, thoughts, and ideas about Oracle Cloud Observability and Management solutions (including OCI Application Performance Monitoring, Stack Monitoring, Logging Analytics, Database Management, and Operations Insights).

Resources:


 

Royce Fu

Principal Database and O&M Solution Architect

Royce Fu is the Principal Database Solution Architect of the North America Cloud Technology and Engineering Team. Royce's area of specialty is core Database Technology and OCI O&M especially in Database Platform Engineering, Architecture, and Integration. He started his career as Java software engineer and spent over a decade in database engineering and architecture.

Show more