GPU Accelerated AI Camera Analytics On OCI

GPU-accelerated AI camera analytics on OCI with edge/cloud processing, queue-driven inference, role-aware rules, and auditable operations.

Audience: enterprise architects, cloud platform teams, and customers designing production-grade AI camera, IP camera, and video analytics platforms at scale.

1. Introduction

Many computer vision initiatives stop at identifying objects in a frame by drawing bounding boxes around people, vehicles, or equipment. However, production-grade enterprise systems demand significantly more. Organizations require a platform capable of ingesting multiple video feeds, handling burst traffic efficiently, executing GPU inference economically, and delivering decision-ready events directly into operational systems.

This article presents a scalable Oracle Cloud Infrastructure (OCI) reference architecture for GPU-accelerated AI camera and video analytics solutions. The architecture treats smart camera events, traditional IP camera streams, uploaded images, and short video clips as potential decision points. CPU-intensive media processing is separated from GPU-based model execution, OCI Queue is used to provide backpressure handling, and policy-driven rules are applied before alerts are surfaced to operators.

The result is a shift from simple detection to intelligent decisioning. Instead of only identifying what appears in a frame, the platform evaluates whether an event is expected, suspicious, urgent, or operationally relevant.

It is important to view this as a reusable reference architecture rather than a solution designed for a single business scenario. The components can be adapted and extended based on specific customer requirements and operational environments. Model evaluation and selection are intentionally outside the scope of this document because the optimal inference model depends on the target use case.

To support these operational requirements, the reference architecture is built around a set of foundational design principles that prioritize scalability, modularity, observability, and decision-driven automation.

1.1 Key Points

• Separate stream handling from GPU inference so CPU containers and GPU pools can scale independently.
• Use OCI Queue as the durable buffer between frame producers and inference consumers.
• Make role, zone, time, confidence and event history first-class inputs to the decision layer.
• Store selected frames, detections, rules, and outcomes in Object Storage so events are auditable and replayable.
• A customizable GPU pipeline to allow customers use custom models, rules, latency control and model-replacement flexibility while maintaining scalability in every aspect.
• Support a hybrid edge/cloud split so smart cameras can run lightweight detection locally while traditional IP cameras can stream to OCI-side processing.
• Reduce bandwidth by using low-resolution detection streams & event-triggered clips
• Treat device identity, camera onboarding, certificate rotation and retention policies as production architecture concerns.
• Route alerts through push notifications, email, SMS, webhooks, ticketing tools, or service-management platforms only after policy evaluation.

These design principles are particularly valuable in environments where AI camera systems must operate at scale and respond to dynamic workloads.

1.2 When to Use This Pattern

This reference architecture is well suited for environments where:

• Multiple camera streams, uploaded media assets, or image batches arrive at inconsistent or bursty rates. 
• GPU infrastructure costs require careful optimization of utilization, batching, and scale-in behavior. 
• Raw detections alone are insufficient, and alerts must be interpreted using business context. 
• Customers require operational auditability, replay capability, policy traceability.
• There is a need for integration of one or more blocks of this architecture with existing enterprise systems.
• Smart cameras, mobile edge devices, and traditional IP cameras must coexist in the same architecture.
• Bandwidth, privacy and storage cost require event-based upload instead of always uploading every high-resolution frame.
• Enterprise customers need multi-tenant controls across sites, cameras, users, device ownership, and retention policies.

2. Reference Architecture

The architecture is designed to isolate ingestion, inference, and decisioning into independently scalable layers. This separation allows organizations to optimize GPU utilization while maintaining operational resilience during burst traffic conditions. 

The central design principle of this architecture is decoupling. Stream processors manage camera sessions, sampling, buffering, and reconnect logic, while GPU workers independently consume queued batches and focus exclusively on inference execution.

The following sections describe the end-to-end architecture and its operational responsibilities. One of the defining characteristics of this solution is its scalability and adaptability. Each component is implemented as a pluggable module exposed through REST APIs, enabling customers to use individual capabilities independently or as part of a complete end-to-end solution.

This effectively provides modular building blocks for enterprise-grade visual AI implementations as shown below.

2.1 Camera Deployment Topologies

Production AI camera systems typically need to support multiple device classes rather than assuming that every source behaves the same way. Considering this point the architecture supports the following topologies as first-class inputs into the ingestion and processing layers.

Smart / AI Cameras: Smart cameras run lightweight models on the device using an embedded SoC, ONNX Runtime Mobile or TensorFlow Lite. They can perform local motion/person filtering, retain continuous footage locally on MicroSD and upload only event metadata or short clips to OCI.

Traditional IP Cameras: Traditional IP cameras usually provide RTSP/H.264 or H.265 streams and may not have meaningful onboard AI. These cameras should stream through a VPN, TLS tunnel or outbound connector into the OCI-side stream processing block where frame decoding and preprocessing for inference preparation can run.

Mobile/Edge Devices: Mobile or constrained edge devices can run a quantized local model to decide when to publish metadata, thumbnails or short evidence clips. This keeps the architecture useful for temporary monitoring, field operations, and low-bandwidth locations.

2.2 Flow Through the System

Ingest: Camera streams, uploaded images, mobile-edge events, and video clips enter the system through a Load Balancer, API Gateway, producer service, or secure camera bridge. The ingress layer remains stateless to ensure predictable retry behavio and simplified routing across distributed workloads and routing across distributed workloads remain predictable.

Stream Processing: Once ingested, containerized stream-processing workers terminate RTSP sessions, decode H.264/H.265 streams, sample frames at configurable intervals. The code snippet below gives a sample JSON output for the Stream processing layer. 

{
  "camera_id": "cam1",
  "count": 1,
  "frames": [
    {
      "frame_id": 42,
      "camera_id": "cam1",
      "timestamp": 1747051200.123,
      "resolution": [
        1280,
        720
      ],
      "jpeg_base64": "/9j/4AAQSkZJRgABAQAAAQABAAD...",
      "session_status": "processing",
      "video_name": "idle worker cement.mp4",
      "source_type": "video_file",
      "total_frames": 3000,
      "progress_ratio": 0.14,
      "source_metadata": {
        "loop_iteration": 0,
        "source_frame_index": 420
      }
    }
  ]
}

Queue Layer: OCI Queue serves as the decoupling layer between producers and GPU consumers, enabling asynchronous communication and smoothing burst traffic conditions without overwhelming GPU resources.

GPU Inference: GPU inference workers consume queued batches, execute detection and classification models, and dynamically scale based on queue depth, latency, throughput, and GPU utilization metrics. The same inference tier can support custom models, customer-specific labels, and batch policies that would be difficult to express in a camera-only AI deployment. Given below a sample json output available at the rest endpoint of the inference engine. 

{
  "frame_id": 42,
  "camera_id": "cam1",
  "timestamp": "2026-05-19T12:00:00",
  "detections": [
    {
      "label": "person",
      "confidence": 0.9721,
      "bounding_box": [
        420.11,
        180.42,
        612.88,
        701.07
      ]
    }
  ],
  "inference_time_ms": 27.4,
  "session_status": "processing",
  "video_name": "idle worker cement.mp4",
  "source_type": "video_file",
  "total_frames": 3000,
  "progress_ratio": 0.14,
  "source_metadata": {
    "loop_iteration": 0,
    "source_frame_index": 420
  },
  "mode": null
}

Decision Services: The inference results are passed into decision services where the event journey becomes detect -> classify -> apply policy -> notify -> audit. Contextual rules evaluate role, zone, confidence score, time, device ownership, tenant policy and historical event patterns before generating alerts for dashboards, APIs, ticketing systems, webhooks, event brokers, or service-management tools.

Operations Layer: Finally, operational services including Object Storage, Logging, Monitoring, IAM, Vault, Connector Hub, Notifications, and lifecycle policies provide observability, retention, access control, secrets management, notification routing, replay support, and downstream integration capabilities across the platform.

2.3 Continuous vs Event-Based Upload

This architecture should not require continuous high-resolution cloud upload for every camera. Smart cameras can continuously record locally while uploading only event clips, thumbnails, and metadata when local thresholds pass. Traditional IP cameras can stream to a local or OCI-side bridge, where the processing layer samples frames, uses a low-resolution detect stream, and retains only the high-resolution clip window associated with a confirmed event.

Bandwidth is reduced through frame sampling, H.265 compression, configurable FPS, motion/person thresholds, zone filters, and short event clip durations. Customers can still enable continuous recording for selected cameras or premium tiers, but the reference architecture considers event-triggered upload as the default implementation pattern.

2.4 OCI Queue and OCI Streaming Roles

OCI Queue can be described as the durable inference work buffer. It absorbs bursty frame or clip workloads and lets GPU workers consume at a controlled pace. OCI Streaming has not been included in the current reference architecture, as the design does not require a Kafka-compatible event streaming layer. However, it can be incorporated if there is a customer requirement for downstream event distribution, analytics consumers, audit pipelines, or integration with external systems that would benefit from a Kafka-compatible event bus.

2.5 Edge AI vs OCI Vision vs Custom GPU on OCI

At this stage, it is worth evaluating the trade-offs between managed services and custom-built components, taking into account factors such as scalability, operational overhead, flexibility, cost, and long-term maintainability.

Option	Best fit	Benefits	Trade-offs
Edge AI on camera/device	Low-latency filtering, bandwidth reduction, privacy-sensitive locations	Reduces cloud upload and can keep continuous recording local	Limited model size, device heterogeneity, firmware lifecycle, and harder centralized tuning
OCI Vision	Quick start for managed image or video analysis	Simpler operations and faster initial adoption	Less control over BYOM, batching, custom post-processing, and specialized decision rules
Custom GPU on OCI	BYOM, specialized models, batching, latency control, and custom policy pipelines	Maximum flexibility, traceability, and cost optimization through autoscaling	Requires GPU capacity planning, MLOps, model lifecycle management, and operational tuning

2.6 Architecture Layer Responsibilities

The following table elaborates the various layers in the reference architecture and the responsibilities of each layer.

Layer	OCI services	Primary responsibility	Operational signal
Ingress and capture	OCI Load Balancer, API Gateway, Object Storage	Accept RTSP streams, video clips, and image batches; keep producers stateless and retryable.	Connection count, ingest errors, upload volume
Stream processing	Container Instances	Manage stream connections, decode frames, buffer workloads, reconnect sessions, and publish inference work items.	Sessions per worker, CPU, reconnect rate
Decoupling layer	OCI Queue	Absorb bursty traffic and let inference workers consume at the right pace.	Queue depth, latency, consumer lag
GPU Inference Engine	OCI Compute GPU instance pools	Run object detection, classification, tracking, temporal correlation, and batch optimization.	GPU utilization, inference latency, batch size
Decision Engine & Rule Services	Functions, custom rules service, Notifications, Connector Hub	Apply role, zone, time, confidence, temporal history, and policy rules before generating alerts, webhooks, or downstream events.	Alert precision, rule matches, escalation volume
Audit and operations	Object Storage, Logging, Monitoring, Vault, IAM	Store frames/events, protect secrets, observe health, and support replay.	Retention, access events, SLO burn rate

2.7 Tenant, Device, and User Model

For SaaS or enterprise multi-site deployments, tenants should own sites, sites should own cameras, and users should receive access only through tenant/site/camera policies. Retention, alerting, review queues, and integration targets are supposed to be separable per tenant so that one customer, facility, or business unit cannot affect another.

2.8 Camera Onboarding and Identity

For camera onboarding, the platform is expected to support capabilities such as device registration, tenant association, certificate issuance and rotation, mTLS-based authentication, camera key management, and signed firmware validation. Where supported by the hardware, a root of trust can also be leveraged to further strengthen device security. To simplify deployment and reduce security concerns, cameras are designed to initiate outbound connections wherever possible, eliminating the need for customers to expose inbound ports within their local networks.

3. Design Patterns for Production

The architecture layers described in the previous section are supported by a set of production-focused design patterns that improve scalability, GPU efficiency, and decision accuracy. These patterns define how ingestion, inference, and decision services operate together as a cohesive visual AI platform on OCI. Rather than treating detection as an isolated AI task, the platform applies distributed processing and contextual decisioning to support enterprise-scale operational workloads. 

3.1 Decoupled Perception Pattern

What it is: The stream-processing layer is separated from the GPU inference layer. CPU-based workers handle media protocol management, frame sampling, buffering, and reconnect logic, while GPU workers focus exclusively on model inference in a modular and replaceable architecture.

Why it matters: GPU infrastructure should maximize inference throughput instead of spending compute cycles managing unstable network streams or session recovery.

OCI design choice: Deploy stream-processing services as containerized workloads while running inference workers as scalable GPU instance pools connected through OCI Queue.

3.2 Queue-Driven Autoscaling Pattern

What it is: The queue layer acts as the operational control point between ingestion and inference. Queue backlog depth, inference latency, consumer lag, and GPU utilization become the primary scaling signals used to dynamically adjust inference capacity.

Why it matters: Camera feeds, uploaded clips, and image batches rarely arrive at predictable rates. Queue-driven autoscaling absorbs burst traffic while preventing unnecessary GPU overprovisioning.

OCI design choice: Use OCI Monitoring alarms with OCI Functions or autoscaling policies to dynamically scale GPU inference pools. Stream-processing containers can also scale independently based on active session count, CPU utilization, ingest throughput and publish latency. A practical starting point is to scale CPU workers out when average CPU remains above 65-70%, session capacity is exhausted, or frame processing rate falls below 80 frames per minute.

3.3 Context-Aware Decision Pattern

What it is: Inference results are evaluated as part of a complete event journey: detect -> classify -> apply policy -> notify -> audit. The decision layer uses contextual attributes such as role, zone, time, motion, confidence score, tenant policy, device ownership, and historical event patterns before alerts are generated.

Why it matters: A detected object or person is not automatically an operational incident. The same visual signal may be considered expected, suspicious, or urgent depending on environmental context and policy requirements.

OCI design choice: Externalize decision rules through JSON, YAML, or dedicated policy services so customers can adapt operational logic without retraining AI models. A snippet of the YAML file used in this reference architecture is given below with an example rule to build similar config files for production implementations. 

rules:
  # -------------------------
  # Person Security Rules
  # -------------------------
  - id: cam1_zone_entry
    enabled: true
    description: Trigger an alert when an unauthorized person is detected inside the configured restricted alert zone for camera cam1.
    match:
      mode: person_security
      clip_label: Non Doctor
      zone: cam1_alert_zone
      restricted_zone: true
      authorization_status: unauthorized
      min_duration_seconds: 0.0
      min_confidence_avg: 0.0
    alert:
      event_type: zone_entry_detected
      severity: high
      message: Unauthorized person detected inside the cam1 restricted alert zone
....

3.4 Replaceable Model Pattern

What it is: Detection, classification, tracking, and rule-evaluation components remain independently versioned and replaceable throughout the platform lifecycle.

Why it matters: Customers may initially deploy RT-DETR, YOLO-family models, CLIP-style classifiers, or OCI Vision services, and later evolve based on changing accuracy, latency, compliance, or operational requirements.

OCI design choice: Externalize model configuration through JSON or YAML definitions, version model artifacts independently, and maintain inference metadata alongside event payloads for traceability and rollback. Given below is an YAML file for model configuration. 

inference:
  backend: "rtdetr"
  model_path: "models/rtdetr_r50vd"
  device: "cuda"
  confidence_threshold: 0.5
  label_allowlist:
    - "person"

clip:
  enabled: true
  model_name: "ViT-B-32"
  pretrained: "laion2b_s34b_b79k"

  target_labels:
    - "person"

  prompts:
    doctor:
      - "a doctor wearing a white coat"
      - "a doctor with a stethoscope"
      - "a medical professional in hospital"
      - "a doctor using a laptop in hospital"
      - "a healthcare worker attending a patient"

    non_doctor:
      - "a normal person"
      - "a hospital patient"
      - "a visitor in hospital"
      - "a person sitting casually"
      - "a person not working as a doctor"
      - "a patient lying on bed"

3.5 Auditable Event Pattern

What it is: Every operational event includes the associated frame reference, model version, confidence score, rule outcome, timestamp, downstream delivery target, and human-review status when review is required.

Why it matters: Enterprise operators must understand why an alert was generated, escalated, or suppressed to support operational trust, compliance, and continuous tuning.

OCI design choice: Store selected frames and structured event payloads in Object Storage, route downstream outputs through Connector Hub or APIs, and monitor operational telemetry end-to-end using OCI observability services. Sensitive workflows such as healthcare, security, and restricted-area decisions should preserve a human review loop and record operator outcomes for audit and tuning.

3.6 Notification and Escalation Pattern

What it is: Alerts are generated only after inference and policy evaluation. The platform can route validated events to Firebase/APNS push notifications, email, SMS, webhooks, ServiceNow, customer ticketing systems, or OCI Streaming consumers depending on severity and tenant policy.

Why it matters: Notification design determines whether the platform creates operational signal or alert noise. Escalation paths should include acknowledgement, retry, deduplication, suppression windows, and audit trails.

3.7 Failure Modes and Recovery Pattern

What it is: Production camera analytics should explicitly handle camera offline events, queue backlog, GPU unavailability, duplicate alerts, missed detections, storage upload failure, and notification delivery failure.

OCI design choice: Use camera heartbeat events, queue-depth alarms, GPU capacity checks, idempotent event IDs, retry policies, dead-letter handling, object storage upload verification and notification delivery status monitoring.

4. Core AI Components

The production architecture combines multiple AI processing stages to transform raw camera inputs into contextual and operationally relevant events. Rather than relying solely on object detection, the platform applies layered inference, contextual enrichment, and rule-driven evaluation to support scalable decision intelligence workflows.

4.1 Detection

The detection layer is responsible for identifying objects, people, vehicles, equipment, and other visual entities within incoming frames. This layer uses RT-DETR for object detection. In production environments, model selection should be based on measured tradeoffs across accuracy, latency, GPU memory consumption, throughput, and customer-specific object classes. To improve downstream inference quality, IoU-based post-processing is applied to reduce duplicate detections before contextual classification and rule evaluation occur. The code snippet below demonstrates how RT-DETR is loaded from Hugging Face and the GPU inference is executed.

from PIL import Image
import torch
from transformers import RTDetrForObjectDetection, RTDetrImageProcessor

class RTDetrDetector:
    def load(self, config):
        self.device = config.get("device", "cpu")
        self.confidence_threshold = config.get("confidence_threshold", 0.5)

        self.processor = RTDetrImageProcessor.from_pretrained(config["model_path"])
        self.model = RTDetrForObjectDetection.from_pretrained(config["model_path"])
        self.model.to(self.device).eval()

        if self.device.startswith("cuda") and config.get("use_fp16", True):
            self.model.half()

    def detect_batch(self, frames):
        pil_images = [Image.fromarray(frame[:, :, ::-1]) for frame in frames]
        inputs = self.processor(images=pil_images, return_tensors="pt")
        inputs = {k: v.to(self.device) for k, v in inputs.items()}

        with torch.inference_mode():
            outputs = self.model(**inputs)

        target_sizes = torch.tensor([image.size[::-1] for image in pil_images]).to(self.device)

        results = self.processor.post_process_object_detection(
            outputs,
            target_sizes=target_sizes,
            threshold=self.confidence_threshold,
        )

        return [
            [
                {
                    "label": self.model.config.id2label[int(label)],
                    "confidence": round(float(score), 4),
                    "bounding_box": [round(float(v), 2) for v in box],
                }
                for score, label, box in zip(
                    result["scores"].cpu(),
                    result["labels"].cpu(),
                    result["boxes"].cpu(),
                )
            ]
            for result in results
        ]

4.2 Classification and Context

Once objects are detected, contextual classification layers enrich the inference pipeline with operational meaning. A CLIP-style classifier or other visual encoder can be used to identify contextual signals such as staff, guards, visitors, vehicles, equipment, uniform colors, or unknown entities. These classifications provide additional context that helps downstream decision services distinguish between expected and suspicious activity patterns. These labels should be treated as probabilistic indicators rather than identity assertions. Human oversight and customer-defined policies remain important for sensitive operational workflows. 

4.3 Rule Evaluation

The rule-evaluation layer transforms AI inference outputs into actionable operational decisions. Inference results are evaluated using contextual signals such as role, zone, time, confidence score, temporal history, and policy definitions before alerts are generated. For example, the presence of an authorized role in a restricted corridor may suppress alerts, while an unknown role detected in the same area after hours may trigger escalation. To maintain operational flexibility, the rules layer should remain modular and externally configurable so customers can customize zones, thresholds, retention policies, escalation paths, and workflow integrations without replacing the underlying AI pipeline. The YAML file given below is used for custom labels. 

project:
  name: "guard-zone-exclusion"
  goal: "Detect authorized guards separately from non-guard people so restricted-zone alerts can be suppressed for guards."

canonical_labels:
  - security_guard
  - visitor

label_guidance:
  security_guard:
    description: "Authorized security personnel who should be excluded from restricted-zone alerts."
    include:
      - "Uniformed guards in grey shirts or the approved site uniform."
      - "Guards standing, walking, or patrolling inside or near restricted zones."
      - "Partially occluded guards if the uniform/role is still visually clear."
    exclude:
      - "Visitors, staff, or workers without the approved guard appearance."
      - "People wearing similar colors without guard context."
  visitor:
    description: "Any non-guard person who should still trigger restricted-zone alerts."
    include:
      - "Visitors, customers, or unknown people."
      - "Employees who are not part of the authorized guard group."
      - "People with unclear appearance when they cannot be confidently labeled as guards."
    exclude:
      - "Authorized guards in approved uniform."

annotation_rules:
  - "Draw one box tightly around each full visible person."
  - "If a person cannot be confidently assigned to security_guard, label them as visitor."
  - "Do not create a generic person class for this custom model; use the canonical labels only."
  - "Keep training examples balanced across lighting, camera angle, scale, and occlusion."
  - "Include both in-zone and out-of-zone samples so the detector learns appearance, not just location bias."

acceptance_criteria:
  - "security_guard and visitor are both detected in the validation set."
  - "Guards in restricted zones do not produce active alerts."
  - "Visitors in restricted zones do produce active alerts."

5. Customer Use Cases

These use cases apply across smart AI cameras, traditional IP cameras, mobile edge cameras, and uploaded video clips. The deployment topology affects bandwidth, local retention, and latency, but the decisioning pattern remains consistent.

Use case	Where it applies	Decision logic	Business outcome
Healthcare operations	Critical areas such as emergency rooms, ICU-adjacent zones, pharmacies, or restricted clinical environments.	Expected staff presence, unknown-person review, after-hours escalation.	Faster operational awareness without continuous manual video monitoring.
Security and restricted zones	Data centers, campuses, warehouses, laboratories, and controlled corridors.	Authorized roles suppress noise; unknown roles, incorrect zones, or after-hours activity trigger escalation.	Higher signal quality for security teams and improved alert prioritization.
Industrial and retail monitoring	Service counters, production floors, loading bays, queues, and waiting zones.	Combine presence, dwell time, motion, queue length, and business policy.	A foundation for throughput visibility, SLA monitoring, and capacity planning.
Vehicle search and tracking	Parking facilities, logistics hubs, smart campuses, toll checkpoints, warehouses, and secured entry zones.	Detect and classify vehicles using attributes such as license plate, vehicle type, color, movement direction, dwell time, or restricted-area presence. Correlate historical sightings and trigger alerts for watch listed or anomalous vehicles.	Improved operational visibility, faster incident investigation, automated vehicle monitoring, and enhanced perimeter security.

Given below are a few videos of the Use Cases explained above:

Healthcare Operation – The video demonstrates the alert getting generated when a person who is neither a doctor nor a patient enters a restricted zone in the hospital.

Security and restricted zones – This solution can be used for monitoring and alerting potential threats near main gates or access points. Individuals who remain stationary in these areas, other than authorized security personnel, are identified and flagged by the system, while moving objects are ignored to reduce false positives.

Industrial and retail monitoring – This video demonstrates a monitoring scenario in which an authorized shop supervisor wearing the required safety/hazard suit is identified but no alert is generated, while another individual without the required protective gear is flagged as unauthorized and alert is generated.

Vehicle search and tracking – This images below demonstrates how this solution can identify and track vehicles with selected attributes.

6. Production Readiness on OCI

Designing a scalable visual AI platform requires more than high-performing inference models. Production deployments need to also address operational resilience, elastic scaling, governance, observability, and long-term maintainability across distributed workloads. The following considerations help ensure that the reference architecture can operate reliably in enterprise environments while supporting evolving customer requirements and operational policies.

6.1 Sample Production Assumptions and NFRs

For production deployments, the sizing assumptions used in this reference architecture should be scaled based on the target workload. As a baseline, the architecture has been sized for approximately 50 camera feeds, sampled at 1-5 FPS for inference, using 720p video streams. Deployments involving higher camera counts, increased frame rates, or higher-resolution video should be sized accordingly. As a general guideline, capacity planning should target an average GPU utilization of around 70% to maintain performance while allowing headroom for workload fluctuations.

6.2 Scaling and Reliability

The architecture is designed around independently scalable processing layers that optimize GPU utilization while maintaining responsiveness during burst traffic conditions. Queue-driven orchestration plays a central role in balancing ingestion throughput, inference latency, and downstream decision processing.

Key operational considerations include:

• Use queue depth, inference latency, and GPU utilization together because relying on a single metric can create unstable scaling behavior.
• Keep model workers stateless wherever possible, storing state, frame references, temporal history, and replay data within OCI-managed services instead of local disks.
• Validate quotas, GPU availability, service limits, and regional capacity before committing to operational SLAs.
• Scale stream-processing containers independently from GPU inference workers so ingestion spikes do not unnecessarily increase GPU consumption.
• Monitor end-to-end inference latency, queue backlog growth, and consumer lag to maintain the agreed SLO for alert freshness and operator response.

6.3 Security, Privacy, and Governance

Because visual AI systems often process operationally sensitive media streams and event data, security and governance controls must be integrated throughout the platform lifecycle.

Recommended practices include:

• Apply least-privilege IAM policies, compartment boundaries, dynamic groups, and Vault-managed secrets across all infrastructure layers.
• Use private subnets, NSGs, service gateways, and private endpoints when customer security requirements demand isolated service communication paths.
• Encrypt stored media assets and structured event payloads, define retention schedules, and restrict direct access to raw frames.
• Treat contextual classifications as operational signals rather than identity assertions and avoid using visual inference to derive protected attributes.
• Preserve human review workflows for sensitive operational decisions and maintain audit trails that include model version, rule version, confidence score, timestamp, and operator outcomes.
• Use device identity controls such as certificate-based onboarding, mTLS, certificate rotation, signed firmware checks, camera key management, and optional hardware root of trust for supported devices.
• Generate short-lived Object Storage access URLs for clip playback and avoid exposing raw media paths directly to client applications.

6.4 Storage Lifecycle and Retention

Event clips and selected frames should be stored in OCI Object Storage with tenant-aware retention policies. Common examples include 7-day retention for standard event clips, 30-day retention for higher-priority security events, and 90-day retention for regulated or investigation workflows. Lifecycle policies can move older clips to Archive Storage or delete them automatically, reducing cost while supporting privacy and compliance requirements.

6.5 Failure Modes and Operational Controls

Operational runbooks should cover camera offline events, local storage fallback, queue backlog, GPU capacity exhaustion, duplicate alert suppression, missed detections, Object Storage upload failure, and notification delivery failure. Each failure mode should have an owner, metric, alarm threshold, retry behavior, and audit record.

7. Current Capabilities and Roadmap

The reference architecture already supports a broad set of production-oriented visual AI capabilities designed for scalable ingestion, GPU inference, contextual decisioning, and operational observability. The current implementation establishes the foundational building blocks for enterprise-grade image and video analytics while also providing a roadmap for more advanced automation, tracking, and adaptive intelligence workflows. The high level summery of current capabilities can be found in the Github link.

7.1 Current Capabilities

• Real-time stream and clip ingestion with containerized stream processors.
• Person and object detection with RT-DETR-style inference workers.
• Dwell-time and zone analytics for richer operational intelligence.
• Role/context classification with CLIP-style embeddings or customer-selected models.
• IoU-based refinement to reduce duplicate detections.
• YAML rule evaluation for authorized, unauthorized, idle, and review-required events.
• Queue-based separation between stream processing and GPU inference.

7.2 Roadmap

• Multi-object tracking and durable event continuity across frames.
• A policy-management interface for customer-specific rules and thresholds.
• MLOps integration for model registry, evaluation, canary rollout, and rollback.
• Human-in-the-loop review workflows that feed labeled outcomes back into model and rule tuning.
• Camera onboarding and tenant/device/user policy management.
• Public GitHub repository with full source code, deployment assets, and sample configurations.
• Event retention policy templates for 7/30/90-day storage tiers and archive movement.

8. Conclusion

The value of this GPU-based AI camera analytics solution extends beyond simply detecting objects within a frame. Its true value lies in transforming visual signals into reliable, explainable, and actionable operational decisions that can be integrated into broader enterprise workflows and real-time response systems. By combining containerized stream processing, OCI Queue, GPU inference instance pools, contextual classification, and configurable policy-driven rules, the architecture provides a scalable and adaptable foundation for production-grade visual AI deployments.

Equally important, the architecture is designed to support operational transparency and long-term maintainability. It follows a microservice architecture and modular building-block framework where each feature block represents a decoupled processing layer. This, coupled with externalized policy evaluation, helps organizations adapt the reference architecture to different business cases by using the solution as a whole or by adopting individual modules. This helps move beyond isolated AI experiments toward resilient, enterprise-ready decision intelligence platforms.

Ultimately, the goal of the architecture is not only to process visual data at scale, but to help operators, analysts, and enterprise systems focus attention on the events, behaviors, and operational conditions that matter most within a specific business context.

9. References and Further Reading

• OCI Queue documentation
• OCI Streaming overview
• OCI Compute autoscaling for instance pools
• OCI Vision documentation
• OCI Vision Stream Video Analysis
• OCI Connector Hub overview
• OCI Compute shapes

10. Contributor Emails

• Shamish Maikoti (maikoti.chandra@oracle.com)
• Akshita Muthayala (akshita.muthayala@oracle.com)
• Viraj Poolabhavi (viraj.p.poolabhavi@oracle.com)
• Chaitanya Chennam (chaitanya.chennam@oracle.com)
• Prashant Gaikwad (prashant.gaikwad@oracle.com)
• Prodipto Ranjan Baksi (prodipto.ranjan.baksi@oracle.com)