Welcome back!  We hope you are enjoying these introductory blogs about Oracle Cloud Infrastructure Data Integration. To get the list of previous blogs related to Oracle Cloud Infrastructure Data Integration, refer to – https://blogs.oracle.com/dataintegration/oracle-cloud-infrastructure-data-integration.

In Oracle Cloud Infrastructure Data Integration, a Data Flow provides a simple interface to let users design their logical flows by including a series of sources, target(s), transformations using joins, aggregations, expressions, etc. A Data Flow allows data engineers, ETL developers, architects, to develop graphical data transformations without writing any code. These Data Flows are executed using scaled-out Apache Spark clusters allocated to your workspace.

Below is the high-level overview of how Data Flow will work in Oracle Cloud Infrastructure Data Integration:

Data Flow Overview in OCI Data Integration > image2020-7-15_20-26-30.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-15_20-26-30.png?version=1&modificationDate=1594824996342&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image1-5.png” style=”width: 602px; height: 422px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-15_20-26-30.png”>

Let us deep-dive, on how to use a Data Flow and explore all the graphical components available in a Data Flow.

Creating the Data Flow

A Data Flow can be created from the Workspace Home page under Quick Actions tile or can be created from within Projects or Folders inside Projects.

Data Flow Overview in OCI Data Integration > image2020-7-16_14-58-57.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-16_14-58-57.png?version=1&modificationDate=1594891746211&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image2-4.png” style=”width: 670px; height: 295px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-16_14-58-57.png”>

Understanding different components within Data Flow

After clicking on “Create Data Flow”, a new tab opens up where we can see all the components related to a Data Flow. Below is the image that shows different components related to the Data Flow.

Data Flow Overview in OCI Data Integration > image2020-7-16_14-59-41.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-16_14-59-41.png?version=1&modificationDate=1594891791821&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image3-4.png” style=”width: 672px; height: 300px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-16_14-59-41.png”>

Operators Panel

Operators are dragged onto the canvas and used to design the logic to perform ETL processes. Operators available are – Source, Target, Filter, Join, Expression, and Aggregate. For connecting two operators, simply click the endpoint of the first operator and drag it over the second operator. In the properties tab of the operators, we can define the specific properties, e.g. in the Source operator, we can define the source table which is associated with the operator. Similarly in the join operator, we can define the join condition between two datasets.

Data Flow Overview in OCI Data Integration > image2020-7-16_15-0-30.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-16_15-0-30.png?version=1&modificationDate=1594891835952&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image4-4.png” style=”width: 443px; height: 400px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-16_15-0-30.png”>

In the above example, we can also “Assign Parameter”. We will be covering Parameters in future blogs.

Example of Join operator –

Data Flow Overview in OCI Data Integration > image2020-6-17_13-52-44.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-6-17_13-52-44.png?version=1&modificationDate=1592382165856&api=v2″ data-unresolved-comment-count=”0″ height=”400″ src=”/wp-content/uploads/sites/121/2025/11/image5-4.png” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-6-17_13-52-44.png” width=”689″>

Properties Panel

There are properties associated with every Operator as well as the Data Flow itself. The properties panel enables the user to select a corresponding data entity, add join conditions, look at the data in Data Xplorer, look at the attributes, validate an operator or select the Project Folder where the Data Flow has to be associated.

Example of DETAILS and VALIDATION tab in Property Panel –

The below image shows the Details tab of the Data Flow where we can change the name of the Data Flow or the Project/Folder associated with the Data Flow.

Data Flow Overview in OCI Data Integration > image2020-6-17_14-13-14.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-6-17_14-13-14.png?version=1&modificationDate=1592383396475&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image6-4.png” style=”width: 655px; height: 217px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-6-17_14-13-14.png”>

According to the Operator, the Details tab gets changed. For e.g. Source Operator “Details tab” is where you can pick a Data Entity from the Data Assets defined in the Workspace. In the Target Operator “Details tab” you can pick the Target Data Entity along with the Integration Strategy to be used.

Data Flow Overview in OCI Data Integration > image2020-7-16_15-5-50.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-16_15-5-50.png?version=1&modificationDate=1594892152646&api=v2″ data-unresolved-comment-count=”0″ height=”351″ src=”/wp-content/uploads/sites/121/2025/11/image7-4.png” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-16_15-5-50.png” width=”700″>

Example of ATTRIBUTES tab in Property Panel –

The functionality of the Attributes tab in the Property panel will be the same for Source, Target, Join, Filter, Aggregate, and Expression operator. Here, the Attributes tab provides the list of all the attributes coming from the previous operator or the source. The user can exclude the columns with “Exclude by Pattern” or “Exclude by Selection”, rename the columns, and edit the rules defined on columns.

Data Flow Overview in OCI Data Integration > image2020-7-15_20-54-34.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-15_20-54-34.png?version=1&modificationDate=1594826677687&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image8-4.png” style=”width: 654px; height: 264px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-15_20-54-34.png”>

Example of DATA tab in Property Panel –

The functionality of the Data tab in the Property panel will be the same for Source, Target, Join, Filter, Aggregate, and Expression operator. In the Data tab, we can preview the data using Data Xplorer and look at the profiling of the data for each attribute. Also, we can perform several transformations on the attributes like Change Case, Null Fill Up, etc. The below screenshot has applied the “Change Case” transformation on one of the attributes. To view Data Profiling, click on the respective attribute.

Data Flow Overview in OCI Data Integration > image2020-6-17_15-34-15.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-6-17_15-34-15.png?version=1&modificationDate=1592388258657&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image9-3.png” style=”width: 679px; height: 359px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-6-17_15-34-15.png”>

After clicking on “Apply”, a new expression operator is automatically added with the new column “UPPER_CASE”

Data Flow Overview in OCI Data Integration > image2020-6-17_15-52-2.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-6-17_15-52-2.png?version=1&modificationDate=1592389328041&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image10-3.png” style=”width: 700px; height: 350px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-6-17_15-52-2.png”>

Example of MAP tab in Property Panel –

The functionality of the Map tab in the Property panel is only available in the Target operator. This tab is used to map all the source/customized attributes coming from previous operators to the Target attributes.

Data Flow Overview in OCI Data Integration > image2020-7-15_20-58-48.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-7-15_20-58-48.png?version=1&modificationDate=1594826931596&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image11-3.png” style=”width: 654px; height: 350px;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-7-15_20-58-48.png”>

This summarizes all the different operations available in a Data Flow. Once the logical design of the Data Flow is completed click on “Save and Close” and the new Data Flow can be seen under Projects/Folders.

Finally, if you want the Data Flow to be deleted from the Project/Folder click on Delete in the Data Flows listing page.

Data Flow Overview in OCI Data Integration > image2020-6-17_16-52-57.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-6-17_16-52-57.png?version=1&modificationDate=1592392979387&api=v2″ data-unresolved-comment-count=”0″ height=”214″ src=”/wp-content/uploads/sites/121/2025/11/image12-3.png” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-6-17_16-52-57.png” width=”654″>

Type “DELETE” and click on Delete for permanently removing the Data Flow

Data Flow Overview in OCI Data Integration > image2020-6-18_13-50-44.png” data-mce-src=”https://confluence.oci.oraclecorp.com/download/attachments/258795856/image2020-6-18_13-50-44.png?version=1&modificationDate=1592468448972&api=v2″ data-unresolved-comment-count=”0″ src=”/wp-content/uploads/sites/121/2025/11/image13-1.png” style=”width: 480px; height: 196px; border-width: 1px; border-style: solid;” title=”Data Integration Service > Data Flow Overview in OCI Data Integration > image2020-6-18_13-50-44.png”>

To run the Data Flow, a new Integration Task needs to be created under Tasks. In the newly created task configure the Data Flow and publish to the Applications for execution. We will be covering Integration Tasks in a future blog.

That’s the end of today’s blog. Thanks for reading! For more on Oracle Cloud Infrastructure Data Integration, check out the Tutorials and Documentation.