The Visual Builder Cloud Service Blog

Shaping Data using Customized Fetch Action Chains

Duncan Mills

As soon as you start to use Visual Builder Cloud Service you'll encounter the Service Data Provider variable type that wraps up your endpoint access for the Get Many operations. You'll be using this type all the time to manage collection user interfaces such as tables and list views,  In this article I'll discuss one of the powerful advanced capabilities of Service Data Providers, that of fetch action chains.

Screen Shot of Service Data Provider Property InspectorWhen you create a new Service Data Provider, either directly as a new variable, or as a side effect of running something like an "Add Data" Quick Start on the List View, you should notice the large button labelled Customize Fetch Action Chain. I've highlighted it here with the red arrow.

The purpose of this feature is to allow you to subvert the normal operation of the Service Data Provider in order to cope with unusual response payloads and mapping requirements. As such, it's an exceptionally powerful feature which will allow you to bind collection based UIs in a totally consistent way, not matter what the shape of your data source might happen to be.

Before we look at an example, consider what the Service Data Provider is doing for you today.  Fundamentally it provides four services to help you bind a collection to data:

  1. Auto-execution of the bound REST endpoint call to fetch the data when needed
  2. Management of the filtering and sorting abstraction over the endpoint via transforms 
  3. Auto-mapping of the requested portions of the REST endpoint payload into your associated JavaScript types
  4. Providing the correct data provider interface to the JET collection components, enabling features such as load-more-on-scroll

When we override the Fetch Action Chain we're generally trying to override the third of these features whilst leaving all the other functionality intact

Data-Pivot Example

I'm going to illustrate the power of the Custom Fetch Action chains using a specific use case that I encountered recently.  The use case here is that I have a service that returns a collection of data that I want to be able to show in a table.  So far so good, but of course there's a catch. Rather than the endpoint being a GET call that returns a JSON array, the form that we use most frequently, it only returns one row!

In fact, the API in this case is to return a single object which happens to contain two attributes: id (number) and result (string).  The twist here is that the result string is actually a string-ified form of a JSON array. So not only is the data the wrong shape (single "row" rather than an array of "rows") but also it needs additional conversion to a valid Javascript object array from one long encoded string.

So here's what the actual source payload might look like:

  "result":"[{\"id\":1,\"data\":\"First row\"},{\"id\":2,\"data\":\"Second row\"},{\"id\":3,\"data\":\"Third row\"}]"

So as we see that result string logically contains three rows that I want to be able to show in a table like this:

Transformed results table

So let's look at the basic steps that you need to carry out to achieve such a transformation:

Step 1: Define a Type to Describe the Incoming Payload 

The first task is to define a type that describes the shape of the incoming payload e.g. defines the {id,result} object. I'll define a new type in the page variable section to do this: 
Create a type for the new inbound REST call payload

That type then gets configured with the relevant attributes that describe the shape of the response:
Inbound payload type defintion

Step 2: Define a Type to Describe the Transformed Payload 

Next, we need to describe the shape of the transformed payload that the Service Data Provider can interpret as the array of objects to map into rows. This object has a shape that mimics the response that an "ideal" well-formed service might have returned to you in the first place. Out of this structure the bit that varies between use cases is just the contents of the individual item object within the items[] array. In this example, the "JSON as a String" results payload defines each member of the array as {id, data}:

Step 3: Define a Transformation Function

In this use case, we need to write the code that will take the string payload in the "result" attribute and transform that into the new array structure.  This is a very simple implementation that assumes that the JSON encoded into the result string is well formed.  In a production use case you would of course validate the inputs and outputs a little more thoroughly. 

The function, called transmuteResultData in this case, is created in the page JS functions:

Notice how this takes the string input (jsonAsString) and uses JSON.parse to convert that to a real array.  The details of course will vary depending on the type of transformation that you need to carry out.  Finally we return the transmutedData array as part of an object that matches the shape of the transformedPayloadType that we defined in Step 2. 

Step 4: Create Your Service Data Provider Variable

Now that we're done with  the groundwork we can go ahead and create the Service Data Provider variable in the target page. 

Once created you need to press the Select Endpoint button on the property inspector which will launch a wizard for you to fill in to configure the variable.  On the first page, select the endpoint that is returning the data that needs transforming. On the second page of the wizard, select Use Existing Type and from the dropdown select the transformed payload type that you defined in Step  2. Finish the wizard there. 

Now that the endpoint is defined for the variable, the property inspector will show the Customize Fetch Action Chain button. Before pressing that, ensure that you set the Item id attribute field in the Property Inspector to the correct value (e.g. id in this example).

Step 5: Assign a Custom Fetch Chain

Next, press the Customize button and choose a suitable Action Chain Id, the default is generally going to be OK. 

Once you have hit the Customize button on the popup, you will be taken into the action chain editor to enable the editing of this custom fetch chain. It will look a little like this with a start, a call rest endpoint action and a return action.

The Call Rest Endpoint action will be pre-wired to the endpoint that you selected when creating the Service Data Provider.  Your first step is to correct the type that this uses.  By default, the Call Rest endpoint Response Type will be mapped to the response type of the Service Data Provider as you defined it.  However, recall that we mapped the type that represented the shape of the data that we want to ultimately obtain when creating the variable, not the shape of the data that is actually being returned from the REST call. So select the Call Rest Endpoint in the action chain, then in the property inspector, correct the Response Type to point to the type that you defined in Step 1 as the true shape if the REST response.

Step 6: Call the Transmute Function

Next, we need to do the actual reshaping of the data using the transformation function that we defined in the page module in Step 3. 

To do this we just drag an drop a CallModuleFunction into the action chain after the REST endpoint call and wire it to the selected module function. 

You'll notice in the screenshot here that I have also set the return type for the function to the required shape of the transformed data. 

Finally of course we need to pass the source data (in this case the result string containing the encoded JSON) into the function.  We do this by selecting the Assign link on the Input Parameters and drawing the relevant line or entering the correct expression to map the source of the data to the parameter.  How this actually ends up looking will depend on where your source data is coming from of course, but it might look a little like this:

Step 7: Return the Transformed Result

All the hard work is now done and all we have to do to complete the custom fetch chain is to pass the transformed data array out as the payload of the Return action. By default this would have been set to the result of the REST endpoint call and we just need to tweak that to change it to the results of the Module Function call.

You can click the (x) link on the payload field in the property inspector to help you here and can select the whole callTransmuteFunction result to pass out as the "result" of the custom fetch action chain.  As the shape of the transmute function payload matches the transformed type that we defined in Step 2, which in turn, defines the operating shape of the Service Data Provider, everything matches up.

The Service Data Provider is now available for binding to your UI components!


Taking It Further

In this simple example I've shown how you can use Custom Fetch Chains for a Service Data Provider to pivot single row data to multi-row data.  However, the general principle can be extended to do all sorts of manipulation of your data, not just this particular use case.  For example you could actually join the results of multiple REST calls to create compound rows or similar transformations, or you could take the payload from a HTTP POST call and use that as the basis of a Service Data Provider.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.