Advanced Filtering with the Redwood Smart Search Component

The Redwood Smart Search component offers an advanced approach for filtering data. One way to look at it is as a smarter and smaller alternative to a form that lets users define queries on data. With the smart search you can offer filters that use UI components along with free search fields that together create compound query conditions.

The component offers built in filter types for numeric and date ranges, select single, select multiple, big numeric ranges and toggles. In addition, you can use keyword search to search for words across columns in your data set.

We are leveraging the component across Oracle Fusion Apps new Redwood modules. You can see an example of the component in action in the Redwood Reference App in the Orders list page:

In the component documentation – that you can find in the component exchange – you'll notice that the steps refer to two types of backends – Oracle ADF REST services and BOSS – both are used by Oracle's Fusion developers to create the backend for Redwood apps. The component however can be used with other backends too – in this blog we show an example of using the component with the Business Objects layer in Visual Builder. For configuration follow the doc steps that are meant for the ADF Business Components services – since the BOSS option uses internal features that are not yet available for customers. 

Setting up the component involves several steps that are all shown in the demo video below. We follow the video with instructions and code samples below.

Creating the Page

You can use various templates and layouts for the search page – including the smart search template. To keep things simple, we are using the Welcome page template for our page. This allows us to include a list component in the default slot, and add the smart-search component to the search slot. You'll find the smart search component in the component palette if you are using the Redwood starter application template, or if you added the redwood components to an existing application manually. For the list we just drag a business object onto the page, dropping it as a list and picking the template for a row and the columns we want to show. This will create an SDP variable bound to the service endpoint in our page's variables section.

Defining Variables and Types

The smart-search component has several attributes that we'll bind to variables in the page. In the demo we define the following variables: 

Important – for the Search Configuration attribute we need to refer to the .provider attribute of the variable:

The HTML code for the smartSearch would then be something like this:

An additional variable we need is a listDataProviderViewer that will be the base of the list component showing the results after filtering. This variable is based on an Oracle JET datatype – that you'll need to add to the page's json section for types – ojs/ojlistdataproviderview. In our example this variable is called EmpDPV. The code for the type definition needed in the json file is:

    "ojs/ojlistdataproviderview": {
      "constructorType": "vb/InstanceFactory"
    },

This variable also requires construction parameters – which points to an SDP that is the based for this data provider, the filter criterion variable, and a function that constructs the search query leveraging a list of columns we allow searching on when users enter text. to define the variable we'll again edit the json of the file directly (access it by right clicking the variable and choosing "go to code") – there you can use this code (replace the bold variables with the variables in your page):

    "EmpDPV": {
      "type": "ojs/ojlistdataproviderview",
      "constructorParams": [
        "[[ $page.variables.employeesListSDP ]]",
        {
          "filterCriterion": "[[ $page.variables.filterCriterion ]]",
          "dataMapping": {
            "mapFilterCriterion": "[[ $page.functions.getSearchMapFilterCriterion($page.constants.KEYWORD_FILTER_ATTRIBUTES) ]]"
          }
        }
      ]
    },

The code above is referencing a constant named KEYWORD_FILTER_ATTRIBUTES. This constant defines a list of attributes that we allow filtering on with free text. We need to define that constant in our page as an array of strings, and assign default value to it. You can do this from the declarative interface and provide a default value formatted as ["att1","att2","att3"] – this will result in the code in your JSON file that looks like this:

  "constants": {
    "KEYWORD_FILTER_ATTRIBUTES": {
      "type": "string[]",
      "defaultValue": [
        "name",
        "job"
      ]
    }
  }

In this example we allow free filtering on the name and job attributes of employees.

Once our EmpDPV is defined we can switch our listView component to be based on it – but remember that you want to use the .instance of the variable so it will look like this:

<oj-list-view data="[[ $variables.EmpDPV.instance ]]">

The list should still look the same once done.

Adding Filters Metadata

The smart search relies on meta-data file that defines the filters that will show up as options inside the search bar. You can define this json file outside of VB, and then import it into the application. You'll reference this file in the searchConfiguration variable. In the demo we are adding this file into the resources/data folder of the app. You can find an example of the file content for the various types of filters supported in the component exchange documentation page. The file we are using defines three filters:

• a number range for salary
• a select single for department
• a select multiple for countries

The source for the file we are using is:

{
  "description": "Emps BO smart search configuration",
  "filters": {
    "DepartmentsFilter": {
      "type": "oj-sp/smart-search/default-filters/SelectSingleFilter",
      "label": "Departments",
      "options": {
        "field": "department",
        "optionsData": "[[ $search.filters.DepartmentsFilter.options.optionsData ]]",
        "optionsKeys": "[[ $search.filters.DepartmentsFilter.options.optionsKeys ]]"
      },
      "data": {}
    },
    "SalaryFilter": {
      "type": "oj-sp/smart-search/default-filters/NumericRangeFilter",
      "label": "Salary",
      "options": {
        "field": "salary",
        "min": 1,
        "max": 10000
      },
      "data": {
        "start": 1,
        "end": 1000
      }
    },
    "LocationsFilter": {
      "type": "oj-sp/smart-search/default-filters/SelectMultipleFilter",
      "label": "Countries",
      "options": {
        "field": "country",
        "optionsData": "[[ $search.filters.LocationsFilter.options.optionsData ]]",
        "optionsKeys": "[[ $search.filters.LocationsFilter.options.optionsKeys ]]"
      },
      "data": {}
    }
  }
}

You'll want to adjust the file to your specific fields, ranges etc. Make sure the filed variables correctly match the names of the attributes in the type associated with your SDP.

Once the file is uploaded go to the source view of the defintion of your searchConfiguration variable and use the following code for the variable definition:

    "searchConfiguration": {
      "type": "oj-sp/smart-search/strategies/JsonSearchConfigProviderFactory",
      "defaultValue": {
        "url": "[[ $application.path + 'resources/data/mySearchConfig.json' ]]"
      }
    },

You'll want to make sure you have the right path to the file you uploaded. Use $application.path if this is a VB app or $extension.path if this is an AppUI extension.

Filter Construction Code

We need a piece of code that will construct the filter criterion by combining the various filters and values added in the smart search. You already referenced that function in the EmpDPV variable definition – $page.functions.getSearchMapFilterCriterion

We'll add this function and related functions to the js file associated with our page. Our code looks like this:

define([], () => {
  'use strict';

  class PageModule {
  }
  function _deepClone(fc) {
    return JSON.parse(JSON.stringify(fc));
  }

  // Recursively inspects the filterCriterion for TextFilter which are keyword enabled (aka matchBy:¿phrase), removing from the criterion
  function _findKeywordFilters(filterCriterion, keywords) {
    if (!filterCriterion) { return null }
    if (filterCriterion.text && filterCriterion.matchBy === 'phrase') {
      keywords.push(filterCriterion.text);

      return true;
    }
    if (Array.isArray(filterCriterion.criteria)) {
      const { criteria } = filterCriterion;
      for (let i = criteria.length - 1; i > -1; i--) {
        if (_findKeywordFilters(criteria[i], keywords)) {
          criteria.splice(i, 1);
        }
      }
    }

    return false;
  }

  // Converts the compound filter into its simplest form.
  function _normalizeCompoundFilter(filter) {
    if (filter.criteria.length === 0) {
      return null;
    } else if (filter.criteria.length === 1) {
      return filter.criteria[0];
    } else {
      return filter;
    }
  }

  // Splits the text into words
  function _splitIntoWords(text) {
    let tokens = text.split(/\W/);
    tokens = Array.isArray(tokens) ? tokens : [];
    // remove empty string tokens
    for (let i = tokens.length - 1; i > -1; i--) {
      if (!tokens[i]) {
        tokens.splice(i, 1);
      }
    }

    return tokens;
  }

  // Builds a contains with string match with the keywords 
  function _buildKeywordToFilter(keywordFilterAttributes, text) {
    const criteria = [];
    const compoundFilter = { op: '$or', criteria };
    keywordFilterAttributes.forEach(attributeName => {
      const words = _splitIntoWords(text);
      const attribute = `UPPER(${attributeName})`;
      words.forEach(val => {
        const value = val.toUpperCase();
        criteria.push({ op: '$co', attribute, value });
      });
    });

    return _normalizeCompoundFilter(compoundFilter);
  }

  function _convertAttributeFilterToAttributeFilterExpr(fc) {
    if (!fc.op || fc.attribute || typeof fc.value !== 'object') { return fc; }
    const path = [];
    let value = fc.value;
    let done = true;
    do {
      const keys = Object.keys(value);
      const key = keys[0];            // only support a single attribute per object
      value = value[key];
      path.push(key);
      done = (value === null || Array.isArray(value) || typeof value !== 'object');
    } while (!done);

    return { op: fc.op, attribute: path.join('.'), value };
  }

  function _convertToRAMP(fc) {
    // RAMP only supports AttributeFilterExpr which is a deprecated interface
    // Convert from the new syntax to what RAMP transforms understands
    if (!Array.isArray(fc.criteria)) {
      return _convertAttributeFilterToAttributeFilterExpr(fc);
    }
    if (Array.isArray(fc.criteria) && fc.op) {
      for (let i = 0; i < fc.criteria.length; i++) {
        let f = fc.criteria[i];
        if (Array.isArray(f.criteria)) {
          f = _convertToRAMP(f);
        } else {
          f = _convertAttributeFilterToAttributeFilterExpr(f);
        }
        fc.criteria[i] = f;
      }
    }

    return fc;
  }

  PageModule.prototype.getSearchMapFilterCriterion = function (keywordFilterAttributes) {

    if (!this._searchMapFilterCriterion) {
      //JS closure capturing the keyword fields.  Transforms keyword TextFilter into a RDBMS friendly search and AttributeFilter to AttributeExprFilter
      this._searchMapFilterCriterion = fc => {
        const filterCriterion = _deepClone(fc);
        const criteria = [];
        const compoundFilter = { op: '$and', criteria };

        const keywords = [];
        _findKeywordFilters(filterCriterion, keywords);
        if (Array.isArray(filterCriterion.criteria) && filterCriterion.$tag === '_root_') {
          // capture any remaining non-keyword filters
          filterCriterion.criteria.forEach(f => {
            criteria.push(f);
          });
        } else if (keywords.length === 0) {
          criteria.push(filterCriterion)
        }

        // transform into starts with filter criterion
        keywords.forEach(text => {
          const filter = _buildKeywordToFilter(keywordFilterAttributes, text);
          if (filter) { criteria.push(filter); }
        });

        return _convertToRAMP(_normalizeCompoundFilter(compoundFilter));
      }


      return this._searchMapFilterCriterion;
    }
  }
  return PageModule;
});

Note that you might need to adjust this code to create other filter formats if your backend is based on other type of REST services and not on business objects or Fusion Apps services.

Defining Variables for Filters

Our filters require variables in specific structures such as SDPs that populate singleSelect and multipleSelect components. An easy declarative way to create those variables is to drop such components on the page and bind them to data using the quick starts for adding options. We can then remove the UI components and keep the variables in place. In our example we created a select single for departments picking the department and id fields, and then defined a select multiple for countries picking the country and the code. 

The result of our declarative definitions are two SDP variables with this code in the json file:

    "countriesListSDP": {
      "type": "vb/ServiceDataProvider",
      "defaultValue": {
        "endpoint": "businessObjects/getall_Countries",
        "keyAttributes": "code",
        "itemsPath": "items",
        "responseType": "getallCountriesResponse",
        "transformsContext": {
          "vb-textFilterAttributes": [
            "name"
          ]
        }
      }
    },
    "departmentListSDP": {
      "type": "vb/ServiceDataProvider",
      "defaultValue": {
        "endpoint": "businessObjects/getall_Department",
        "keyAttributes": "id",
        "itemsPath": "items",
        "responseType": "getallDepartmentResponse",
        "transformsContext": {
          "vb-textFilterAttributes": [
            "department"
          ]
        }
      }
    }

Binding SearchContext

The search context variable maps the filters we defined in the json meta-data file, to the variables that hosts the data for the filters. Now that the two SDP variables are in place, we can define the searchContext variable's default value referencing the filters and the variables we created. Our default value is:

{
    "$search": {
        "filters": {
            "LocationsFilter": {
                "label": "Countries",
                "options": {
                    "optionsData": "[[ $page.variables.countriesListSDP ]]",
                    "optionsKeys": {
                        "label": "name",
                        "value": "code"
                    }
                }
            },
            "SalaryFilter": {
                "label": "Salary"
            },
            "DepartmentsFilter": {
                "label": "Departments",
                "options": {
                    "optionsData": "[[ $page.variables.departmentListSDP ]]",
                    "optionsKeys": {
                        "label": "department",
                        "value": "id"
                    }
                }
            }
        }
    }
}

Summary

When we run our page, the search bar shows options to filter by departments, countries and salary – showing a specific UI widget for each type of filter. In addition, we can type words that will then be used to search across both the name and job fields of an employee.  

Update October 2024 – release 24.10 of the Redwood components and templates is out and a key advantage for the smart search component is that it now uses filter chips to allow for easier selection of conditions. You can see how this looks in the image below. This video shows how to upgrade your VB app to use newer Redwood components and how to switch to use the dedicated template for smart search.