Test frequent patches faster with smaller datasets

Organizations need to apply security fixes promptly, but testing can become a bottleneck. As software vulnerabilities are identified and addressed, teams face increasing pressure to test and deploy patches within shorter maintenance windows. Yet before patching a database, application server, or application, teams still need confidence that their application will work as expected.

That testing depends on realistic data, often drawn from a full copy of production. These full-sized copies can run to terabytes, adding storage and compute costs and taking time that short maintenance windows don’t allow. They also carry sensitive information into test environments, creating additional data protection considerations. For faster test data preparation, teams need a smaller dataset that supports the relevant testing scenarios, with the option to replace sensitive values with fictitious ones.

Figure 1. Faster fixes need faster test data.
Figure 1. Faster fixes need faster test data.

Now generally available, Data Subsetting in Oracle Data Safe helps teams build that smaller dataset. It removes unneeded rows from a non-production database while helping preserve the relationships among retained records based on the configured subsetting policy, reducing the data teams must store, refresh, and process across repeated patch cycles.

Build a test subset with Data Subsetting

Patch tests can exercise many parts of an application, but they rarely need all the data. Take a subscription billing application with millions of customers and years of history. Checking that invoicing, payment retries, and refunds still work after a patch doesn’t need a decade of closed accounts. And if the application is being tested for North America, it doesn’t need customers from other regions either. Yet a full copy brings everything along: at 2 TB per copy, giving five team members their own database means 10 TB to store and refresh every patch cycle. A subset keeps the data your tests need and removes the rest, shrinking the storage and compute required for testing.

Figure 2. In this illustrative example, Data Subsetting reduces a 2 TB production copy to a 100 GB test subset, and the same job can also mask sensitive values.
Figure 2. In this illustrative example, Data Subsetting reduces a 2 TB production copy to a 100 GB test subset, and the same job can also mask sensitive values.

To build that subset, define selection rules in a subsetting policy, choose how to retain related rows, and optionally apply a separate masking policy:

  1. Choose where selection starts: That’s the driving table, usually a core business table such as CUSTOMERS in a billing application. A rule on that table determines which rows to keep: a condition (customers billed in the last three months), a percentage (a 10% sample), or both. You can combine conditions, such as billing date and region. A policy can also use several driving tables, each with its own rule, and Data Safe combines the results.
  2. Keep related data intact. A customer’s records can span invoices, payments, and refunds. Data Safe follows defined table relationships when selecting data, helping keep related records together so the subset remains useful for testing. You can adjust the default retention settings as needed.
  3. Mask what remains. After subsetting removes unneeded data, an optional Data Safe masking policy can help reduce exposure of sensitive information by replacing selected values, such as names, email addresses, and payment details, with fictitious ones.

Define once, subset every patch

Here’s how subsetting works for an illustrative billing application. Its production database holds about 10 million customers, 100 million invoices, and 120 million payments, roughly 2 TB in all. After each patch, the team runs its full test suite, from invoicing to payment retries and refunds.

Most of that data isn’t needed for these tests. Years of closed accounts fill the invoice and payment tables, but the tests only need a sample of recently billed customers. For example, the team may decide to keep a 10% sample of customers billed in the last three months, along with their invoices, payments, and refunds. To build that subset, the team:

  1. Clones the latest production data into a test database.
  2. Creates a subsetting policy to keep a sample of recent customers. A rule on the CUSTOMERS table keeps a 10% sample of customers billed in the last three months. Data Safe helps retain their related invoices, payments, and refunds according to the configured policy, so the subset remains useful for testing. A policy can hold many more rules across tables and schemas.
  3. Runs the subsetting job, which removes the rows the policy doesn’t keep.
  4. Optionally masks the subset in the same job, using the associated masking policy after subsetting finishes.
Figure 3. The billing team’s illustrative subsetting policy and report: 95% smaller, with 218.5 million fewer rows.
Figure 3. The billing team’s illustrative subsetting policy and report: 95% smaller, with 218.5 million fewer rows.

With the subset ready, each team member gets their own 100 GB copy instead of a 2 TB copy. For the next patch cycle, the team can clone more recent production data and rerun the same subsetting and masking policies, refreshing the test environment without redefining the rules.

Helps slash the time and cost of test databases

Suppose five team members each need a database to test a patch. In this illustrative example, creating five full 2 TB copies takes four hours each, totaling 20 hours of preparation effort and 10 TB of storage for the five copies.

With Data Safe, the team first spends four hours cloning the database and one hour subsetting and masking the clone. Each team member then receives a 100 GB copy. Assuming 12 minutes per copy, preparing all five takes another hour.

Under these assumptions, aggregate preparation time falls from 20 hours to six, while storage for the five tester copies drops from 10 TB to 500 GB, a 95% reduction. The storage comparison excludes the initial staging clone used for subsetting and masking. Actual results depend on database size, configuration, infrastructure, and workload.

Figure 4: Same patch tests, a fraction of the time, storage, and compute.
Figure 4: Same patch tests, a fraction of the time, storage, and compute.

Explore Data Subsetting to help reduce the time and cost of patch testing

Start with your next patch test and compare the subset’s coverage, preparation time, and cost with your current approach. Data Safe is included at no additional cost for Oracle cloud databases in OCI and Multicloud environments, within applicable usage limits. Service charges are also waived for other eligible Oracle databases through February 28, 2027.

Learn more