Data Extraction for Sub-sets of Accounts.
By StuRamage on Feb 21, 2012
Migrating sets of data to support testing has been an issue on a number of projects that I have been involved in, with most sites requiring a method of transferring subsets of accounts from Production to other CC&B environments.
The ConfigLab framework supports the ability to perform Account Sync actions, but these have a couple of limitations:
- the transfer is expected to be performed via ConfigLab database links, so both CC&B instances must be visible to each other via Database links.
- whilst the process is quick for small amounts of accounts (< 100), performance degrades very quickly if large amounts of accounts are scheduled to be processed.
- The changes required to ensure that the resulting data does not contain any customer-specific identifiers can be onerous (i.e. difficulty in producing anonymous data).
As a result I have been working on a set of extract tools which will generate SQL insert statements, which can then be zipped and transferred to a separate CC&B instance, to be used when populating the target database (i.e. offsite development, system testing, etc).
This should enable mass transfers, with the only limitation being the resulting size of the resulting SQL script, and your ability to transfer it to the target database.
Drop me a quick comment if you would like more details..