Where is the Complexity?

April 5, 2024 | 10 minute read
Todd Little
Chief Architect, Transaction Processing Products
Text Size 100%:

One of the common arguments I hear about avoiding XA distributed transactions is due to their complexity.  In this series of blog posts I’ll examine that claim by looking at three versions of the same application.  The first version of the application ignores data consistency issues and operates as though failures essentially don’t occur.  This unfortunately is a pretty common practice due to the perceived complexity of introducing distributed transactions into an application.  The second version adopts the saga pattern that is all the rage.  It uses Eclipse MicroProfile Long Running Actions to implement the saga.  Unlike the toy examples used to illustrate how sagas work, this version will include the necessary logic to actually be able to complete or compensate a transaction.  Finally, the third version will use the XA pattern to ensure data consistency.

Basic Problem to Solve

The application I’ll use to illustrate the issues associated with ensuring data consistency is one that provides the transfer of money from one account to another account where each account is serviced by a different microservice.  Fundamentally a very simple application if you ignore failures.  The flow is basically a Transfer microservice:

  1. Accepts a request to transfer an amount of money from one account to another account
  2. Makes a request to withdraw the amount from the first account
  3. Makes a request to deposit the amount in the second account
  4. Returns success

 

non-transactional microservices

A very simple application, what could possibly go wrong?

Simplistic Application Without Considering Failures

Let’s look first at a possible simple Transfer microservice.  It offers a single service named “transfer”, which transfers money from an account in one microservice (department1) to an account in a different microservice (department2).  Here is a simple Spring Boot based teller service that handles transferring the money:

@RestController
@RequestMapping("/transfers")
@RequestScope
public class TransferResource {

    private static final Logger LOG = LoggerFactory.getLogger(TransferResource.class);

    @Autowired
    RestTemplate restTemplate;

    @Value("${departmentOneEndpoint}")
    String departmentOneEndpoint;

    @Value("${departmentTwoEndpoint}")
    String departmentTwoEndpoint;

    @RequestMapping(value = "transfer", method = RequestMethod.POST)
    public ResponseEntity<?> transfer(@RequestBody Transfer transferDetails) throws TransferFailedException {
        ResponseEntity<String> withdrawResponse = null;
        ResponseEntity<String> depositResponse = null;

        LOG.info("Transfer initiated: {}", transferDetails);
        try {
            withdrawResponse = withdraw(transferDetails.getFrom(), transferDetails.getAmount());
            if (!withdrawResponse.getStatusCode().is2xxSuccessful()) {
                LOG.error("Withdraw failed: {} Reason: {}", transferDetails, withdrawResponse.getBody());
                throw new TransferFailedException(String.format("Withdraw failed: %s Reason: %s", transferDetails, withdrawResponse.getBody()));
            }
        } catch (Exception e) {
            LOG.error("Transfer failed as withdraw failed with exception {}", e.getLocalizedMessage());
            throw new TransferFailedException(String.format("Withdraw failed: %s Reason: %s", transferDetails, Objects.nonNull(withdrawResponse) ? withdrawResponse.getBody() : withdrawResponse));
        }

        try {
            depositResponse = deposit(transferDetails.getTo(), transferDetails.getAmount());
            if (!depositResponse.getStatusCode().is2xxSuccessful()) {
                LOG.error("Deposit failed: {} Reason: {} ", transferDetails, depositResponse.getBody());
                LOG.error("Reverting withdrawn amount from account {}, as deposit failed.", transferDetails.getFrom());
                redepositWithdrawnAmount(transferDetails.getFrom(), transferDetails.getAmount());
                throw new TransferFailedException(String.format("Deposit failed: %s Reason: %s ", transferDetails, depositResponse.getBody()));
            }
        } catch (Exception e) {
            LOG.error("Transfer failed as deposit failed with exception {}", e.getLocalizedMessage());
            LOG.error("Reverting withdrawn amount from account {}, as deposit failed.", transferDetails.getFrom());
            redepositWithdrawnAmount(transferDetails.getFrom(), transferDetails.getAmount());
            throw new TransferFailedException(String.format("Deposit failed: %s Reason: %s ", transferDetails, Objects.nonNull(depositResponse) ? depositResponse.getBody() : depositResponse));
        }
        LOG.info("Transfer successful: {}", transferDetails);
        return ResponseEntity
                .ok(new TransferResponse("Transfer completed successfully"));
    }
    
    /**
     * Send an HTTP request to the service to withdraw amount from the provided account identity
     *
     * @param accountId The account Identity
     * @param amount    The amount to be withdrawn
     */
    private void redepositWithdrawnAmount(String accountId, double amount) {
        URI departmentUri = UriComponentsBuilder.fromUri(URI.create(departmentOneEndpoint))
                .path("/accounts")
                .path("/" + accountId)
                .path("/deposit")
                .queryParam("amount", amount)
                .build()
                .toUri();

        ResponseEntity<String> responseEntity = restTemplate.postForEntity(departmentUri, null, String.class);
        LOG.info("Re-Deposit Response: \n" + responseEntity.getBody());
    }

    /**
     * Send an HTTP request to the service to withdraw amount from the provided account identity
     *
     * @param accountId The account Identity
     * @param amount    The amount to be withdrawn
     * @return HTTP Response from the service
     */
    private ResponseEntity<String> withdraw(String accountId, double amount) {
        URI departmentUri = UriComponentsBuilder.fromUri(URI.create(departmentOneEndpoint))
                .path("/accounts")
                .path("/" + accountId)
                .path("/withdraw")
                .queryParam("amount", amount)
                .build()
                .toUri();

        ResponseEntity<String> responseEntity = restTemplate.postForEntity(departmentUri, null, String.class);
        LOG.info("Withdraw Response: \n" + responseEntity.getBody());
        return responseEntity;
    }

    /**
     * Send an HTTP request to the service to deposit amount into the provided account identity
     *
     * @param accountId The account Identity
     * @param amount    The amount to be deposited
     * @return HTTP Response from the service
     */
    private ResponseEntity<String> deposit(String accountId, double amount) {
        URI departmentUri = UriComponentsBuilder.fromUri(URI.create(departmentTwoEndpoint))
                .path("/accounts")
                .path("/" + accountId)
                .path("/deposit")
                .queryParam("amount", amount)
                .build()
                .toUri();

        ResponseEntity<String> responseEntity = restTemplate.postForEntity(departmentUri, null, String.class);
        LOG.info("Deposit Response: \n" + responseEntity.getBody());
        return responseEntity;
    }

}

Note that this simplistic implementation of the transfer service at least considers the possibility that the deposit service fails and if so attempts to redeposit the withdrawn amount back into the source account.  However as should be obvious, it’s possible that the redeposit fails thus leaving the funds in limbo.

Here is the code providing the REST service interface for withdraw for Department1:

    @RequestMapping(value = "/{accountId}/withdraw", method = RequestMethod.POST)
    public ResponseEntity<?> withdraw(@PathVariable("accountId") String accountId, @RequestParam("amount") double amount) {
        try {
            this.accountOperationService.withdraw(accountId, amount);
            return ResponseEntity.ok("Amount withdrawn from the account");
        } catch (NotFoundException e) {
            return ResponseEntity.status(HttpStatus.NOT_FOUND).body(e.getMessage());
        } catch (UnprocessableEntityException e) {
            LOG.error(e.getLocalizedMessage());
            return ResponseEntity.status(HttpStatus.UNPROCESSABLE_ENTITY).body(e.getMessage());
        } catch (Exception e) {
            LOG.error(e.getLocalizedMessage());
            return ResponseEntity.internalServerError().body(e.getLocalizedMessage());
        }
    }

The withdraw service calls the withdraw method on the accountOperationService,  Here is the code providing the withdraw method used by the above REST service.  It uses an injected EntityManager for JPA:

/**
 * Service that connects to the accounts database and provides methods to interact with the account
 */
@Component
@RequestScope
@Transactional
public class AccountOperationService implements IAccountOperationService {

    private static final Logger LOG = LoggerFactory.getLogger(AccountOperationService.class);

    @Autowired
    EntityManager entityManager;

    @Autowired
    IAccountQueryService accountQueryService;

    @Override
    public void withdraw(String accountId, double amount) throws UnprocessableEntityException, NotFoundException {
        Account account = accountQueryService.getAccountDetails(accountId);
        if (account.getAmount() < amount) {
            throw new UnprocessableEntityException("Insufficient balance in the account");
        }
        LOG.info("Current Balance: " + account.getAmount());
        account.setAmount(account.getAmount() - amount);
        account = entityManager.merge(account);
        entityManager.flush();
        LOG.info("New Balance: " + account.getAmount());
        LOG.info(amount + " withdrawn from account: " + accountId);
    }

    @Override
    public void deposit(String accountId, double amount) throws NotFoundException {
        Account account = accountQueryService.getAccountDetails(accountId);
        LOG.info("Current Balance: " + account.getAmount());
        account.setAmount(account.getAmount() + amount);
        account = entityManager.merge(account);
        entityManager.flush();
        LOG.info("New Balance: " + account.getAmount());
        LOG.info(amount + " deposited to account: " + accountId);
    }
}

The withdraw method gets the current account balance and checks for sufficient funds and throws an exception if insufficient funds.  Otherwise it updates the account balance and saves the account.  We can also see the deposit method which gets the current account balance, adds the amount to deposit and saves the updated account information.

What About Failures?

The developer of this teller service realizes there might be some failure scenarios to handle.  For example, what happens if the deposit request fails?  The developer solved that by having the teller service takes the corrective measure of redepositing the money back into the first account.  Problem solved.  But what happens if between the time of the withdrawal request and the request to redeposit the funds the teller microservice dies?  What happens to the funds that were withdrawn?  As it stands, they’re lost!

The developer could solve this problem by creating a table of pending operations that could be examined when the teller microservice starts up.  But that would also mean that the deposit service must be idempotent as the only thing the teller service can do is retry the deposit request until it succeeds at which point it would remove the entry from its pending operations table.  Until the deposit succeeds, the funds are basically in limbo and inaccessible to the account owner or anyone else.

So far, the developer has only handled some of the possible failures by adding error recovery logic into their microservice.  And this is only for a trivial microservice.  As more state information is updated, more complex recovery mechanisms may need to be added to the microservice. In the next post, we’ll look at how we can apply the saga pattern to solve this data consistency problem using Eclipse MicroProfile Long Running Actions, coordinated by MicroTx.  Check out this brief video introducing MicroTx.  https://www.youtube.com/watch?v=4j74C4GobzY

#MicroTx #oracle #transactions #dataconsistency #microservices #sagas #microprofile

Todd Little

Chief Architect, Transaction Processing Products

I'm currently the Chief Architect for a family of transaction processing products at Oracle including Oracle Tuxedo product family, Oracle Blockchain Platform, and the new Oracle Transaction Manager for Microservices (MicroTx).  My main areas of focus are on security, privacy, confidentiality, performance, and scalability.  My job is to provide the technical strategy for these products to ensure they meet customer requirements.

Prior to being acquired by Oracle, I was Chief Architect for BEA Tuxedo at BEA Systems, Inc. While at BEA Systems, I was responsible for defining the technical strategy and direction for the Tuxedo product family. I developed the Tuxedo Control for WebLogic Workshop that greatly simplified the usage of Tuxedo services from Workshop based applications. I also received two patents for methods allowing design patterns in a UML modeling tool to control the generation of software artifacts.

During my nearly 50 years of software architecture and development experience, I have worked on a wide range of software systems and technology and have 44 published patents. At Science Applications International I worked on microcoded plasma display systems and command, control, and communication systems for naval applications. As a senior software consultant at Digital Equipment Corporation, I was the New York Area Regional Tools Consultant and also helped develop a multi-language multi-threaded distributed object oriented runtime environment with concurrent garbage collection.


Previous Post

Proper SQL comes to MongoDB applications .. with the Oracle Database!

Hermann Baer | 11 min read

Next Post


ORA-04030 out of process memory when trying to allocate - 3 Step Resolution

Troy Anthony | 7 min read