Broadleaf Microservices
  • v1.0.0-latest-prod

Externally-Executed Checkout Payment Transactions

Payment transactions triggered outside the Broadleaf ecosystem present unique challenges in becoming aware of the transaction results, maintaining the data state of relevant entities within Broadleaf, & effectively reacting to the result of the transaction. These scenarios are mostly commonly encountered with 3DS verification (the second pattern described here) & HPP (Hosted Payment Page) interactions where the submission of the gateway-hosted page results in the execution of the payment transaction.

Typical Gateway Patterns

Typical 3DS Pattern

While payment gateways continue to evolve their 3DS verification implementations to reduce friction & generally improve checkout UX capabilities, leveraging a browser redirect to a gateway-hosted page to complete 3Ds verification remains a common pattern.

This flow begins with attempting a payment transaction as part of checkout processing, as you would without 3DS being involved. Based on the information provided with this transaction, the card issuer will declare if the transaction is allowed to be processed (i.e. a "frictionless" flow) or if the transaction must be challenged. In the frictionless flow, the extra verification steps most commonly associated with 3DS do not need to be engaged. If the transaction is challenged, then typically the payment gateway will include a url in its response which the customer must be taken to, to verify their ownership of the card.

Once the customer verification step is successfully completed, payment gateways often automatically execute the transaction, & redirect the customer’s browser back to the storefront. The implication of this is that an Authorize or AuthorizeAndCapture transaction was attempted outside of PaymentTransactionServices, and therefore, Broadleaf services don’t have awareness of the transaction results. From there, we must obtain transaction results & react accordingly.

Typical HPP Pattern

HPP patterns typically start with an API call to the payment gateway where they provide you with url to the HPP in exchange for details about the order, esp. how much you intend to charge the customer. From there, the expectation is that the customer’s browser is redirected to that HPP url, where they’ll provide their payment method details. Upon submission of this gateway-hosted page, the payment gateway executes the requested Authorize or AuthorizeAndCapture transaction, & redirects the customer’s browser back to the storefront. Once again, we must obtain the transaction results & react accordingly.

Available Tools for Understanding Transaction Results

Browser Redirect/Callback

  • Both 3DS & HPP interactions involve redirecting back to the eCommerce site upon completion of the interaction. In 3DS scenarios, this involves declaring the redirect urls as part of the initial Authorize or AuthorizeAndCapture transaction attempt. In HPP scenarios, the redirect urls are typically declared as part of the setup API interaction. Typically, you’ll find that gateways expect a success url vs an error/failure url, but some also ask to be given a cancellation url - i.e. communicating the customer’s choice to opt-out of continuing the interaction

  • This gives a great indication of when the transaction was executed, but it only provides a low to moderate level of trust.

    • The interaction driven through the browser, so there’s not a 100% guarantee that it will successfully complete. There is a window of opportunity where customers can break this interaction by taking action on the browser (e.g. closing the window, hitting the back button, etc.). If the customer takes one of these actions against their browser, then you simply will not receive the callback. Depending on the circumstances around this interaction, the window of opportunity can be tiny or large, but no matter what, this window of opportunity exists.

    • Security of the request varies per gateway. In most cases, gateways do not provide a security mechanism to ensure the validity of callback requests.

  • Conclusion:

    • Callback redirects are great for driving the UX & providing a good indication that the transaction has been executed.

    • On the other hand, it should not be fully trusted.

      • The results (success vs failure) of these interactions should only be treated as a hint, triggering you to verify the result by looking up the transaction.

      • Due to the customer’s ability to interfere with the browser interaction, this cannot be the only mechanism used to gain awareness of the transaction result & finalize the checkout.

Webhooks

  • Payment gateways generally use webhook notifications to asynchronously notify you of completed (successful or failed) transactions. In this context where checkout transactions are being executed outside of our ecosystem, they’re an especially helpful tool.

  • Webhooks give a great indication of when the transaction has occurred.

    • Keep in mind that sometimes notifications can be delayed to some degree. In our experience, this delay usually at most a matter of seconds.

  • Webhook interactions can typically be given a high level of trust.

    • The interaction driven by a server-to-server call between gateway & your backend server.

    • Security of the request is generally high. Most gateways include a mechanism to verify the validity of the transaction results being provided.

    • Webhooks include retry mechanism to help ensure that transaction results are provided. Keep in mind that this doesn’t necessarily guarantee delivery in the case of a server outage.

  • Conclusion:

    • Webhook notifications are another great way of achieving near-real-time awareness of transaction results, & should be used to help finalize checkouts, esp. when the callback interaction fails.

    • Due to the asynchronous nature of webhook notifications, they really can’t be used to drive any aspect of the primary checkout UX (i.e. the customer’s primary experience via the checkout flow in the browser).

Transaction Lookup API

  • Gateways almost always provide APIs to lookup payments/transactions. These APIs can also be helpful for becoming aware of transaction results, but there is a disconnect between the ability to lookup a transaction vs when to start looking for the result. Especially in the context of 3DS & HPP interactions, the user could never complete the interaction, & therefore, no transaction is ever executed.

  • Transaction lookups can be given a high level of trust.

    • The interaction driven by a server-to-server call between your backend server & the gateway.

    • Security of these lookup requests are generally high, aligning with the gateway’s API security.

  • Conclusion:

    • Transaction lookups can be a great way of gaining awareness of the transaction result, but they’re naturally rather inefficient given your inability to predict when they should be engaged.

Guiding Principles & How they Apply to Broadleaf

Given the payment gateway concepts & tools mentioned above, & the Broadleaf checkout plumbing described here, there are many ways to potentially support these 3DS & HPP interactions. The key is to assess & understand the series implications that each design choice has on the overall ecosystem. Before getting into the improved pattern vs legacy pattern, lets consider some guiding principles:

  • Don’t execute payment transactions until you’ve verified that the cart & its related entities are in a valid state & are ready for checkout.

    • In the Broadleaf ecosystem, this means that the cart should pass the checkout workflow validation activities prior to engaging a 3DS verification or HPP interaction.

  • Don’t let the customer mess something up via the browser. Instead, ensure that cart finalization happens after a successful transaction, even if things break down with the browser.

    • One of the primary challenges/issues of the legacy pattern is that it places too much responsibility on the callback redirect & frontend to complete the interaction. In short, if the callback fails, then the customer could be left in a state where the checkout is not fully completed. Again this goes back to the window of opportunity where a customer can interfere with the callback. Depending on multiple factors, the window could be tiny & there’s very little risk, or the window is large, introducing more risk of the interaction breaking down.

  • Support the ability to create a UX that is clear & easy to navigate.

    • Our goal is to provide mechanisms/patterns that strike the right balance between UX flexibility with reduced friction in the checkout flow vs having enough structure to produce robust interactions.

Improved Pattern (Available with Broadleaf 2.1.1-GA & Beyond)

Iterating on our legacy pattern, our improved design was able to achieve the following goals:

  • Eliminate the customer’s ability to interfere via browser interaction

  • Reduce reliance on the callback redirect

  • Reduce reliance on the frontend to coordinate the recording of transaction results & the reaction to those results

  • After obtaining successful transaction results, simply finalize the cart (i.e. setting the cart status to SUBMITTED, adding an order number, & recording the submission date) instead of going through the full checkout workflow

    • Implication: If the cart is in a trusted state, then extra potential points of failure can be avoided.

  • Introduce the ability to finalize the cart after obtaining successful transaction results via the webhook

Design

Improved 3DS & HPP Pattern

The following steps are relevant for both 3DS verification & HPP interaction

  1. In the payment section of the checkout flow, create one or more payments via PaymentTransactionService APIs.

    • For 3DS, this involves gathering payment method details (e.g. producing a payment token) in preparation for a potential frictionless transaction.

    • For HPP, this typically doesn’t involve gathering payment method details since they’ll be gathered via the HPP. Instead, the customer should declare their intention to use the payment method, & the PaymentTransactionServices payment should contain the necessary data points to execute the HPP setup interaction.

    • In both cases, it’s important to provide redirect urls that point to the CartOps callback endpoint (ExternalPaymentTransactionCallbackEndpoint).

  2. Submit checkout

    • Within the checkout workflow, the cart & its related entities are validated & checkout payment transactions are attempted via PaymentTransactionServices calling the payment gateway’s APIs.

      • For 3DS, this attempts the frictionless transaction. If the transaction is not challenged, then transaction results are provided. If the transaction is challenged, then a redirect url is provided by the gateway.

      • For HPP, this gathers the HPP url by calling the gateway’s setup API.

    • For either 3DS or HPP, if a redirect url is provided, we update the cart status to AWAITING_PAYMENT_FINALIZATION, add a customer mutability block to the cart’s payments, & provide the redirect url in the process checkout API response

      • Changing the cart status to AWAITING_PAYMENT_FINALIZATION & adding a customer mutability block to the cart’s payments is done to block any customer-driven modifications to the cart or its payments. Since the cart just passed the checkout workflow validation, we can confidently say that the last step before we can finalize the cart is to complete the payment interactions. Since the cart and its payments cannot be modified, we can maintain that confidence throughout the interaction, and therefore only need to finalize the cart when we receive transaction results, instead of having to go back through the entire checkout workflow. More on how to opt out of this state below.

  3. In response to a 3DS challenge or a HPP setup, the customer’s browser should be redirected to the provided url.

  4. Once the 3DS or HPP interaction has been completed by the customer…​

    1. The gateway redirects the browser to the specified url, which is handled by the CartOps callback endpoint

      • This endpoint validates the request, looks up transaction results via an API call to the gateway, records the transaction results, & then evaluates the cart’s payments to determine how to proceed.

        • If the transaction was successful & cart’s payments have successful transactions matching the cart total, then…​

          • A message is sent to finalize the cart

          • A 302 response is returned from the endpoint causing the browser to redirect back to the storefront where the order confirmation page should be rendered.

        • If the transaction was successful & cart’s payments do NOT have successful transactions matching the cart total, then…​

          • Another cart payment requires 3DS verification or a HPP interaction, at which point the callback endpoint’s 302 response indicates the need to continue to the next external payment interaction.

        • If the transaction result reflects a failure or cancellation, then…​

          • The failed payment is archived

          • A 302 response is returned from the endpoint causing the browser to redirect back to the storefront where the customer should be prompted to provide a different payment method. More on this below!

    2. The gateway pushes transaction results to the configured webhook endpoint in PaymentTransactionServices

      1. This endpoint records the results & publishes a message notifying the general ecosystem that transaction results are now known.

      2. A CartOps message listener listens for this message, determines if the sum of the successful checkout transactions matches the cart total, & if so, sends a message to finalize the cart.

  5. A CartOps message listener is used to finalize carts based on messages from both the callback & webhook paths

    • As part of cart finalization, the CheckoutCompletionEvent is sent throughout the ecosystem notifying that a successful checkout has just been completed.

Note
  • Using the webhook to also trigger cart finalization ensures that if the customer does something unexpected with their browser (e.g. hits the back button or closes the window) at any point, then the cart finalization will continue. From there, the customer will be notified of the completed order via the order confirmation email (triggered by the CheckoutCompletionEvent).

  • The PaymentTransactionExecutionActivity now continues processing payments when it recognizes that a payment requires 3DS verification or HPP interaction (categorized as a type of "failed" transaction). Prior to this, the PaymentTransactionExecutionActivity would stop processing payments as soon as it recognized any type of failed transaction. The implication of this continued processing is that in a multi-payment scenario, you could have one transaction requiring 3DS verification & one successful transaction & the checkout attempt will leave you with just the need to finalize the 3DS interaction. Prior to this, the checkout attempt would only result in requiring 3DS verification for the first payment, but not processing the second payment at all, thus requiring a second checkout submission.

    • Keep in mind that in the case of a legitimate transaction failure (e.g. insufficient funds or card stolen), then PaymentTransactionExecutionActivity will stop processing payments. This decision is made with the thought that the customer will need to declare a different form of payment, & in doing so may completely change the set of payment methods being used. Therefore, lets not execute extra transactions that will need to be reversed if they do completely change the payment methods being used.

Handling Callback Redirects

Generally speaking, the callback redirect interaction is responsible for driving the customer’s experience via the browser. With that being said, we don’t want the storefront’s frontend app to be responsible for orchestrating the interaction. With that in mind, the 3DS/HPP redirect urls should be configured to redirect to the callback endpoint in CartOps to determine the state of the cart & its payments, then advise the frontend app how to proceed. This interaction can be a bit confusing, so it’s key to remember that two 302 redirects are involved. First the gateway sends a 302 redirecting the browser to the CartOps callback endpoint. Then the CartOps callback endpoint returns a 302 causing the browser to redirect to the storefront frontend app.

The CartOps redirect url (going to the storefront app) is built to reflect the result of the interaction & advise the frontend how to proceed. This is done by:

  • Configuring separate URIs to indicate that payment is finalized vs the customer should be prompted to provide a different payment method vs a subsequent 3DS/HPP interaction must be completed.

    • Note: A single, default URI can alternatively be used.

  • Regardless of the URI used, redirect attributes are included to indicate the status of the payment transaction (payment_result_status) & the overall payment finalization status (payment_finalization_status).

    • Note: The cart ID is also communicated via the cart_id redirect attribute.

The expectation is that the frontend app handles these redirects & advances the customer’s experience as needed.

Redirect Parameters

  • cart_id the cart ID

  • email_address - the customer’s email address. Present only if the customer is anonymous

  • payment_finalization_status - the state of the cart’s payments following the external payment interaction

    • FINALIZED - all of the cart’s payments have been finalized

    • REQUIRES_PAYMENT_MODIFICATION - one or more of the cart’s payments cannot contribute to a successful checkout, & the customer must be prompted to provide one or more alternative payment methods

    • REQUIRES_ADDL_EXTERNAL_INTERACTION - one or more of the cart’s payments requires action from the customer. This can happen if the cart has multiple payments that require 3DS verification or a HPP interaction. The first 3DS/HPP interaction may have been successfully completed, but the customer must complete the 3DS/HPP interaction for the next payment

    • UNKNOWN - the payment finalization status could not be determined. This is typically the case if an unexpected exception was encountered

  • payment_result_status - the result of the payment interaction that was completed, causing the redirect to CartOps

    • SUCCESS - the external payment interaction was successful. This represents both a successful interaction (3DS or HPP) & a successful transaction

    • PAYMENT_FAILED - the external payment interaction failed. This is meant to represent things like failed 3DS verification or a failed transaction following successful 3DS or HPP interaction

    • PAYMENT_CANCELED - the customer has decided to exit the external payment interaction, & therefore, the usage of the payment method is canceled

    • PAYMENT_EXPIRED - the customer did not complete the external payment interaction (e.g. 3DS verification or HPP) before it expired. Therefore, the payment is no longer usable

    • UNKNOWN - the result of the external payment interaction could not be determined. This is typically the

  • case if an unexpected exception was encountered

  • gateway_type - the gateway type (STRIPE, CHECKOUT_COM etc)

  • callback_error=INVALID_CALLBACK_REQUEST - indicates that the callback request failed. Usually, it can happen when the security validation failed. See Callback Endpoint Security

Managing the Cart Finalization Race Condition

Since both the callback endpoint & the webhook (indirectly) have the capability of triggering cart finalization, a race condition is created as these two competing threads attempt to resolve & modify the same cart. To resolve this race condition, we have both logical paths produce the same message to finalize the cart. The especially important aspect of this message is that both use the cart id as their idempotency keys (the MESSAGE_IDEMPOTENCY_KEY header on the message). Therefore, both messages can be present in the queue. The first one will be processed, & the second one will be ignored due to the message idempotency check in ExternalPaymentTransactionCartFinalizationListener. Both messages trigger the exact same work, so we don’t care which one gets processed first.

Rendering the Checkout Confirmation Page

While using a message listener to finalize the cart is a very simple & clean way to resolve the race condition, it has possible implications on rendering the order confirmation page for the customer. By asynchronously finalizing the cart, while simultaneously communicating successful payment finalization when redirecting to the storefront app, a different & far less consequential race condition is produced. Depending on the speed at which cart finalization is processed, you could get to rendering the order confirmation page with a cart that has not yet been finalized. The primary implication here is that you don’t yet have the order number since it’s produced as part of the finalization process. If/when this encountered, our recommendation would be one of the following:

  1. Tweak the confirmation page messaging slightly.

    • We have high confidence that the cart will be finalized shortly & the customer will receive their confirmation email.

  2. Possibly poll the cart history endpoint a couple of times to get the finalized cart.

    • Note: this isn’t necessarily guaranteed to get the finalized cart if the finalization queue is backed up.

This design choice reflects a trade-off where we prefer a clean & simple solution to transitioning completed payment transactions into finalized carts, ahead of a technically complex (and likely more brittle) cart finalization interaction. The alternative trade-off creates more risk of payments & carts getting out of sync for the sake of possibly having slightly better communication to the customer on the confirmation page.

Given that the order number is not yet known, we recommend using the following endpoints for gathering the cart when rendering the order confirmation page:

Note
If the cart is anonymous, then the CartOps callback endpoint will include the email_address redirect attribute to support this interaction.

Payment TTL Concerns & Transaction Reversal Candidates

Since 3DS verification & HPP interactions are completed asynchronously by the customer, & we don’t know if/when the interaction will be completed, we need to have a clear understanding of the influence of any time-based cart or payment interactions. The following diagram depicts the relevant out-of-box components and their relative windows of operation:

[--- Catalog Pricing TTL - 1hr -] (doesn't interfere)
        [--- Cart Guest Token - 1hr ----]
            [--- Guest Payment TTL - 1hr ---]
            [--- Pmt Callback Token TTL - 2hrs ------------------------------]
              [-- Pmt Finalization TTL - 1hr -]
                [--- Reversal job - 2hrs ----------------------------------------]

Cart Catalog Pricing TTL

  • Purpose: Reprices the cart using updated catalog prices if the pricing is old

  • Default time to live: 1 hour

  • When the timer starts: first add-to-cart

Cart Guest Token

  • Purpose: Protect anonymous customer data by clearing PII on cart & its related entities, & by archiving the cart’s related payments.

  • Default time to live: 1 hour

    • broadleaf.cartoperation.service.checkout.guest-token.token-timeout

  • When the timer starts: When a guest token is created (entering the checkout flow)

Guest Payment TTL

  • Purpose: Protects anonymous customer payment data by archiving payments once the TTL has expired. Note: This differs from the cart guest token TTL since the guest token verification can be disabled.

  • Default time to live: 1 hour

    • broadleaf.paymenttransaction.service.anonymous-payment-ttl

  • When the timer starts: When the anonymous payment is created

Payment Callback Token TTL

  • Purpose: Helps to guarantee that the callback request is only comes from a trusted source.

  • Default time to live: 2 hours

    • broadleaf.payment.callback-token.ttl

  • When the timer starts: When the payment is created

Payment Finalization TTL

  • Purpose: Sets the timeframe in which the customer can work to finalize their payments. This is relevant for both anonymous & registered customer carts.

  • Default time to live: 1 hour

    • broadleaf.cartoperation.service.checkout.external-payment-transaction.cart-payment-finalization.cart-lock-ttl

  • When the timer starts: When the cart enters the AWAITING_PAYMENT_FINALIZATION status

Reversal Candidate Job

  • Purpose: Looks for successful transactions marked as reversal candidates (PaymentTransaction#managementState = "REVERSAL_CANDIDATE") that are older than the TTL, & executes the relevant reversing payment transaction. For example, reverse-authorizing an Authorize transaction or refunding an AuthorizeAndCapture transaction.

    • This process is used to clean up any lingering transactions that do not correlate to a completed checkout. This is most relevant for multi-payment scenarios, but transactions that are recorded via webhook or transaction lookup are recorded as reversal candidates in the off chance that the cart finalization does not happen (not expected, but just to give some backup protection).

    • Note: When the cart is finalized, we also clear the managementState. At that point, the transaction is no longer a reversal candidate.

    • Note: After reversing the transaction, the payment is also archived.

  • Default time to live: 2 hours

    • broadleaf.paymenttransaction.reversal.reversal-candidate-ttl

  • When the timer starts: When the transaction results are recorded

Conclusions & Important Notes

  • The cart catalog pricing TTL is not a concern. This repricing logic is skipped if the cart status is AWAITING_PAYMENT_FINALIZATION.

  • In the case of an anonymous cart, the guest token TTL primarily defines the window in which the customer can complete their checkout interaction, including any 3DS or HPP interactions.

    • Note: Guest token validation can be disabled, at which point the payment TTL takes over.

  • In the case of a registered customer cart, the payment finalization TTL defines the window in which the customer can complete any 3DS or HPP interactions.

  • All of these TTL configurations are configurable, but it’s important that their values are considered relative to each other.

    • It’s important that reversal candidates are given the longest TTL, so that transactions are not reversed while the customer is interacting with the cart. Out-of-box, added a significant buffer between the reversal candidate TTL & any other TTL to ensure that we’re very confident that other TTLs have expired before processing reversal candidates.

    • The Payment Callback Token TTL is intentionally longer to avoid scenarios where the customer takes a significant amount of time to submit checkout after defining the payment. Since the Payment Callback Token TTL starts when the payment is created, we don’t want it to time out before the Payment Finalization TTL for registered customers.

Handling Payment Transaction Failures

When a payment transaction fails during this process of completing 3DS verification & HPP interactions, the relevant payment will be archived, but we keep the cart in the AWAITING_PAYMENT_FINALIZATION status and any other cart payments will have their customer mutability block removed. This data state represents the fact that we’ve successfully validated the cart, but just need one or more different forms of payment matching the cart total to complete the checkout. Removing the customer mutability block allows the customer to remove an existing payment, if they wish to completely change the payment methods that are used, rather than replace the one failing payment.

In this failed payment, we recommend navigating the customer to a payment management view where they can re-allocate the cart total amongst their payments. Keep in mind that since the cart remains in the AWAITING_PAYMENT_FINALIZATION status, we recommend not rendering actions that act against the cart since they will fail with the cart in this status.

The following screenshot represents the kind of UI that we have in mind for this interaction:

external payment interaction failed
Note
This is meant to provide the user with a familiar interface by mimicking the checkout UI, while limiting the available actions. Of course, you’re welcome to build any UI that you’d like, but the intention of this state and UI is to the customer towards a completed checkout by finishing the last step: providing payment.

Once the user has updated their payment methods (i.e. added/updated/removed payments via PaymentTransactionService), you should once again call the process checkout endpoint (CheckoutCartEndpoint#processCheckout(…​)). Since the cart is in the AWAITING_PAYMENT_FINALIZATION status & the state of the cart is trusted, the checkout workflow is able to focus only on payment-related activities to verify the payments relative to the cart via CartPaymentMethodValidationActivity & process any relevant payment transactions via PaymentTransactionExecutionActivity. Keep in mind that the result of this interaction could lead to another round of 3DS verifications or HPP interactions.

Note
If after re-allocating the cart total amongst a new set of payments, the customer decides to keep a previously-defined payment method requiring 3DS/HPP, then the PaymentTransactionExecutionActivity will process the other payments & re-prompt the customer to complete the same 3DS/HPP interaction.
Note
We’ve mostly talked about entering this prompt for providing alternative payment methods after a 3DS/HPP interaction resulting in a failed transaction, but there’s actually no reason that you can’t also enter this state after any checkout submission that resulted in a failed payment transaction. In other words, if your initial checkout submission resulted in a failed payment transaction, you could enter the AWAITING_PAYMENT_FINALIZATION cart status (blocking cart modifications) & prompt the customer to provide a new form of payment. By default, this is disabled, but it can be enabled by setting broadleaf.cartoperation.service.checkout.external-payment-transaction.cart-payment-finalization.enter-awaiting-payment-finalization-on-payment-failure = true.

The AWAITING_PAYMENT_FINALIZATION Cart Status

The AWAITING_PAYMENT_FINALIZATION cart status provides significant benefits as the key mechanism to avoid customer modification of the cart during 3DS verification or HPP interactions. While in this status, all customer-driven cart modification requests are rejected as an invalid request. If a resolved cart is in this status, we recommend prompting the customer to either continue their 3DS/HPP interactions or break out of the status, allowing the cart to once again be modified.

In our out-of-box demo, this prompt involves recognizing the AWAITING_PAYMENT_FINALIZATION status, then redirecting to the following page where they’re met with a similar prompt:

cart awaiting payment finalization review
Note

This redirection & prompting to continue with 3DS/HPP interactions or break out of the cart status is meant to handle both of the following cases:

  • When the customer has entered the 3DS/HPP interaction & hit the back button in their browser to get back to your storefront.

  • When the customer navigates to any other page on your site.

The status "break out" interaction described above is meant to support the case that the customer legitimately decides that they want to abandon the payment finalization interactions, & instead, go modify the cart. While the intention is to drive the customer towards completing the payment & ultimately converting the order, we want to at least provide the option to go back to modifying the cart. In our demo site, we chose to surface this "break out" action in a single location via the "Cancel & Continue Shopping" action. In your UI/UX, you’re more than welcome to trigger calls to the release payment finalization lock endpoint (ManageCartEndpoint#releasePaymentFinalizationLock(…​) in any way that you wish!

When this "break out" action is triggered, the cart’s status is shifted back to IN_PROCESS (making the cart editable), & either the cart’s payments are archived or their customer mutability block is removed (making them once again editable/removable). The decision on whether to archive the payments or remove their mutability block is decided by the following property: broadleaf.cartoperation.service.checkout.external-payment-transaction.cart-payment-finalization.archive-payments-when-cart-is-unlocked (defaulted to true). Note: our decision to archive the payments by default is driven by the idea that if the customer wants to edit their cart, then there’s a high likelihood that the cart total will be different, leading to different decisions for allocating the cart total amongst one or more payment methods. Also keep in mind that depending on the nuances of the payment gateway being used, the monetary amount previously declared with the gateway may not be modifiable. Therefore, the customer needs to re-declare the payment method for the gateway to recognize the new cart total.

Callback Endpoint Security

To ensure that the callback request comes from the gateway & not some other malicious actor, we implemented a passcode token. This token is generated at the time of payment creation & is added as a request param on provided redirect urls. Additionally, we store the token in the Payment#paymentMethodProperties map (not shared outside of PaymentTransactionServices). The gateway will then return that token during the callback, & we can verify it against the encrypted one we’ve stored in the payment datastore. If the token does not match or if the time-to-live has been exceeded, then the callback is considered invalid.

Other properties to configure for payment callback passcode tokens include:

  • broadleaf.payment.callback-token.allowed-characters

    • The character set to use when generating payment callback tokens, defaults to alphanumerics.

  • broadleaf.payment.callback-token.length

    • The length of a generated payment callback token, defaults to 32.

  • broadleaf.payment.callback-token.ttl

    • The duration in which a payment callback token is valid, default to 2 hours.

Adopting the Improved Pattern

The following steps are required to adopt the updated 3DS/HPP pattern:

  1. Update your gateway integration callback urls to point to the CartOps callback endpoint (ExternalPaymentTransactionCallbackEndpoint)

    • Make sure that the cart’s application & tenant IDs are communicated via the request params (or request body) that are provided to the callback endpoint.

      • Note: These values should be declared using the applicationId & tenantId names.

  2. Configure the following properties so that the CartOps callback endpoint can effectively redirect back to your storefront:

    • broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.storefront-base-url

    • broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.default-redirect-uri

    • broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.payment-finalized-uri

    • broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.payment-modification-uri

    • broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.external-payment-interaction-uri

    • Note: Keep in mind that these discriminated properties, so they can be declared globally, at the tenant level, or at the application level.

      • Default: broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.storefront-base-url

      • Tenant: broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.{tenantID}.storefront-base-url

      • Application: broadleaf.cartoperation.service.checkout.external-payment-transaction.callback-redirection.{applicationID}.storefront-base-url

  3. Update your frontend app to recognize the data states communicated via the redirect url & its redirect attributes, & react accordingly.

    • If the cart payments are finalized, then show the order confirmation page.

    • If another 3DS/HPP interaction is required, then identify the relevant redirect url & redirect the browser.

    • If the transaction failed or was cancelled, then prompt the customer to provide a new form of payment.

  4. Update the order confirmation UI to gather the cart by id using this cart history endpoint (CartHistoryEndpoint#readCustomerCart(…​) & CartHistoryEndpoint#readAnonymousCustomerCart(…​))

  5. Update your frontend app to include a payment modification UI to be used when the cart is in the AWAITING_PAYMENT_FINALIZATION status

  6. Optional: Update your frontend UI to support the AWAITING_PAYMENT_FINALIZATION "break out" action

  7. Ensure that your webhook interactions effectively communicate the relevant application & tenant IDs. Note: Depending on the gateway, this may have to be done as request params on the webhook url.

Integration Example of the Improved Pattern

CartOperationServices Configuration

broadleaf:
  cartoperation:
    service:
      checkout:
        external-payment-transaction:
          callback-redirection:
            storefrontBaseUrl: https://heatclinic.localhost:8456
            default-redirect-uri: /checkout/payment-confirmation

Frontend Changes

After the payment result is checked the callback endpoint will redirect to the configured page - https://heatclinic.localhost:8456/checkout/payment-confirmation in this example.

Example of the payment-confirmation page:
export const PaymentConfirmation: FC = () => {
  const router = useRouter();
  const { query } = router;

  const paymentFinalizationStatus = get(query, 'payment_finalization_status');
  const callbackError = get(query, 'callback_error');

  useEffect(() => {
    if ('REQUIRES_PAYMENT_MODIFICATION' === paymentFinalizationStatus) {
      router.push({
        pathname: '/checkout/payment',
        query: {
          payment_finalization_status: paymentFinalizationStatus,
          payment_result_status: get(query, 'payment_result_status'),
          gateway_type: get(query, 'gateway_type'),
        },
      });
    }
  }, [paymentFinalizationStatus, router, query]);

  return (
    <main className="container mx-auto px-4 py-8 xl:px-0">
      <section className="flex justify-center">
        {'FINALIZED' === paymentFinalizationStatus && (
          <VerifyCartAndPaymentStatus />
        )}

        {('UNKNOWN' === paymentFinalizationStatus ||
          callbackError === 'INVALID_CALLBACK_REQUEST') && (
          <ErrorMessage errorMessage={'There was an error processing your request. Please check your info and try again.'} />
        )}
      </section>
    </main>
  );
};

const VerifyCartAndPaymentStatus: FC = () => {
  const [historicalCart, setHistoricalCart] = useState<Cart>();

  const [errorMessage, setErrorMessage] = useState<string>();

  const { startHistoricalCartPolling, active: pollCartActive } =
    usePollHistoricalCartById({
      setHistoricalCart,
    });

  const handleSuccessResult = useHandleSuccessResult();

  useEffect(() => {
    if (pollCartActive) {
      return startHistoricalCartPolling();
    }
  }, [pollCartActive, startHistoricalCartPolling]);

  useEffect(() => {
    let isCanceled = false;

    if (!isCanceled && !pollCartActive) {
      if (
        !historicalCart ||
        (historicalCart.status !== 'SUBMITTED' &&
          historicalCart.status !== 'AWAITING_PAYMENT_RESULT')
      ) {
        setErrorMessage('There was an error processing your request. Please check your info and try again.');
      } else {
        handleSuccessResult(historicalCart);
      }
    }

    return () => {
      isCanceled = true;
    };
  }, [pollCartActive, historicalCart, handleSuccessResult, formatMessage]);

  return (
    <>
      {!errorMessage && <PageLoading />}

      {errorMessage && <ErrorMessage errorMessage={errorMessage} />}
    </>
  );
};

const PageLoading: FC = () => {
  return (
    <>
      <header className="mb-4">
        <h3 className="mb-6 mt-4 text-center text-4xl font-bold">
          We are checking your payments, please wait...
        </h3>
      </header>
      <PageLoader loading={true} />
    </>
  );
};

type ErrorMessageProps = {
  errorMessage: string;
};

const ErrorMessage: FC<ErrorMessageProps> = ({ errorMessage }) => {
  return (
    <strong className="block my-4 text-red-600 text-lg font-normal">
      {errorMessage}
    </strong>
  );
};

const usePollHistoricalCartById = ({ setHistoricalCart }) => {
  const router = useRouter();

  const { query } = router;

  const cartId = get(query, 'cart_id');
  const emailAddress = get(query, 'email_address');
  const paymentFinalizationStatus = get(query, 'payment_finalization_status');

  const [active, setActive] = useState(true);

  const startHistoricalCartPolling = useEventCallback(() => {
    let isCancelled = false;
    let currentTimeout = null;
    let count = 0;
    let errorCount = 0;

    async function poll() {
      let cart;

      try {
        // Send the request to the "/api/cart-operations/cart-history?cartId={cartId}" endpoint if the "emailAddress" is blank
        // or to the "/api/cart-operations/cart-history?cartId={cartId}&emailAddress={emailAddress}" otherwise
        cart = await getHistoricalCartById(cartId, emailAddress);

        // update counts
        count++;
      } catch (error) {
        errorCount++;
      }

      const cartStatus = cart?.status;

      if (!isCancelled && active) {
        if (
          count < 3 &&
          errorCount < 2 &&
          (!cart || cartStatus === 'AWAITING_PAYMENT_FINALIZATION')
        ) {
          currentTimeout = setTimeout(poll, 1000);
        } else {
          setHistoricalCart(cart);
          setActive(false);
        }
      }
    }

    if (paymentFinalizationStatus === 'FINALIZED') {
      // execute the initial poll
      poll();
    } else {
      setActive(false);
    }

    return function cancel() {
      isCancelled = true;
      clearTimeout(currentTimeout);
    };
  }, [cartId, emailAddress, setActive, getHistoricalCartById]);

  return { startHistoricalCartPolling, active };
};

const useHandleSuccessResult = () => {
  const router = useRouter();

  const { setCart, setGuestToken } = useCartContext();

  const [, setCookie] = useCookies([CookieName.SUBMITTED_CART]);

  return useEventCallback(
    async (historicalCart: Cart) => {
      pushGtmPurchase(historicalCart);
      setCookie(
        CookieName.SUBMITTED_CART,
        JSON.stringify({
          emailAddress: historicalCart.emailAddress,
        }),
        {
          path: '/',
          maxAge: 15 * 60 * 1000, // 15 minutes
          sameSite: 'strict',
        }
      );

      if (historicalCart.status === 'AWAITING_PAYMENT_RESULT') {
        await router.push(
          `/checkout/cart-awaiting-payment-result?emailAddress=${historicalCart.emailAddress}`
        );
      } else {
        await router.push(
          `/checkout/confirmation?orderNumber=${historicalCart.orderNumber}`
        );
      }

      setGuestToken(null);
      setCart(null);
    },
    [setCookie, setCart, setGuestToken]
  );
};

There is a possibility that the customer closed the external page and returned to the storefront while the cart is still locked (has the status AWAITING_PAYMENT_FINALIZATION). In this case, show the message that the cart can’t be changed and the payments should be finalized. It is also possible to unlock the cart in this state in case the customer wants to make any changes by calling /api/cart-operations/cart/${cartId}/release-payment-finalization-lock.

Example handling customer’s return to storefront with cart awaiting payment finalization
export const CartAwaitingPaymentFinalizationMessage: FC = () => {
  const router = useRouter();
  const { query } = router;

  const { cart } = useCartContext();

  const paymentFinalizationStatus = get(query, 'payment_finalization_status');
  const requiresPayment = get(query, 'requires_payment');

  if (
    (cart?.status !== 'AWAITING_PAYMENT_FINALIZATION' ||
      requiresPayment !== 'true') &&
    paymentFinalizationStatus !== 'REQUIRES_PAYMENT_MODIFICATION'
  ) {
    return null;
  }

  let message;

  if (paymentFinalizationStatus === 'REQUIRES_PAYMENT_MODIFICATION') {
    message = 'There was an error during processing your payment. Please provide another payment method.';
  } else {
    message = 'Your payments require additional action before they can be processed & the order is submitted.';
  }

  return (
    <div className="flex content-center justify-between">
      <strong className="block mr-2 text-red-600 text-sm font-normal">
        {message}
        <Button
          href="/"
          className="ml-2 text-link hover:text-link-hover hover:underline text-sm transition"
          onClick={() => {
            const confirmed = window.confirm('You are about to cancel your checkout submission to continue shopping. Are you sure you want to continue?');
            if (confirmed) {
              // Execute a request to the "/api/cart-operations/cart/${cartId}/release-payment-finalization-lock" endpoint
              releasePaymentFinalizationLock().then(() => router.push('/'));
            }
          }}
        >
          Cancel & Continue Shopping
        </Button>
      </strong>
    </div>
  );
};

Legacy Pattern (Standard Prior to Broadleaf 2.1.0-GA)

Note
If you wish to continue using the legacy pattern, it can be enabled via the following property: broadleaf.payment.legacy-external-payment-pattern-enabled. Doing so will disable several of the key components used for the improved pattern, & will avoid entering the AWAITING_PAYMENT_FINALIZATION cart status.

This flow begins with attempting a payment transaction as part of checkout processing, as you would without 3DS being involved. Based on the information provided with this transaction, the card issuer will declare if the transaction is allowed to be processed (i.e. a "frictionless" flow) or if the transaction must be challenged. In the frictionless flow, the extra verification steps most commonly associated with 3DS do not need to be engaged. If the transaction is challenged, then typically the payment gateway will include a url in its response which the customer must be taken to, to verify their ownership of the card.

Note
Passing more information with these transactions allows the payment gateway & card issuers to make more informed fraud decisions.
Checkout 3DS Challenge

The same 3DS challenge url that’s returned from the payment gateway will be returned from CartOperationServices, along with a CheckoutResponse#failureType of PAYMENT_REQUIRES_3DS_VERIFICATION. In this case, the customer must be taken to the challenge url, either by redirect or within an iframe, to complete the extra verification step.

Once the customer verification step is successfully completed, card issuers and/or payment gateways often automatically execute the transaction. The implication of this is that an Authorize or AuthorizeAndCapture transaction was attempted outside of PaymentTransactionServices, and therefore, Broadleaf services don’t have awareness of the transaction attempt or results. Additionally, there may be a successful transaction that does not correlate to a checkout submission. To gain this awareness of the transaction & its result, we leverage the following approaches:

  1. Once verification is complete & the transaction is executed, call the Record 3DS Transaction Results API in PaymentTransactionServices to record the result immediately.

  2. Listen to the gateway’s webhook notifications & record results for these transactions.

  3. For subsequent checkout submissions, recognize the previous request for 3DS verification, & attempt to lookup transaction results, instead of attempting a new transaction for the payment. See DefaultPaymentTransactionExecutionService#lookup3DSTransactionResults(…​) in CartOperationService.

Note
Once successful 3DS verification results are known (and hopefully transaction results as well), we suggest re-submitting checkout automatically. If the verification failed, then there’s no reason to resubmit checkout. In this case, the payment should be archived in PaymentTransactionServices, & a new form of payment should be requested from the customer.

A subset of these approaches can be used, but we strongly suggest making use of all three, since they reinforce each other.

  • Approach 1 is the quickest & easiest way to become aware of transaction results.

    • If this approach is used, then the out-of-box Broadleaf checkout workflow will recognize that the payment is already processed, & it won’t attempt any additional transactions for this payment.

    • NOTE: With this approach, it’s very important to validate the authenticity of request to ensure that it can be trusted, & that we’re not recording fraudulent transaction results.

  • Approach 2 is the best means of guaranteeing that the Broadleaf system becomes aware of the transaction & its results.

    • Regardless of what happens with frontend interactions that could affect Approach 1 or Approach 3, we’ll always get webhook notifications.

    • If this approach is used, then the out-of-box Broadleaf checkout workflow will recognize that the payment is already processed, & it won’t attempt any additional transactions for this payment.

  • Approach 3 is the final back-stop to ensure that we understand transaction results for the 3DS interaction.

    • If successful transaction results are found, then the checkout process will continue beyond this payment.

    • If failed transaction results are found, then the checkout process will be blocked in the same way that would be done for any other failed payment transaction.

    • If transaction results are still not known, then the transaction & checkout responses will reflect the original request for 3DS verification.