Broadleaf Microservices
  • v1.0.0-latest-prod

Release Notes for 1.0.0-GA

Requirements

  • JDK 11 is now required for Broadleaf release trains 1.7.0-GA, and beyond.

New Features & Notable Changes

Feature/Notable Change Related Services Links

Introduction of PaymentTransactionServices to store payment data and handle payment gateway interactions

CartOperationServices, OrderOperationServices, PaymentTransactionServices

Cart & Order payment storage and management moved to PaymentTransactionServices

CartServices, CartOperationServices, PaymentTransactionServices

Replace Payment status concept with PaymentSummary

CartOperationServices, OrderOperationServices, PaymentTransactionServices

Move 3DS transaction recording endpoint to PaymentTransactionServices

CartOperationServices, PaymentTransactionServices

Introduced a payment time-to-live threshold for anonymous customers to protect their data

PaymentTransactionServices

Update guest token logic to archive the cart’s related payments via PaymentTransactionService when a new guest token is created

CartOperationServices, PaymentTransactionServices

Update CSR and admin preview functionality to restrict access to payments in PaymentTransactionServices while the cart is owned by a CSR or preview session

CartOperationServices, PaymentTransactionServices

Introduced a payment locking mechanism to avoid multiple processes modifying a payment simultaneously

CartOperationServices, OrderOperationServices, PaymentTransactionServices

Job identifying stale payment transactions for reversal has been moved to PaymentTransactionServices

CartServices, PaymentTransactionServices

Job responsible for reversing transactions marked with "REQUIRES_REVERSAL" have been moved to PaymentTransactionServices

CartServices, CartOperationServices, PaymentTransactionServices

Introduce ability to gather saved payment methods from CustomerServices when creating a payment, so that the sensitive payment method data doesn’t have to be passed via the browser

CustomerServices, CartOperationServices, PaymentTransactionServices

Introduce an endpoint so that payment methods used in checkout can be saved for future use

CustomerServices, CartOperationServices, PaymentTransactionServices

Added tenant narrowing logic for scheduled jobs to ensure that they execute within the context of a single tenant

MicroMessagingCommon, DataTracking, ScheduledJobServices, CartServices, InventoryServices, CartOperationServices, MicroBulkCommon, ImportServices, MicroExportCommon, AuthenticationServices

Upgrade Guide

API Changes

Additions/Updates Removals
  • POST /payments

  • GET /payments

  • POST /payments/lock

  • POST /payments/unlock

  • GET /payments/{id}

  • PATCH /payments/{id}

  • DELETE /payments/{id}

  • POST /payments/{id}/lock

  • POST /payments/{id}/unlock/{token}

  • POST /payments/{id}/authorize

  • POST /payments/{id}/reverse-authorize

  • POST /payments/{id}/authorize-and-capture

  • POST /payments/{id}/capture

  • POST /payments/{id}/refund

  • POST /payments/{id}/record-transaction-results

  • GET /sensitive-payments

  • POST /customer-payments

  • GET /customer-payments

  • PATCH /customer-payments/{id}

  • DELETE /customer-payments/{id}

Schema Changes & Data Migrations

Liquibase Change Sets

Create/Update SQL

Drop SQL

Payment and Payment Transaction Data Migrations

CartServices and OrderServices both contain payment and payment transaction data that must be migrated to PaymentTransactionServices. It is very important to note that the data migration for CartServices must be done first, since the OrderServices migration depends on the data from CartServices.

Options for Executing the Migrations

There are two options for how to do the data migration:

  1. Run the Migration Scripts via Liquibase

    • This option is quite simple, but requires that the cart, order, and paymenttransaction schemas all reside in the same database server, and therefore, Liquibase can effectively access each schema.

  2. Run the Migration Scripts via DBA Interaction

    • If the schemas reside in different database servers, then the migration needs to be executed via DBA interaction. These steps might look like:

      1. Copy the tables from the cart and order schemas into the paymenttransaction schema

      2. Run the same scripts, referencing the paymenttransaction schema, instead of the cart (or order) schema

Migrating Payments & Transactions from CartServices to PaymentTransactionServices

To migrate the data from cart.blc_cart_payment and cart.blc_cart_payment_transaction to paymenttransaction.blc_payment and paymenttransaction.blc_payment_transaction, we can use the following migration scripts, once the tables have been created.

Important
Migration from CartServices to PaymentTransactionServices must be done before migration from OrderServices to PaymentTransactionServices
Migrating Payments
INSERT INTO "paymenttransaction".blc_payment
(ID, OWNER_TYPE, OWNER_ID, NAME, SAVED_PAYMENT_METHOD_ID, TYPE, GATEWAY_TYPE, AMOUNT, SUBTOTAL, ADJUSTMENTS_TOTAL, FULFILLMENT_TOTAL, TAX_TOTAL, ADDRESSES, PAYMENT_METHOD_PROPERTIES, ATTRIBUTES, DISPLAY_ATTRIBUTES, SINGLE_USE_PAYMENT_METHOD, SHOULD_SAVE_PMT_TO_CUSTOMER, VERSION, TRK_ARCHIVED, AUDIT_CREATOR, AUDIT_CREATION_TIME, AUDIT_UPDATER, AUDIT_UPDATE_TIME, TRK_TENANT_ID, ACCESS_RESTRICTIONS)

SELECT cp.ID, 'BLC_CART', cp.CART_ID, cp.NAME, cp.CUSTOMER_PAYMENT_ACCOUNT_ID, cp.TYPE, cp.GATEWAY_TYPE, cp.AMOUNT, cp.SUBTOTAL, cp.ADJUSTMENTS_TOTAL, cp.FULFILLMENT_TOTAL, cp.TAX_TOTAL, CONCAT('[{"BILLING": ', cp.BILLING_ADDRESS, '}]'), cp.PAYMENT_GATEWAY_PROPERTIES, cp.ATTRIBUTES, cp.DISPLAY_ATTRIBUTES, cp.SINGLE_USE_PAYMENT_METHOD, cp.SHOULD_SAVE_PMT_TO_CUSTOMER, COALESCE(cart.version, 1), cp.ARCHIVED, cart.AUDIT_CREATOR, cart.AUDIT_CREATION_TIME, cart.AUDIT_UPDATER, cart.AUDIT_UPDATE_TIME, COALESCE(cart.TRK_TENANT_ID, 'UNKNOWN_TENANT_ID'),
	CASE
		WHEN cart.STATUS = 'SUBMITTED' THEN '["CUSTOMER_MUTABILITY_BLOCKED"]'
		WHEN cart.STATUS = 'CSR_OWNED' THEN '["CSR_OWNED"]'
		WHEN cart.STATUS = 'TEST' THEN '["TEST_USER_OWNED"]'
	END
FROM "cart".blc_cart_payment cp
LEFT JOIN "cart".blc_cart cart on cart.ID=cp.CART_ID
Default Values

If the cart cannot be found based on payment.cartId or the field is null, then the following fields will be set with the corresponding default values:

  • VERSION: 1

  • TRK_TENANT_ID: 'UNKNOWN_TENANT_ID'

  • AUDIT_CREATOR: NULL

  • AUDIT_CREATION_TIME: NULL

  • AUDIT_UPDATER: NULL

  • AUDIT_UPDATE_TIME: NULL

Additional Notes
  • The payments' access restrictions are set based on carts' status

  • The value of PAYMENT_GATEWAY_PROPERTIES in the old table is copied to PAYMENT_METHOD_PROPERTIES in the new table

  • TRK_CHANGE_DETAILS is not copied over to the new table

Migrating Payment Transactions
INSERT INTO "paymenttransaction".blc_payment_transaction
(ID, PAYMENT_ID, TYPE, SOURCE_ENTITY_TYPE, SOURCE_ENTITY_ID, MANAGEMENT_STATE, STATUS, TRANSACTION_REFERENCE_ID, SOURCE, REQUEST_ID, AMOUNT, DATE_RECORDED, GATEWAY_RESPONSE_CODE, FAILURE_TYPE, DECLINE_TYPE, THREE_D_SEC_VERIFICATION_URL, RAW_RESPONSE, PARENT_TRANSACTION_ID, ATTRIBUTES, CUSTOMER_IP_ADDRESS, INDETERMINATE_RESULT, VERSION, TRK_ARCHIVED, AUDIT_CREATOR, AUDIT_CREATION_TIME, AUDIT_UPDATER, AUDIT_UPDATE_TIME, TRK_TENANT_ID)

SELECT ctx.ID, ctx.PAYMENT_ID, ctx.TYPE, 'CHECKOUT_REQUEST', ctx.REQUEST_ID,
    CASE
	WHEN cart.STATUS = 'SUBMITTED' THEN 'AUTOMATIC_REVERSAL_NOT_ALLOWED'
	WHEN ctx.MANAGEMENT_STATE != 'REQUIRES_REVERSAL' AND ctx.MANAGEMENT_STATE != 'REVERSAL_IN_PROGRESS' AND ctx.MANAGEMENT_STATE != 'REVERSED' AND ctx.MANAGEMENT_STATE != 'FAILED_REVERSAL' AND ctx.MANAGEMENT_STATE != 'REVERSAL_TRANSACTION' THEN NULL
	ELSE ctx.MANAGEMENT_STATE
    END, ctx.STATUS, ctx.TRANSACTION_REFERENCE_ID, 'CART_OPERATION_SERVICES', ctx.REQUEST_ID, ctx.AMOUNT, ctx.DATE_RECORDED, ctx.GATEWAY_RESPONSE_CODE, ctx.FAILURE_TYPE, ctx.DECLINE_TYPE, NULL, ctx.RAW_RESPONSE, ctx.PARENT_TRANSACTION_ID, ctx.ATTRIBUTES, ctx.CUSTOMER_IP_ADDRESS, ctx.INDETERMINATE_RESULT, COALESCE(p.VERSION, 1), p.TRK_ARCHIVED, p.AUDIT_CREATOR, p.AUDIT_CREATION_TIME, p.AUDIT_UPDATER, p.AUDIT_UPDATE_TIME, COALESCE(p.TRK_TENANT_ID, 'UNKNOWN_TENANT_ID')
FROM "cart".blc_cart_payment_transaction ctx
LEFT JOIN "paymenttransaction".blc_payment p on p.ID=ctx.PAYMENT_ID
LEFT JOIN "cart".blc_cart cart on p.OWNER_ID=cart.ID;
Default Values

The script copies the following details from payments that would be created from the previous script (which would be obtained from the associated cart). If the payment cannot be found based on paymentTransaction.paymentId or the field is null, the these fields will be set with the corresponding default values:

  • VERSION: 1

  • TRK_TENANT_ID: 'UNKNOWN_TENANT_ID'

  • AUDIT_CREATOR: NULL

  • AUDIT_CREATION_TIME: NULL

  • AUDIT_UPDATER: NULL

  • AUDIT_UPDATE_TIME: NULL

  • ARCHIVED: NULL

Assumptions

THREE_D_SEC_VERIFICATION_URL is set to NULL. Since the old payment data didn’t capture a 3DS URL, the column is assumed to be NULL. This requires additional analysis on a per-client basis. In some cases, these urls may be stored in the transaction’s attributes map.

Additional Notes
  • Transactions' management states are set to AUTOMATIC_REVERSAL_NOT_ALLOWED if the associated cart status is SUBMITTED. If the cart status is not SUBMITTED and the management state is not one of the reversal states, then NULL is set. Otherwise, the existing management state value will be used.

Migrating Payments & Transactions from OrderServices to PaymentTransactionServices

Important
Migration from CartServices to PaymentTransactionServices must be done before migration from OrderServices to PaymentTransactionServices
Migrating Payments
INSERT INTO "paymenttransaction".blc_payment
(ID, OWNER_TYPE, OWNER_ID, NAME, SAVED_PAYMENT_METHOD_ID, TYPE, GATEWAY_TYPE, AMOUNT, SUBTOTAL, ADJUSTMENTS_TOTAL, FULFILLMENT_TOTAL, TAX_TOTAL, ADDRESSES, PAYMENT_METHOD_PROPERTIES, ATTRIBUTES, DISPLAY_ATTRIBUTES, SINGLE_USE_PAYMENT_METHOD, SHOULD_SAVE_PMT_TO_CUSTOMER, VERSION, TRK_ARCHIVED, AUDIT_CREATOR, AUDIT_CREATION_TIME, AUDIT_UPDATER, AUDIT_UPDATE_TIME, TRK_TENANT_ID, ACCESS_RESTRICTIONS)

SELECT op.ID, 'CART', blcOrder.CART_ID, op.NAME, op.CUSTOMER_PAYMENT_ACCOUNT_ID, op.TYPE, op.GATEWAY_TYPE, op.AMOUNT, op.SUBTOTAL, op.ADJUSTMENTS_TOTAL, op.FULFILLMENT_TOTAL, op.TAX_TOTAL, CONCAT('[{"BILLING": ', op.BILLING_ADDRESS, '}]'), op.PAYMENT_GATEWAY_PROPERTIES, op.ATTRIBUTES, op.DISPLAY_ATTRIBUTES, op.SINGLE_USE_PAYMENT_METHOD, op.SHOULD_SAVE_PMT_TO_CUSTOMER, 1, op.ARCHIVED, blcOrder.AUDIT_CREATOR, blcOrder.AUDIT_CREATION_TIME, blcOrder.AUDIT_UPDATER, blcOrder.AUDIT_UPDATE_TIME, COALESCE(blcOrder.TRK_TENANT_ID, 'UNKNOWN_TENANT_ID'), '["CUSTOMER_MUTABILITY_BLOCKED"]'
FROM "order".blc_order_payment op
LEFT JOIN "order".blc_order blcOrder on blcOrder.ID=op.ORDER_ID
WHERE op.ID NOT IN
      (SELECT ptsPayment.ID from "paymenttransaction".blc_payment ptsPayment);
Important Notes
  • Only payment records that don’t already exist in PaymentTransactionServices are migrated, this is done by checking the payment_id

  • The access_restrictions are set to ["CUSTOMER_MUTABILITY_BLOCKED"]

  • These payment records include Payment#ownerType = CART and Payment#ownerId references the cart id. This is done to reflect the fact that they originated with the order’s related cart, and that OrderOperationServices does not overwrite this reference.

  • The order version is set to 1 by default.

Migrating Payment Transactions
INSERT INTO "paymenttransaction".blc_payment_transaction
(ID, PAYMENT_ID, TYPE, MANAGEMENT_STATE, STATUS, TRANSACTION_REFERENCE_ID, SOURCE, REQUEST_ID, AMOUNT, DATE_RECORDED, GATEWAY_RESPONSE_CODE, FAILURE_TYPE, DECLINE_TYPE, THREE_D_SEC_VERIFICATION_URL, RAW_RESPONSE, PARENT_TRANSACTION_ID, ATTRIBUTES, CUSTOMER_IP_ADDRESS, INDETERMINATE_RESULT, VERSION, TRK_ARCHIVED, AUDIT_CREATOR, AUDIT_CREATION_TIME, AUDIT_UPDATER, AUDIT_UPDATE_TIME, TRK_TENANT_ID)

SELECT otx.ID, otx.PAYMENT_ID, otx.TYPE, 'AUTOMATIC_REVERSAL_NOT_ALLOWED', otx.STATUS, otx.TRANSACTION_REFERENCE_ID, 'ORDER_OPERATION_SERVICES', otx.REQUEST_ID, otx.AMOUNT, otx.DATE_RECORDED, otx.GATEWAY_RESPONSE_CODE, otx.FAILURE_TYPE, otx.DECLINE_TYPE, NULL, otx.RAW_RESPONSE, otx.PARENT_TRANSACTION_ID, otx.ATTRIBUTES, otx.CUSTOMER_IP_ADDRESS, otx.INDETERMINATE_RESULT, COALESCE(p.VERSION, 1), p.TRK_ARCHIVED, p.AUDIT_CREATOR, p.AUDIT_CREATION_TIME, p.AUDIT_UPDATER, p.AUDIT_UPDATE_TIME, COALESCE(p.TRK_TENANT_ID, 'UNKNOWN_TENANT_ID')
FROM "order".blc_order_payment_transaction otx
LEFT JOIN "paymenttransaction".blc_payment p on p.ID=otx.PAYMENT_ID
WHERE NOT EXISTS
    (SELECT 1 FROM "paymenttransaction".blc_payment_transaction
        AS ptsTransaction
    WHERE ptsTransaction.TRANSACTION_REFERENCE_ID = otx.TRANSACTION_REFERENCE_ID
      AND ptsTransaction.TYPE = otx.TYPE);
Important Notes
  • Only transactions that don’t already exist in PaymentTransactionServices are migrated, this is done by checking the combination of transaction_reference_id and type

  • Transaction management states are set to `AUTOMATIC_REVERSAL_NOT_ALLOWED `

  • Source is set to ORDER_OPERATION_SERVICES since this migration will only include transactions that were executed via OrderOperationServices

Migrating Transaction Logs

The legacy transaction log concept is replaced by having sourceEntityType, sourceEntityId, parentSourceEntityType, and parentSourceEntityId attributes on the payment transaction itself. This migration helps to align fulfillment-related actions with their relevant payment transactions.

-- update the source entity type and id for all of the transactions
UPDATE  "paymenttransaction".blc_payment_transaction ptsTransaction
SET SOURCE_ENTITY_ID = transactionLog.ENTITY_ID,
    SOURCE_ENTITY_TYPE = transactionLog.ENTITY_TYPE
FROM "order".BLC_ORDER_PMT_TRANSACTION_LOG transactionLog
WHERE transactionLog.PAYMENT_TRANSACTION_ID = ptsTransaction.ID;

-- update the parent source entity type and id for all of the child transactions
UPDATE "paymenttransaction".blc_payment_transaction childTransaction
SET PARENT_SOURCE_ENTITY_ID = parentTransaction.SOURCE_ENTITY_ID,
    PARENT_SOURCE_ENTITY_TYPE = parentTransaction.SOURCE_ENTITY_TYPE
FROM "paymenttransaction".blc_payment_transaction parentTransaction
WHERE childTransaction.PARENT_TRANSACTION_ID = parentTransaction.ID;
Verifying the Migrations

Once the migration scripts have been executed, it’s important to verify that the migrated data looks correct.

Especially keep an eye out for any records that contain TRK_TENANT_ID = 'UNKNOWN_TENANT_ID'. These records indicate that a cart could not be identified for the payment.

Configuration Properties

Added Properties

  • broadleaf.paymenttransaction.service.payment-lock-ttl

    • Description: The amount of time that a payment lock is held, before it expires.

    • Default value: 10 seconds

  • broadleaf.paymenttransaction.service.anonymous-payment-ttl-enabled

    • Description: Whether to expire payments for anonymous users that have exceeded the anonymous payment TTL duration.

    • Default value: true

  • broadleaf.paymenttransaction.service.anonymous-payment-ttl

    • Description: The amount of time payments for anonymous users should be considered valid.

    • Default value: 60 minutes

  • broadleaf.paymenttransaction.customerprovider.url

    • Description: The base url for an external customer service.

  • broadleaf.paymenttransaction.customerprovider.payment-accounts-uri

    • Description: The URI path for basic CRUD operations on payment accounts.

  • broadleaf.paymenttransaction.customerprovider.service-client

    • Description: The service client to use when calling customer services.

    • Default value: "paymenttransactionclient"

Gateway Configuration

This service needs to be registered with both the Commerce and Admin gateways, using the following configuration:

Commerce Gateway

broadleaf:
  gateway:
    proxyurls:
      paymenttransaction: https://localhost:8476
    predicates:
      paymenttransaction: /api/payment/**
    filters:
      paymenttransaction: /api/payment/?(?<segment>.*), /$\{segment}
spring:
  cloud:
    gateway:
      routes:
        - id: payment
          uri: ${broadleaf.gateway.proxyurls.paymenttransaction}
          predicates:
            - Path=${broadleaf.gateway.predicates.paymenttransaction}
          filters:
            - RewritePath=${broadleaf.gateway.filters.paymenttransaction}
            - ApplicationToken
            - OAuth2ClientCredentials=anonymous

Admin Gateway

broadleaf:
  gateway:
    proxyurls:
      paymenttransaction: https://localhost:8476
    predicates:
      paymenttransaction: /api/payment/**
    filters:
      paymenttransaction: /api/payment/?(?<segment>.*), /$\{segment}
spring:
  cloud:
    gateway:
      routes:
        - id: payment
          uri: ${broadleaf.gateway.proxyurls.paymenttransaction}
          predicates:
            - Path=${broadleaf.gateway.predicates.paymenttransaction}
          filters:
            - RewritePath=${broadleaf.gateway.filters.paymenttransaction}

Application Configuration

broadleaf:
  paymenttransaction:
    liquibase:
      change-log: 'classpath:/db/changelog/paymenttransaction.flexdemo.postgresql.changelog-master.yaml'
      liquibase-schema: public
      default-schema: paymenttransaction
    delegating:
      schema: paymenttransaction
      delegate-ref: composite
    customerprovider:
      url: 'https://localhost:8447'
Note

Default ports for customer provider:

  • Min flex package: 8447

  • Balanced flex package: 9458

  • Granular flex package: 8463

Auth Configuration

Change in default client names for service-to-service communication

The default client configurations have changed for service-to-service communication.

In short, the following properties need to be configured:

spring:
  security:
    oauth2:
      client:
        registration:
          paymenttransactionclient:
            authorization-grant-type: client_credentials
            client-id: paymenttransactionclient
            client-secret: payment_transaction_secret
        provider:
          paymenttransactionclient:
            token-uri: https://localhost:8443/oauth/token

For more details on the full scope of these changes, please review the AuthServices release notes.

New Permissions

There are new permissions and scopes for some service OAuth2 clients. Permissions and scopes can be added via admin or sql script.

See AuthServices release notes for more details.

Auth Server Service Service ID New Scopes New Permissions

Services

Anonymous Gateway Client

anonymous

CUSTOMER_PAYMENT_MANAGEMENT

ALL_CUSTOMER_PAYMENT_MANAGEMENT

Services

Payment Transaction Service Client

paymenttransactionclient

SENSITIVE_CUSTOMER_PAYMENT_ACCOUNT

READ_SENSITIVE_CUSTOMER_PAYMENT_ACCOUNT

Services

Customer Service Client

customerclient

SENSITIVE_PAYMENT

READ_SENSITIVE_PAYMENT

Services

Cart Operation Service Client

cartopsclient

SYSTEM_PAYMENT_MANAGEMENT, EXECUTE_AUTHORIZE, EXECUTE_AUTHORIZE_AND_CAPTURE

ALL_SYSTEM_PAYMENT_MANAGEMENT, ALL_EXECUTE_AUTHORIZE, ALL_EXECUTE_AUTHORIZE_AND_CAPTURE

Services

Order Ops Service Client

orderopsclient

SYSTEM_PAYMENT_MANAGEMENT, EXECUTE_AUTHORIZE, EXECUTE_AUTHORIZE_AND_CAPTURE, EXECUTE_REVERSE_AUTHORIZE, EXECUTE_CAPTURE, EXECUTE_REFUND

ALL_SYSTEM_PAYMENT_MANAGEMENT, ALL_EXECUTE_AUTHORIZE, ALL_EXECUTE_AUTHORIZE_AND_CAPTURE, ALL_EXECUTE_REVERSE_AUTHORIZE, ALL_EXECUTE_CAPTURE, ALL_EXECUTE_REFUND