Broadleaf Microservices

Sandboxing In Detail

Entity Structure

Here’s a basic JPA domain class that is configured with sandboxing support:

@Entity
@Table(name = "BLC_PRODUCT", indexes = {...})
@Inheritance(strategy = InheritanceType.JOINED)
@Data
@EqualsAndHashCode(exclude = {"_id", "options", "includedProducts", "fulfillmentFlatRates"})
@EntityListeners(TrackingListener.class)                                                        (1)
@TrackableExtension({TrackableBehavior.SANDBOX, TrackableBehavior.CATALOG})                     (2)
public class JpaProduct
        implements CatalogTrackable<CatalogJpaTracking>                                         (3)
        , ModelMapperMappable                                                                   (4)
        , BusinessTypeAware                                                                     (5)
        , Serializable, CurrencyProvider, CurrencyConsumer,
            Translatable, ActiveAware, Indexable {

    ...

    @Id
    @GeneratedValue(generator = "blcid")
    @GenericGenerator(name = "blcid", strategy = "blcid")
    @Type(type = "com.broadleafcommerce.data.tracking.jpa.hibernate.ULidType")
    @Column(name = "ID", nullable = false, length = CONTEXT_ID_LENGTH)
    private String _id;                                                                         (6)

    @Column(name = "CONTEXT_ID", length = CONTEXT_ID_LENGTH)
    @Convert(converter = UlidConverter.class)
    private String contextId;                                                                   (7)

    @Embedded
    private CatalogJpaTracking tracking;                                                        (8)

    ...

    @Override
    @NonNull
    public ModelMapper fromMe() {                                                               (9)
        final ModelMapper mapper = new ModelMapper();
        mapper.getConfiguration().setMatchingStrategy(MatchingStrategies.STRICT);

        ...

        return mapper;
    }

    @Override
    @NonNull
    public ModelMapper toMe() {
        ModelMapper mapper = new ModelMapper();
        mapper.getConfiguration().setMatchingStrategy(MatchingStrategies.STRICT);

        ...

        return mapper;
    }

    ...

    @Override
    public Class<?> getBusinessDomainType() {                                                   (10)
        return Product.class;
    }

    ...
  1. TrackingListener is required for JPA sandboxable domain. It performs some additional setup before persistence.

  2. (Optional) Use @TrackableExtension to describe the granular Tracking features supported. If not declared, this is generally inferred by (3). Note, TenantJpaTracking and CustomerContextJpaTracking already do not support sandboxing, so there’s no need to include a @TrackableExtension declaration in order to omit that feature.

  3. A Trackable interface is required. JPA entity classes generally declare a more derived type to meet the specific discrimination needs with the minimum table column requirements (e.g. CatalogTrackable). Mongo entity classes just use Trackable here.

  4. (Optional) Declare the class to use explicit mapping exposed by the ModelMapperMappable interface. Provides more control over mapping to projection. If not declared, the system will infer mapping.

  5. (Optional) Declare the class is aware of an explicit projection type and that results should be mapped via the implemented BusinessTypeAware interface methods. If not declared, the system will infer a projection and auto map results.

  6. (Optional) Declare a primary key column separate from the context id. This is required for entities that are catalog and/or sandbox discriminated. Entities that require neither sandbox or catalog behavior don’t require a primary key definition separate from the context id.

  7. Context id is a requirement of the Trackable interface. It serves as a grouping construct for sandbox and catalog overrides and serves as the primary vehicle the system uses to expose the correct version of an entity base on the caller’s viewing context. In some cases, it is also the primary key when catalog or sandbox grouping is not required.

  8. A concrete Trackable implementation is required that matches the Trackable interface type for the entity class (e.g. CatalogJpaTracking). In JPA, this is included as an Embeddable. For Mongo entities, this can simply be declared as a vanilla Tracking type.

  9. (Optional) If declaring the ModelMapperMappable interface for the entity class, implement the required mapping methods to achieve fine-grained control.

  10. (Optional) If declaring the BusinessTypeAware interface for the entity class, implement the required method to return the explicit projection type that should be used.

Note
Broadleaf suggests using the ULID type for any primary key and context id values. This type has advantageous behavior across datacenters, as it avoids the likelihood of id collision. It is also lexicographically sortable, which makes it work well for database indexes. The default configuration is suitable for most cases and Broadleaf offers a JPA converter and Hibernate Type and generator for primary keys (see example above). See the spec for more information on ULID and it’s inherent advantages.

Here is an example of a JPA entity class that does not leverage most of the optional items:

@Entity
@Table(name = "JPA_SAMPLE")
@Data
@EqualsAndHashCode(exclude = "id")
@EntityListeners(TrackingListener.class)
@TrackableExtension(TrackableBehavior.APPLICATION)
public class Sample implements Serializable, ApplicationTrackable<ApplicationJpaTracking> {

    @Id
    @GeneratedValue(generator = "blcid")
    @GenericGenerator(name = "blcid", strategy = "blcid")
    @Type(type = "com.broadleafcommerce.data.tracking.jpa.hibernate.ULidType")
    @Column(name = "ID", nullable = false, length = CONTEXT_ID_LENGTH)
    private String contextId;

    @Embedded
    private ApplicationJpaTracking tracking;

    ...
}

This sample does not declare ModelMapperMappable or BusinessTypeAware, so the system will infer a projection type and will auto map to the inferred projection. This entity is also not catalog discriminated, and it explicitly omits sandbox behavior in its @TrackableExtension, so the contextId and primary key are collapsed into a single field.

Note
When the system infers a projection type, you can programmatically gain access to the projection’s API for mutator methods, etc…​ via the Projection interface. In the "Sample" entity class case above, service or endpoint code can instantiate a new projection proxy instance by simply calling the projection factory like this: Projection<Sample> myProjection = Projection.get(Sample.class). Once you have the projection instance, you can mutate it through the proxy using the entity’s API using: Sample proxy = myProjection.expose(). Any mutator calls on the proxy will now pass through to the delegated projection. This can be useful, for example, in customization scenarios where you need to pass an inferred projection instance to a service (or other Broadleaf code) that requires a projection instance. This is covered in more detail as part of extensibility.

It is possibly to achieve ala carte behavior using @TrackableExtension. It is not uncommon to declare the annotation with APPLICATION behavior only, for example. In this case, changes to the entity will be application discriminated, but will be immediately live in production and will not use an approval workflow.

Behaviors

If using the @TrackableExtension annotation, an entity only adheres to the behaviors declared.

  1. SANDBOX - Will persist user changes to a sandbox and participate in promotion and deploy workflows.

  2. CATALOG - Will persist user changes to a catalog and discriminate reads based on the context of the request. Catalog discriminated entities also participate in propagation where field-level changes trickle down to subordinate catalogs.

  3. APPLICATION - Will persist user changes to an application and discriminate reads based on the context of the request. An application can see its discriminated entities, any catalog discriminated entities belonging to catalogs assigned to the application, and any tenant discriminated entities based on the tenant to which the application belongs.

  4. TENANT - Will persist user changes to a tenant and discriminate reads based on the context of the request. Tenants are composed of one or applications and catalogs. A single Broadleaf installation can contain multiple tenants, each consisting of different applications. A Tenant discriminated entity will be visible to all contexts for the tenant, including all applications.

  5. CUSTOMER-CONTEXT - Discriminate based on the customer context ID of the request. This is a special case type and is usually used in conjunction with TENANT. Entities that specifically relate to a customer include this behavior. For example, JpaCustomer and JpaCustomerAddress leverage this type.

In general, there will be variations in persistence behavior based on several domain configuration factors:

Operation Sandbox Enabled Catalog Discrimination Enabled Application Discrimination Enabled Action

User Save

X

X

-

Sandbox version is created and catalog assigned (if an ADD)

User Save

X

-

X

Sandbox version is created and application assigned (if an ADD)

User Save

-

X

-

Production version is updated (or created for ADD) and catalog propagation executes immediately

User Save

-

-

X

Production version is updated (or created for ADD)

User Save

X

-

-

Sandbox version is created

Promote

X

X

-

New sandbox version is created in same sandbox with increased visibility

Promote

X

-

X

New sandbox version is created in same sandbox with increased visibility

Promote

X

-

-

New sandbox version is created in same sandbox with increased visibility

Deploy

X

X

-

Production version is updated (or created for ADD) and catalog propagation executes immediately

Deploy

X

-

X

Production version is updated (or created for ADD)

Deploy

X

-

-

Production version is updated (or created for ADD)

Data Requirements

The Trackable domain is used by a repository platform-specific NarrowExecutor for read and write operations. When reading, additional query parameters are applied to the query based on the current ContextInfo. When writing, the tracking field is populated with information based on the current ContextInfo. See the Repository section for more information.

To achieve appropriate sandbox, catalog and application filtering in a service with discriminatable entities, there are data requirements in three areas:

  1. ContextRequest fields for sandbox, catalog and application passed in the request header

  2. Tracking information on the discriminated entity itself referencing sandbox, catalog or application

  3. Application and Catalog hierarchy persisted in the same datastore

The X-Context-Request header allows service requests to request filtering parameters. These include:

  1. applicationId - For application discriminated entities, the applicationId is used to directly filter on the entity’s application discriminator. For catalog discriminated entities, the applicationId is used to determine the visible catalogs for the requested application and filtering is performed using those visible catalogs. This applicationId is also used when application discriminated entities are added and is set as the newly persisted entity’s application discimination value.

  2. catalogId - For catalog discriminated entities, the catalog id is used when the entities are added and is set as the newly persisted entity’s catalog discrimination value. During fetch requests, it is also used when the ContextRequest forceCatalogForFetch value is true and serves to force restriction of returned entity instances to only this catalog and its parent hierarchy. This is different than when using the application, which filters on all application visible catalogs, rather than just a single explicit catalog.

  3. sandboxId - For sandboxable entities, the sandboxId drives the sandbox targeted for all CRUD operations. Newly persisted sandbox versions based on user changes will be associated with this sandboxId. Furthermore, fetch requests will be limited to this sandbox version, or the production version, whichever is closer.

Discriminated entities leverage several fields that drive storage of application, catalog and sandbox information used during filtered query fetches.

  1. Tracking.sandbox.contextId - Represents the sandboxId this entity is associated with. This value will be null for production entities.

  2. Tracking.catalog.contextId - Represents the catalogId this entity is associated with. This value can be null for catalog discriminated entities, but in such a case, the entity will be available to all requests.

  3. Tracking.application.contextId - Represents the applicationId this entity is associated with. This value can be null for application discriminated entities, but in such a case, the entity will be available to all requests.

Tracking information is generally set automatically during persistence operations through the admin tool. However, it is useful to be aware of these requirements in the case of import operations coming from another source.

Finally, valid application and catalog structures are required to derive important hierarchy and catalog visibility information based on the simple applicationId and catalogId provided in the request header. These structures are available in the same microservice datastore as the discriminated entities themselves. These structures are generally not kept up-to-date by this microservice, but are instead managed by the Tenant microservice and subsequent synchronization keeps the local stuctures in parity with those of the Tenant service. The primary describing business domain is represented by:

  1. com.broadleafcommerce.micro.common.data.state.tracking.tenant.domain.Catalog

  2. com.broadleafcommerce.micro.common.data.state.tracking.tenant.domain.Application

Sandbox Transition Handling

A transition request (WorkflowTransitionRequest) contains information around a specific sandbox change. The transition request is sent by the sandbox service and is consumed by all resource tier services that manage sandboxable entities. Each resource tier service is responsible for making the determination of whether or not it can and should handle a transition request for a particular type. The information includes sandbox, catalog, application, business domain, context id, and the function to perform on the change. The four base functions are promote (Promotion), deploy (Deployment), revert (Reversion), and reject (Rejection). Each of these has a message channel using Spring Cloud Stream. Our default binder implementation is Kafka.

A TransitionHandler defines how to handle sandbox transitions for a specific provider entity in regards to each of the types. Out of the box, we provide a DefaultTransitionHandler that utilizes TransitionHelper for performing the specified function on the request details.