- v1.0.0-latest-prod
Data Workflow (Sandboxing) requires that a user be able to make a change to a sandboxable entity and have those changes isolated to an individual instance that only that user can see. Furthermore, promotion workflows require that a user be able to promote a change forward into one or more approval views that possibly have visibility to multiple users. As a result, approval sandboxes contained merged results of work from multiple users and employ an optimistic strategy where the last change wins, rather than locking records. Merging of changes are performed on a field-by-field basis, rather than an entire entity.
Users may revert changes made in the user’s sandbox. Approvers may reject changes made in an approval sandbox, which results in one or more changes being moved backwards in the workflow - possibly all the way back to the originating user.
Users capable of scheduling deployments may do so once the change has passed all approval gateways. Scheduling for deployment may be immediate, or may be set for a future date/time.
Sandbox state for an entity is maintained in supporting resource tier services. The data tracking shared library provides the sandbox support that is common to all supporting resource tier services. Messages controlled by the Sandbox Service notify supporting resource tier services to transition entities in sandbox state through the entire sandbox lifecycle.
Core - Core support across all provider types
JPA BinaryId - JPA Binary storage for ULID primary key
JPA CharID - JPA Varchar storage for ULID primary key (default)
JPA Core - JPA core support (Spring Data JPA)
JPA Hibernate - Hibernate specific support behind JPA
DefaultTrackableDomainMapperMemberSupport manages the sandbox and catalog state during CRUD operations on Trackable entities:
CREATE - Establish a new entity instance marked with the user sandbox id and level (TrackingLevel#USER).
Newly created items are also set to the current context catalog and their created application property is set to
the current context application.
UPDATE - If updating a production record (TrackingLevel#PRODUCTION), then create a new copy of the
entity marked with the user sandbox id and level. Otherwise, if updating an existing sandbox record, simply
update the sandbox state. The sandbox record’s catalog should match the item being updated.
DELETE - If deleting a production record, then create a new copy of the entity marked with the user sandbox
id and level. Additionally, the entity is marked with a change type of OperationType#DELETE. See
Sandbox#changeType. The sandbox record’s catalog should match the item being deleted.
DefaultTrackableDomainMapperMemberSupport is also responsible for maintaining audit information on changes detected during CRUD operations.
ChangeDetail instances represent a change made by the user and are stored in the Trackable entity’s change details.
TrackableRepository manages the persistence state of Trackable entities and more specifically manages the fetch of
sandboxable entities, including narrowing behavior. For example, when a fetch query is issued, narrowing functionality
should return the most specific version applicable to the context of the caller. If User A has a user sandbox version of
a product, then fetch results that include that product should return the sandbox version, rather than the production
version of the product. This should result in a trickle-down effect from the production version of an item, down through
approval sandbox versions, and finally to the user sandbox version.
Narrowing is accomplished by including an additional Narrow annotation to an interface repository definition
extending from TrackableRepository. Narrow annotations take a required NarrowExecutor class argument. The NarrowExecutor
is responsible for executing the fetch query defined by the repository query method (or by direct Query definitions
through repository specific helper - for example - JpaNarrowingHelper). Furthermore, it is responsible for possibly
enhancing the query to perform additional filtering, such as sandbox narrowing and/or catalog filtering.
Tracking info is kept resident to the record, including change details
These change details can also serve as a robust audit trail served via an event at the time of production deployment
The admin CRUD lifecycle works relatively well with NOSQL document atomicity alone (i.e. no transactions), especially when you consider the tracking/change information is kept resident in the document
This design also gives some thought to promotions, including mixed promotions from multiple users.
Change details can be mixed from different users and timestamps
This model also opens up the possibility of rejecting individual field changes nicely.
The data structure itself is resilient during a multi-item tracking promote. A scheduled process can run looking for unfinished promotions and just pick up any items it finds still active and resident in the previous, unfinished tracking.
There’s also a "stage" concept that serves as a generic tracking advancement tag concept. Different workflows could be configured in this system to advance changes through different "stages".
Promotion is the mechanism by which user sandbox state changes are moved to higher visibility levels, until the changes
are eventually promoted to the production record. Presumably, at each stage, the change is available for review
and may be advanced or rejected (usually via the admin tool). WorkflowTransitionHelper is responsible for
executing the promotion steps and is capable of mapping ChangeDetails from the original user change and
applying those changes to another version of the record with higher visibility in the same sandbox. The highest
level promotion (referred to as a deployment) occurs when WorkflowTransitionHelper applies ChangeDetails to
an existing production record, or in the case of an ADD, creates a new production record.
Promotion is only one type of transition available. Sandbox state can also be transitioned backward for unwanted changes, or changes requiring further edits. The revert transition can be used to undo a user level change, which completely removes the change from the system. An approval level user can reject a promoted change, send the change back to the author for further edits.
There are also approval and deployment transitions. Approval transitions denote that an approval level user has reviewed the change and declared it ready for deployment. Deployment transitions apply the change to the production state, either now - or a scheduled time in the future.
Catalogs allow grouping of items under a common label. It is commonly used by merchandisers to represent how products are distributed across store and channels. For example, it is common to have catalogs by region as a vehicle for defining how stores in different localities are merchandised differently. This is often thought of as products, but can be any piece of data in the catalog service that is influenced by your catalog partitioning scheme.
The Application is used to drive catalog (and therefore merchandise) visibility during the shopping experience. It is useful to note that there are two tenant filtering concepts: catalog discrimination and application discrimination. Catalog discrimination is based on the catalogs visible to the current application. Application discrimination is more straightforward, and for those entities that are filtered on application rather than catalog, visibility can be based solely on the current application, and catalog is ignored.
See MultiTenancy and Tenant Services for more details.
Propagation occurs at the time of deployment and involves taking the deployed
ChangeDetails and applying those changes to more derived versions of the record in child catalogs. At this point,
the notion of collision detection becomes important. If the more derived version of the record in the child
catalog contains pre-existing changes on one or more of the fields in the propagating ChangeDetails, this is
considered a collision and that individual change is not advanced further, while other non-colliding changes
are allowed to continue to trickle down.