@DataRouteByExample(InventoryLocation.class)
@FrameworkRestController
@FrameworkMapping(InventoryLocationEndpoint.INVENTORY_LOCATION_URI)
public class InventoryLocationEndpoint {
//...
}
- v1.0.0-latest-prod
Data Routing is a general term that describes a somewhat transparent, but important architectural concept in Broadleaf Commerce Microservices. Data Routing is the dynamic routing of database activity based on the entry point or flow of the application. Broadleaf’s Microservices can be deployed as stand-alone services, or as FlexPackages. FlexPackages are bundled service deployments, where 2 or more services are composed into a single Spring Boot application. The individual services within the FlexPackage usually have their own database (or at least their own database schema within a database).
We’re addressing the following with Data Routing:
Ensure data is written to and read from the correct database(s) given a specific context
Prevent Spring Bean collisions that would otherwise occur with composed services
Support the use of cross-cutting libraries and database tables
Provide a mechanism to allow for common "pipeline contributions"
Routes are generally established at entry points into the application. The most obvious places are Spring REST Controllers and Message Listeners. Consider the following Spring REST Controller from Broadleaf’s Inventory Service:
@DataRouteByExample(InventoryLocation.class)
@FrameworkRestController
@FrameworkMapping(InventoryLocationEndpoint.INVENTORY_LOCATION_URI)
public class InventoryLocationEndpoint {
//...
}The @DataRouteByExample annotation provides the AOP annotation to indicate which data route should be used. Again, this would not normally be required if Inventory Service was deployed by itself. But when it’s colocated in the same application as other services that use different databases, this is required. Incidentally, this is the same as (synonymous with) this:
@DataRouteByKey("inventory") // RouteConstants.Persistence.INVENTORY_ROUTE_KEY
@FrameworkRestController
@FrameworkMapping(InventoryLocationEndpoint.INVENTORY_LOCATION_URI)
public class InventoryLocationEndpoint {
//...
}Also, consider that a Message Listener is an entry point into the application. As such, it also requires us to establish a route:
@DataRouteByKey(RouteConstants.Persistence.INVENTORY_ROUTE_KEY)
public class OrderSubmittedInventoryAdjustmentMessageListener extends
AbstractInventoryAdjustmentListener {
@StreamListener(InventoryCheckoutCompletionConsumer.CHANNEL)
public void listen(final Message<String> message) {
//...
}
}Broadleaf leverages Spring Data to reduce the boiler plate code typically required for Database interactions. Typically, Spring uses defaults for configuring JPA repositories to reduce configuration code. When using multiple databases in the same application, additional configuration is required by Spring Data - even when you are not using Broadleaf!
Spring Data JPA reduces boiler plate code that typically exists in DAO / repository layers. When using multiple repositories with Spring Data, additional configuration is required requires configuration for each repository.
@Configuration
@EnableJpaRepositories(
basePackageClasses = {MyJpaRepository.class},
entityManagerFactoryRef = "myEntityManager",
transactionManagerRef = "myTransactionManager"
)
@EntityScan("my.domain")
public class MyRepositoryConfig {
@Bean
@Primary
public LocalContainerEntityManagerFactoryBean myEntityManager() {
//...
}
@Primary
@Bean
public DataSource myDataSource() {
//...
}
@Primary
@Bean
public PlatformTransactionManager myTransactionManager() {
//...
}
}The code example, shows a typical configuration. In it, you need to:
Enable the Spring Data JPA repository by setting the basePackageClasses to our Repository
Enable an entity manager for this repository
Enable a transaction manager for this repository
Scan the domain classes that we want this entity manager and repository to manage
Broadleaf extends Spring’s default behavior:
@Configuration
@EnableJpaRepositories(
basePackageClasses = {MyJpaRepository.class},
repositoryFactoryBeanClass = JpaTrackableRepositoryFactoryBean.class,
entityManagerFactoryRef = Constants.MY_ENTITY_MANAGER_FACTORY,
transactionManagerRef = Constants.MY_TRANSACTION_MANAGER)
@JpaEntityScan(basePackages = {"my.domain"},
routePackage = {Constants.MY_ROUTE_PACKAGE})
@EnableConfigurationProperties(MyJpaProperties.class)
@JpaDataRoute(
boundPropertiesType = MyJpaProperties.class,
routePackage = Constants.MY_ROUTE_PACKAGE,
routeKey = Constants.MY_ROUTE_KEY,
supportingRouteTypes = {TrackingDataRouteSupporting.class,
TranslationDataRouteSupporting.class,
ImportDataRouteSupporting.class})
public class MyJpaAutoConfiguration {
// No custom beans required
}This code example represents the additions to the Spring Data JPA configuration required by Broadleaf.
Spring allows us to specify the class to instantiate for our JpaRepository. We want to use a JpaTrackableRepositoryFactoryBean in order to get data tracking behavior.
We use a strict naming convention for the entityManager, transactionManager and route-related properties. The @JPADataRoute component registers beans for this repository based on the ROUTE_KEY. For example, if my RouteKey was "inventory" then it would register an EntityManagerFactory bean named "inventoryEntityManagerFactory". The conventions used in other microservices for naming for these constants is intentional and important.
Since we need to configure datasource properties for multiple services, we need a property prefix for our common properties. We also need to enable it and pass it as a property to the @JPADataRoute annotation. In this example, that is defined by MyJpaProperties. For example,
@Data
@ConfigurationProperties("my.domain")
public class MyJpaProperties implements JpaPropertyRelated {
private JpaProperties jpa = new JpaProperties();
private DataSourceProperties datasource = new DataSourceProperties();
private LiquibaseProperties liquibase = new LiquibaseProperties();
private SchemaDelegatingProperties delegating = new SchemaDelegatingProperties();
}@JPADataRoute is a workhorse annotation that performs a lot of tasks that allow Data Routing to work. It actually registers a number of Spring Bean Registrars:
@Import({
JpaSpringLiquibaseFactoryBeanRegistrar.class,
JpaDataRouteEntityManagerFactoryBeanRegistrar.class,
JpaDataRoutableDataSourceBeanRegistrar.class,
JpaPackageDataRouteSupplierBeanRegistrar.class,
JpaEmbeddedDataSourceFactoryBeanRegistrar.class,
JpaPooledDataSourceFactoryBeanRegistrar.class,
JpaTransactionManagerFactoryBeanRegistrar.class,
JpaTransactionTemplateFactoryBeanRegistrar.class})
public @interface JpaDataRoute {
//...
}These extra annotations provide the benefit of auto-configuration, allowing the necessary beans to be auto-registered without additional (manual) configuration.
Finally, the supportingRouteTypes are one of the main reasons that routing exists. For every common data concept your service uses, you’ll need to provide a supportingRouteType.
| Supporting Route Type | Description |
|---|---|
TrackingDataRouteSupporting.class |
Support for tracking of data changes |
ApplicationDataRouteSupporting.class |
Provide service with synced application data |
TranslationDataRouteSupporting.class |
Support for common translation data and functionality |
CatalogDataRouteSupporting.class |
Support for shared catalog concepts |
ImportDataRouteSupporting.class |
Support for import features |
ExportDataRouteSupporting.class |
Support for export features |
BulkDataRouteSupporting.class |
Support for bulk features |
MessagingDataRouteSupporting.class |
Supports common messaging and message routing features |
MarketplaceDataRouteSupporting.class |
Supports common marketplace data and functionality |
These different DataRouteSupporting classes are essentially marker interfaces that indicate that the particular route will participate in those shared features. It also means that the database (or schema) associated with the route will contain the necessary tables to support those features. We’ll talk about Translations as an example later.
There are several components related to the javax.sql.DataSource that are used for data routing in Broadleaf Microservices. Here is a summary:
| Data Source Component | Description |
|---|---|
RoutableDataSoure |
An interface that has a single method called |
RoutingJpaDataSource |
This is an extension of Spring’s |
CompositeDataSource |
Concept that allows services to share the same
To work around that, we created a |
DataSourceUtil |
This is a utility that essentially constructs the |
Proper configuration, described above, will configure your DataSources for you and routing will be handled automatically if the proper route is established.
DomainMapperMember components implement the interface of the same name. They participate in the "pipeline" of activities to map a projection instance into a persistence instance and vice versa. However, not all DomainMapperMember instances should be invoked for all entities. For example, if an entity is not translatable or not trackable, then those members responsible for that functionality should not be invoked in that route. All DomainMapperMembers implement DataRoutePartitionAware, which informs the pipeline about which route(s) should be involved in that particular mapping activity. Consider this code snippet:
public class DomainMapperManager {
@Override
public <P> P fromRepositoryDomain(Object repositoryDomain,
Class<P> businessDomainType,
ContextInfo contextInfo) {
ParamHolder<P> param = new ParamHolder<P>()
.withParam(typeFactory.get(businessDomainType));
DataRouteSupportUtil.findMembersInScopeByExample(members, reference, businessDomainType)
.forEach(
member -> param.setParam(
member.fromRepositoryDomain(repositoryDomain, param.getParam(),
contextInfo)));
return param.getParam();
}
}The DomanMapperManager uses the DataRouteSupportUtil to determine which DomainMapperMember should be invoked for the given pipeline, mapping a repository (persistent) entity to a projection (business) entity. This is informed by the route, or the context or scope of the entity that we are mapping.
Translation functionality is used by multiple services (Catalog, Offers, Menu, Personalization, etc.). Translation functionality is provided by a common library called TranslationsCommon. This common library allows storing and retrieving of translated values for one or more properties of an entity based on locale. Translation data needs to be co-located with the data that it supports (e.g. Menu). The TranslationCommon library provides translation support, including Spring Data (JPA) entities and Repositories, Projections, Services, etc. This functionality always uses the BLC_TRANSLATION table.
But if MenuService and CatalogService are bundled together, we know that they each have their own BLC_TRANSLATION table in their database or schema. So which one do we use?
Any business service (e.g. Menu) that uses TranslationCommon will be able to access Translation APIs, Repository, and Data. As discussed, since Services are designed to be deployed in a decoupled way from one another, TranslationCommon normally contributes tables, APIs, Entities, Projections, Mappers, and Repositories to the services in which they are deployed. When deployed as a Common library in a composed or otherwise "FlexPackage" service, we need a way to share the code and functionality but otherwise isolate the data.
Inside a "FlexPackage-composed" service deployment, we use the table in the database or schema that is associated with the current route. Consider the following image that shows a snippet of a database table browser. Notice the menu schema and the offer schema. They each have a BLC_TRANSLATION table. The one that will be used will depend on which route we are in - menu or offer.
Incidentally, notice that BLC_APPLICATION and BLC_APPLICATION_CATALOG are also tables that are duplicated in this example as they are associated with the ApplicationDataRouteSupporting route type.
Translation Services APIs can be called (and properly routed) directly from an Endpoint. For example, the MenuEndpoint provides an REST API to fetch translations for a menu:
@FrameworkRestController
@FrameworkMapping(MenuEndpoint.MENUS_URI)
@RequiredArgsConstructor
@DataRouteByExample(Menu.class)
public class MenuEndpoint {
//...
@FrameworkPutMapping(value = "/{id}/translations/{locale}",
consumes = MediaType.APPLICATION_JSON_VALUE)
@Policy(permissionRoots = {"MENU"})
public TranslationsPayload replaceAllMenuTranslations(
@ContextOperation(OperationType.UPDATE) ContextInfo context,
@PathVariable("id") String id,
@PathVariable("locale") Locale locale,
@RequestBody TranslationsPayload translationRequest) {
// will return 404 if entity doesn't exist
menuService.readByContextId(id, context);
String entityType = menuService.getRepositoryDomain();
List<Translation> updatedTranslations =
translationEntityService.bulkReplaceTranslationsForEntityInLocale(entityType, id,
locale, translationRequest.getTranslations(), context);
return new TranslationsPayload(updatedTranslations);
}
}