Broadleaf Microservices
  • v1.0.0-latest-prod

Import Release Notes for 1.8.1-GA


The changes in this release are largely motivated by the goal of simplifying and improving the import processes.

This release contains significant changes, including several breaking changes. However, in most cases, the required migrations will be trivial.

The main target of change is product import, which, combined with the changes in Catalog Services 1.8.1 and Import Consumer 1.7.1, has been fully re-written.

As ImportServices is a central component to import processes for many microservices, the common functionality and specification changes made to support the new product import affect other imports as well.

To be clear, the contract between ImportServices and resource tier services has not changed. The resource tier handlers in services other than CatalogServices have not been modified in this release, and will continue to operate as they did before. The 'required migration' for the other imports is mainly about updating their specifications (located in ImportServices) to use the new methods/implementations to define field configuration and move away from the now-deprecated methods.

Notable Changes

Breaking Changes and Migration Steps

  • The existing ProductSpecification and corresponding resource-tier handler (on the CatalogServices side) are deprecated and disabled by default in favor of the new CompleteProductImportSpecification and handler. To continue using the deprecated components and disable the new components, set broadleaf.import.product.specification.use-legacy to true in both the ImportServices properties and CatalogServices properties. Furthermore, any existing PRODUCT imports and batches should be allowed to fully complete processing with the old handler (and no more batch requests should be in the Kafka message queue) before the new components are enabled - this way the new product batch handler will not be forced to consume messages intended for the old handler.

  • All out-of-box specifications extending DefaultSpecification (except the deprecated ProductSpecification, which is unchanged) have had their initializeFieldConfigMap() methods removed. Field configuration has been moved to the populateHeaderFieldConfigsByRowType() method, which extensions should migrate to. The configuration itself should be the same as before, with the only difference being that it is now nested under a specific row type value. For specifications that only have a single row type, this should match the 'main record type' (which itself defaults to 'import type' if not set). Even for client specifications that do not currently extend the out-of-box specifications, it is strongly recommended to move away from all deprecated fields - while there are backwards compatibility measures in place to support the deprecated fields, the migrations are easy enough to recommend moving everything at once.

  • Import table now has a new specification_ref column - it’s nullable and existing rows can just have it as null. This supports the new specificationRef field in Import, discussed later in these notes. The liquibase changesets will need to be run to ensure the new column is present.

  • Mark operation in BatchItemCompletion as Nullable, as operation type is not necessarily provided in the input file or generated by the import service anymore. If clients have custom logic that references this field, they must ensure null-safety is checked.

  • Incorrect metadata message keys import.fields.specification and import.fields.options.specification.csv have been removed and replaced with import.fields.file-type and import.fields.options.file-type.csv, and also updated in StartImportAction and DefaultImportFileTypes

  • ImportManager/DefaultImportManager no longer directly injects/keeps a List<ImportSpecification> and instead relies on the new ImportSpecificationService for providing specifications. The findSpecificationOrDefault() method has been removed in favor of this new component.

  • ImportEndpoint no longer directly injects/keeps a List<GlobalImportSpecification> and instead relies on ImportSpecificationService.

  • Basic plumbing has been introduced to support fetching specifications with context discrimination. For security purposes, the context info used to fetch the specification needs to be pre-validated before it is used. This means the importing context info has to be built and validated in an earlier phase than before, which resulted in breaking changes to the initiateImport flow in DefaultImportManager. At a high level, initializeContextFields no longer instantiates importing catalog ID, and that initialization is deferred until after the specification is fetched.

  • PriceDataExampleImportResolver no longer delegates to DefaultExampleImportResolver

  • The getRowWithAdditionalDataAdded() method in CSVErrorsReportService has a changed signature that now accepts an ImportSpecification for the purpose of honoring dynamic mapping configuration

  • Unused RecordTypes static class has been removed from OfferCodeSpecification. Should not affect most clients, since the specification itself is only available since 1.8.0, which clients have not yet started consuming.

General Changes

  • Introduce new ImportCompletedItemsGroup in metadata with separate successful and non-successful items grids, deprecating use of ImportDetailsGridExternal from ImportDetailsView and ImportDetailsFormView

  • Introduce a new optional specificationRef field to Import and ImportRequest, which is currently unused out-of-box, but provides an opportunity for extensions to implement behavior for requesting/finding specific specifications

  • ImportFieldConfig now supports a defaultValue, which will be set on the row by DefaultImportProcessor if the value for the field is null in the original input

  • Global variables in ImportFieldConfig are now explicitly marked as final

  • Update batching logic in DefaultImportProcessor such that dependents are now also counted against batch size limits instead of only counting main records. For example, let’s say the batch size limit is 2 and the input contains mainRecordA with 3 dependents and mainRecordB with 3 dependents. Previously, only main records were counted against batch size, so the batching logic would have only counted 2 records and thus included all of those items in a single batch. Now, since dependents count, mainRecordA with its dependents (4 total records) will reach the batch size limit and will be sent in a separate batch from mainRecordB with its dependents. To be clear, batching will still always include all dependents of a main record in the same batch and will not break them across multiple batches.

  • Introduce new resolveRecordTypeBeforePropertyMapping() method on ImportSpecification to describe how record type should be resolved from a row before any property mapping occurs, and update DefaultImportProcessor to use it

  • Introduce new shouldAutoGenerateOperationTypeForEachRecord() and shouldAutoGenerateResourceTierIdForEachRecord() methods on ImportSpecification to describe whether the import service should eagerly resolve/generate operation types and resource tier IDs for rows, and update DefaultImportProcessor to use them. This is primarily intended to support newer, simpler imports which do not rely on resource tier ID, correlation ID, and operation type concepts.

  • Introduce new shouldAllowUnmappedHeaders() method on ImportSpecification to describe whether unmappable headers should be passed through to the resource tier service or pruned, and update DefaultImportProcessor to use it

  • Introduce new dynamic field mapping methods to ImportSpecification and DefaultSpecification, as well as a DynamicHeaderFieldMapping interface that specifications can leverage to easily add dynamic mapping logic as an alternative to explicit, hard-coded field mappings via ImportFieldConfig

  • Introduce a default TranslationDynamicHeaderFieldMapping that can be used by specifications to map translations following a specific syntax pattern

  • Deprecate ImportSpecification 's 'global' getFieldConfigMap() and getRequiredHeaders() methods in favor of new row-type-specific getRequiredHeaders and getHeaderFieldConfigsByRowType, and update DefaultSpecification to make populating the new data easy via a new populateHeaderFieldConfigsByRowType() method. DefaultImportProcessor has been updated to preserve backwards compatibility with the deprecated methods while honoring the new ones.

  • Introduce new getName() method on ImportSpecification which is used by default as the specification ref

  • In DefaultSpecification, update the isRequiresRowTypeHeader computation and the deprecated getRequiredHeaders computation to happen in a thread-safe manner

  • Update CSVErrorsReportService to support the new dynamic mapping concepts, as well as honor the new methods and fields on ImportSpecification when generating errors report files

  • Introduce new CompleteProductImportSpecification representing the new product import specification, along with a new CompleteProductExampleImportResolver for its example file

  • Deprecate the old ProductSpecification

  • ImportManager now exposes a public buildImportingContextInfo() method for use by other components such as CSVErrorsReportService

  • Update DefaultImportValidator to honor new import specification methods and fields

  • Introduce new BooleanNormalizer, BooleanValidator, EnumeratedValueValidator, and MoneyAmountValidator components for use by specifications

  • The GET /imports/specifications endpoint (defined in ImportEndpoint) now accepts and resolves a ContextInfo argument. This is intended to support the possibility of client customizations filtering specifications by context. The caller of the endpoint will be expected to provide a context matching the 'importing context' they want to target (instead of whatever arbitrary context they may be browsing or initiating the import from). This will ensure the specification filtration occurs on the right context. Existing API callers should largely be unaffected with the default implementation, which does not truly use the context argument for any filtration logic.


  • Fix a bug in ImportDetailsGridExternal configuration where pagination was broken due to numbered vs narrowed misconfiguration

  • Fix an issue where CSV parsing logic in CSVFileReader and CSVErrorsReportService failed on Windows and classic Mac line separators

  • Introduce SkuInventoryImportSpecification to fix an issue where INVENTORY imports did not require inventory-specific scopes/permissions. With the latest changes, INVENTORY imports now require the INVENTORY scope and ALL_INVENTORY permissions.