Broadleaf Microservices
  • v1.0.0-latest-prod

Broadleaf Audit Services

since 2.1.4|2.2.0

This feature is still in Beta. Functionality and documentation are subject to change. Audit events will be more deeply integrated into the framework in future releases, but the functionality is available for review now - allowing the creation of custom audit events.

Overview

Audit Services manages and stores audit events from across the system. This includes key domain mutations, as well as key application lifecycle events that benefit from a forensic history store.

Key Concepts

In general, there are three types of audit events created and stored in the system:

  1. Audit event without detail. This is an event that only contains header information regarding time, description, user, etc…​ This type of event is useful for simple application lifecycle demarcation without requiring extensive informational details.

  2. Audit event with raw detail. This is similar to the event without detail, but includes a collection of detailed information that is unstructured and may be represented in whatever format the caller desires.

  3. Audit event with structured field detail. This is similar to the event with raw detail, but the detail objects adhere to a structured format of before/after field values for a specific entity mutation.

See Advanced Audit for details on using the audit features.

Requirements

  • The audit client library must be available on the classpath to initiate audit events. Whether you are using a Broadleaf starter-based project, the framework component override module in your project should include a maven dependency on the client library. A version should not be required and should be auto-resolved via the inherited common dependencies BOM.

<dependency>
    <groupId>com.broadleafcommerce.microservices</groupId>
    <artifactId>broadleaf-common-audit</artifactId>
</dependency>

  • A shared filesystem must be available and visible to all applicable microservices. When running locally, this is defaulted to a directory under the java temp directory. In higher environments (such as Kubernetes), this should be a persistent volume with a ReadWriteMany access mode. If not using the default temp directory locations (local dev), these Spring environment properties are available for specifying the location of the shared filesystem:

    • broadleaf.audit.processing.storage.filesystem.working-directory

    • broadleaf.audit.processing.storage.filesystem.completion-directory

    • broadleaf.audit.processing.storage.filesystem.ingestion-directory

  • The audit feature must be enabled. This can be done with a single property: broadleaf.basic.audit.group set to true. This property is a group property, so it will enable or disable both of the AuditProcessingProperties.enabled and FileSystemStorageProviderProperties.enabled properties together.

  • Audit Services must be running as part of the stack. This is generally achieved by updating the Broadleaf starter manifest to include the audit service under the flexUnits section of the flexPackage.

    • Remember to run mvn flex:generate after updating the manifest to ensure the new service is included in the generated flex packages, as well as including relevant changelogs. After this, verify that the broadleaf-audit-services dependency is added to the relevant flex package and data directory POMs.

components:
- name: audit
  routed: true
  domain:
    cloud: audit
    docker: audit
    local: localhost
  enabled: false
  ports:
    - port: 8489
      targetPort: 8489
    - debug: true
      port: 8089
      targetPort: 8089
flexPackages:
- name: supporting
  domain:
    cloud: supporting
    docker: supporting
    local: localhost
  enabled: false
  flexUnits: adminnav,adminuser,metadata,notification,sandbox,search,tenant,audit
You may choose a different flexpackage than the example, or configure a new granular flexpackage to contain the audit service.

Components

The following components are used to support the audit services:

  • ChangeAuditHandler - responsible for catching MutationNotifyEvent instances for Trackable entity changes and converting to AuditEvent instances via registered converters.

  • KeyEventAuditHandler - responsible for catching AuditRequestEvent instances (generally created via AuditEventUtility) and converting to AuditEvent instances.

  • AuditEventRecorder - responsible for catching AuditEvent instances as an async operation and storing as batches via a registered StorageProvider (FileSystemStorageProvider by default).

  • IngestionProcessor - responsible for reading batches of AuditEvent instances from the StorageProvider (FileSystemIngestionProcessor by default) and storing in the database via AuditHeaderRepository.

  • DefaultAuditPruneService - responsible for pruning the database of old audit events based on a configurable retention period.

Audit Domain Concepts

Audit Header Domain

This is the primary domain for using and understanding audit records. It represents the overall change event, and acts as the parent for additional details represented by the AuditDetail domain as needed. It supports the builder pattern by using AuditEventHeader.builder().

Key properties

  • eventType: In the case of a Trackable entity change, the name of the top-level entity that changed, which is typically snake-cased and capitalized, i.e., PRODUCT, USER_CREATE, LOGIN.

    • Required.

    • Arbitrary audit events may use this field to represent the type of event.

    • Typically used in conjunction with entityRef to look up headers.

  • entityType: Fully qualified class name of the top-level entity being changed. Optional. For audit changes to a particular entity.

  • entityRef: Identifying reference to the entity. Usually a contextId for Trackable domain.

    • Optional. For audit changes to a particular entity.

  • containerEventType: In the case of a Trackable entity change, the name of the containing entity, which is typically snake-cased and capitalized, i.e., PRODUCT.

    • This is a grouping concept for correlating multiple related changes to a singular concept. For example, a marketing message change, which from the admin perspective looks like a change to an offer.

    • Optional.

  • containerRef: Identifying reference to the containing entity. Usually a contextId for Trackable domain.

    • Optional.

    • See containerEventType.

  • detailType: The type of detail information associated with this audit event.

    • Default is DetailType.NONE

    • Could also be:

      • CHANGE_DETAIL: The AdminAuditEventDetail records are harvested from ChangeDetail instances taken from the admin CRUD flow. This is related to Broadleaf’s sandboxing.

      • RAW: The change information is not necessarily coming from an admin flow and the data is not harvested from ChangeDetail. Instead, the data is represented as a string or JSON.

  • additionalAttributes: Any additional information, possibly required in extension scenarios.

    • Optional.

    • Can be used in lieu of AuditDetails depending on the use case.

Audit Detail Domain

Represents the field-by-field detail information about changes to an entity. See AuditHeader. Generally only used when AuditHeader#detailType is DetailType.CHANGE_DETAIL. Also supports the builder pattern by using AdminAuditEventDetail.builder().

Key properties

  • beforeRepresentation: The value for the field before the change was made. Json representation. May be simple, or a more complex structure - like a list.

  • afterRepresentation: The value for the field after the change was made. Json representation. May be simple, or a more complex structure - like a list.

  • changeType: The operation type that mutated this field value. Maps to com.broadleafcommerce.data.tracking.core.type.OperationType (CREATE, READ, UPDATE, DELETE).

Trackable Domain Change Audit

Trackable JPA domain represents a basic concept in Broadleaf that covers admin maintenance and sandboxing. As part of this architecture, at a field-level, the system is aware of what changed, including before/after values. During Trackable maintenance, the system can be configured to audit these field-level changes.

Usage

To audit Trackable domain changes, the following steps are required:

  • Declare one or more Trackable domain types in Spring environment configuration at: broadleaf.audit.processing.default-mutation-notify-event-types.[subject].[list of fully qualified class names]. This alone is enough to engage Tracking-based audits for targeted entities with default field granularity.

  • If more fine-grained control is required over what is audited, and when - a converter may be used instead of the property-driven approach to change a MutationNotifyEvent coming from the Tracking architecture into an AuditEvent.

Example Spring Environment Configuration
broadleaf:
  audit:
    processing:
      default-mutation-notify-event-types:
        arbitraryName:
          - com.broadleafcommerce.catalog.provider.jpa.domain.product.JpaProduct
          - com.broadleafcommerce.catalog.provider.jpa.domain.category.JpaCategory
Example Converter
class TestConverter extends DefaultAbstractMutationNotifyEventConverter {

    public TestConverter(TypeFactory factory) {
        super(factory);
    }

    @Override
    protected boolean isValid(MutationNotifyEvent source, Trackable changed) {
        // You should qualify the type of entity you want to audit
        return super.isValid(source, changed) && changed instanceof JpaTestItem;
    }

    @Override
    protected boolean isValid(MutationNotifyEvent source, ChangeDetail detail) {
        // You have the option to filter out specific fields from being audited (i.e. ChangeDetail)
        return super.isValid(source, detail);
    }
}

  • The converter should be registered as a Spring bean.

@Bean
public Converter<MutationNotifyEvent, AuditEvent> testConverter(TypeFactory factory) {
    return new TestConverter(factory);
}
By default, DefaultAbstractMutationNotifyEventConverter qualifies validity based on the status of the change - specifically checking if the change is to the production level of the entity. This avoids tracking sandbox level changes.

Application Lifecycle Audit

Audit events may also apply to key application lifecycle events. These need not be tied to entity mutation like the structured events described above. Support for this type of arbitrary event may be inserted programmatically at any spot having access to the audit client library API.

Usage

To audit lifecycle events, the following steps are required:

  • Use the AuditEventUtility to construct an AuditEvent and fire the event.

AuditEvent start = AuditEvent.builder().header(AuditEventHeader.builder()
        .eventType("TEST")
        .timestamp(Instant.now())
        .tenantId("test")
        .detailType(DetailType.RAW.name())
        .build())
        .details(List.of(RawAuditEventDetail.builder()
                .author("admin")
                .rawDetail("A test")
                .timestamp(Instant.now())
                .build()))
        .build();
AuditEventUtility.fire(start);

  • A usage variation can exclude the detail if not needed.

AuditEvent start = AuditEvent.builder().header(AuditEventHeader.builder()
        .eventType("TEST")
        .timestamp(Instant.now())
        .tenantId("test")
        .detailType(DetailType.NONE.name())
        .build())
        .details(Collections.emptyList())
        .build();
AuditEventUtility.fire(start);
Refer to the javadocs for AuditEventHeader and RawAuditEventDetail for more information on the fields available for use.

Custom Audit

The Audit Common library provides access to an AuditEventUtility for resource tier services. This utility allows a service to fairly easily create an AuditHeader with details and the trigger persistence.

Here is an example usage to record a login attempt by a user:

// build the event
AuditEvent event = AuditEvent.builder()
        .header(AuditEventUtility.prepareHeader(ref -> ref
                .entityType(User.class.getName())
                .entityRef(user.getId())
                .eventType("LOGIN")
                .detailType(DetailType.NONE.name()), null))
        .details(Collections.emptyList())
        .build();
// publish the event
AuditEventUtility.fire(event);

The following property should also be set to enable this type of custom audit tracking:

broadleaf:
  basic:
    audit:
      group: true

Storage

When using the default FileSystemStorageProvider, a shared filesystem must be available and visible to all applicable microservices. When running locally, this is defaulted to a directory under the java temp directory. In higher environments (such as Kubernetes), this should be a persistent volume with a ReadWriteMany access mode. If not using the default temp directory locations (local dev), these Spring environment properties are available for specifying the location of the shared filesystem:

  • broadleaf.audit.processing.storage.filesystem.working-directory

  • broadleaf.audit.processing.storage.filesystem.completion-directory

  • broadleaf.audit.processing.storage.filesystem.ingestion-directory

Retrieving Audit Records

Audit APIs

See OpenAPI specs here

Audit Service provides an AuditEndpoint for retrieving AuditHeaders and AuditDetails.

  • Read Headers

    • Read all: GET /audit. Paginated.

    • Read all by Customer: GET /audit?customerId=:id. Paginated.

    • Read all by Account: GET /audit?accountId=:id. Paginated.

  • Read Details

    • Read all for Header: GET /audit/:headerId. Paginated.

    • Read all for Header and Customer: GET /audit/:headerId?customerId=:id. Paginated.

    • Read all for Header and Account: GET /audit/:headerId?accountId=:id. Paginated.

Using Audit in the Admin

At the time of writing, there is only limited support for reviewing audit information in the admin tool. For example, there is currently no centralized view for viewing all audits. However, in the admin metadata for Trackable grids, the optional .auditable('ENTITY_TYPE') may be included in the setup to cause audit information to be included in views for that entity type. The ENTITY_TYPE should be replaced with the actual entity type, which should match the AuditEventHeader#entityType value.