- name: tmp
{{- if .Values.tmpDirMedium }}
emptyDir:
medium: {{ .Values.tmpDirMedium -}}
{{ else }}
emptyDir: {}
{{- end }}
ImportantBase Helm Charts are only applicable to legacy MicroservicesDemo
-based projects. If you are using anInitializr
-based project, you can leverage thehelm:generate
plugin as described here
Broadleaf provides base charts that enable deployment of various resources needed to support the entire Broadleaf ecosystem on Kubernetes using the Helm package manager.
These Helm charts are designed to be a lightweight way to configure Broadleaf’s demo Docker images and can be used as a baseline and example for deploying your specific flex package composition, which can vary based on your client-specific implementation.
We recommend that the Helm chart version is aligned to the corresponding Broadleaf Release Train version that you want to deploy. This will ensure that you are using a chart version that has been tested against the corresponding release train version of all the applicable services in the stack. This will also ensure that the documentation and examples for the chart will work with the version of the release, you are installing.
For example, if you want to deploy the full Microservices 1.7.x Release Train
onto your cluster, use the corresponding 1.7.x
chart tag.
Generally you can assume that the Features/Notable Changes
and Bug Fixes
that are applied to prior minor versions are also applied to the appropriate later versions (i.e. a change noted in 1.7.5
will in most cases be applicable and also applied to 1.8.x +
etc…)
Updated charts to allow control of the medium
of the default tmp
directory that several services use. Previously, the settings for the tmp
directory were hard-coded and could not be customized. A new helm value, tmpDirMedium
, has been added that allows changing the medium of the volume. In order to disable the explicit medium
setting and instead use the Kubernetes default, set tmpDirMedium
to null
.
Template Changes:
- name: tmp
{{- if .Values.tmpDirMedium }}
emptyDir:
medium: {{ .Values.tmpDirMedium -}}
{{ else }}
emptyDir: {}
{{- end }}
Override Changes:
# Set the medium of the default tmp directory. Set to 'null' to use the default value
tmpDirMedium: "Memory"
Updated ZooKeeper, Solr, and Kafka to use storageClassName
instead of storageClass
in the statefulset volumeClaimTemplates
section. For ZooKeeper, also distinguishing between dataDir and dataLog for both storageClassName
and volumeMode
.
Template Changes:
{{- if .Values.persistence.dataDirStorageClassName }}
storageClassName: {{ .Values.persistence.dataDirStorageClassName }}
{{- end }}
volumeMode: {{ .Values.persistence.dataDirVolumeMode }}
{{- if .Values.persistence.dataLogStorageClassName }}
storageClassName: {{ .Values.persistence.dataLogStorageClassName }}
{{- end }}
volumeMode: {{ .Values.persistence.dataLogVolumeMode }}
Updated supporting services (zookeeper, solr, kafka) to support adding a storageClassName
. By default storageClassName
is not defined and defaults to the kubernetes default setting. Also added volumeMode defaulting to "Filesystem"
Template Change:
{{- if .Values.persistence.storageClassName }}
storageClass: {{ .Values.persistence.storageClassName }}
{{- end }}
volumeMode: {{ .Values.persistence.volumeMode }}
Override Changes
# Only override the storageClassName if you don't want to use the default
# storageClassName: ""
# Override the volumeMode if you don't want the default (Filesystem)
volumeMode: Filesystem
Updated flex-balanced-browse removing the hardcoded SPRING_SERVLET_MULTIPART_MAXFILESIZE
and SPRING_SERVLET_MULTIPART_MAXREQUESTSIZE
values and changed them to use variables
Template Change:
- name: SPRING_SERVLET_MULTIPART_MAXFILESIZE
value: {{ .Values.env.SPRING_SERVLET_MULTIPART_MAXFILESIZE | quote }}
- name: SPRING_SERVLET_MULTIPART_MAXREQUESTSIZE
value: {{ .Values.env.SPRING_SERVLET_MULTIPART_MAXREQUESTSIZE | quote }}
Overfide Change:
SPRING_SERVLET_MULTIPART_MAXFILESIZE: "30000000"
SPRING_SERVLET_MULTIPART_MAXREQUESTSIZE: "30000000"
Added the ability to add pod annotations
to the supporting pods (zookeeper, solr, kafka)
Template Change:
{{- if or .Values.podAnnotations .Values.prometheus.jmx.enabled }}
annotations:
{{- end -}}
{{- if .Values.podAnnotations -}}
{{ toYaml .Values.podAnnotations | nindent 8 }}
{{- end -}}
{{- if .Values.prometheus.jmx.enabled }}
prometheus.io/scrape: "true"
prometheus.io/port: {{ .Values.prometheus.jmx.port | quote }}
{{- end }}
Added the ability to include hostAliases
to the spec
block for all services and flex packages. This can be done by adding an hostAliases
section to your override file and adding the required key/value pairs.
hostAliases:
- ip: "127.0.0.1"
hostnames:
- "foo.local"
- "bar.local"
Added the ability to include extra volumes
and volumeMounts
to the auth
deployment. As with other services, this can be done by adding extraVolumes
and extraVolumeMounts
sections to your override file and adding the required key/value pairs.
extraVolumeMounts:
- mountPath: /extra_mount
name: extra-mount
extraVolumes:
- name: extra-mount
emptyDir: {}
Added the ability to include extra envFrom
to the pod spec.containers
block for all services and flex packages. This can be done by adding an extraEnvFrom
section to your override file and adding the required key/value pairs.
extraEnvFrom:
- configMapRef:
name: routes-env
This is to support release train 1.8.5-GA. The Search Service now does a Tenant look up to get required locale information. This Chart update adds in the necessary Search Client / Tenant Provider references into the base charts.
Key changes:
flex-balanced-supporting / flex-min-core
- name: SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_SEARCHCLIENT_TOKENURI value: $(BROADLEAF_GATEWAY_PROXYURLS_AUTH)/oauth/token
flex-granular-search
- name: SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_TENANTCLIENT_TOKENURI value: $(BROADLEAF_GATEWAY_PROXYURLS_AUTH)/oauth/token
Added the ability to include extra annotations to the pod spec.template.metadata.annotations
block. This can be done by adding an extraAnnotations
section to your override file and adding the required key/value pairs.
extraAnnotations:
annotation1: value1
annotation2: value2
Added support for Broadleaf’s Fulfillment
Microservice. More details can be found here (https://developer.broadleafcommerce.com/services/fulfillment-services)
Bumped the Admin Gateway
version from 1.4.7-GA
to 1.4.10-GA
Bumped the Commerce Gateway
version from 1.4.7-GA
to 1.4.10-GA
Updated charts with deprecated Kubernetes API definitions (to support K8 cluster versions of 1.25+). This included the following resources: solr: poddisruptionbudget
, zookeeper: poddisruptionbudget
, psp-default: podsecuritypolicy
, psp-permissive: podsecuritypolicy
. These resources previously defined the apiVersion: policy/v1beta1
and has now been updated to be apiVersion: policy/v1
Note that this release contains important version updates which will impact the following charts. If you do not wish to undergo this upgrade, be sure to override these versions for your specific implementation going forward.
solr
: repository.broadleafcommerce.com:5001/broadleaf/docker-solr
changed broadleaf solr version from: 1.5.2-GA
→ 1.7.0-GA
note: this advanced the base docker image from solr:8.3.1-slim
to solr:9.2.0
note: advancing to solr 9.2.0
required updates to all solrconfig.xml
files which will need to be re-"up-config" to Zookeeper
solr-exporter
: solr
changed solr version from: 8.2-slim
→ 9.2.0
added a check-zk
init container to validate connectivity to Zookeeper
added readiness and liveness probes
Added an elasticAPM
block to admingateway
, commercegateway
, and auth
. For these services, it adds a volume, volume mount, and initContainer
to load and run the Elastic APM Java agent. It also provided environment variables to do some needed configuration of the agent. The new helm values and their defaults are below. Note that imageVersion
is only used by Java services and serviceName
defaults to the appropriate service name.
elasticAPM:
enabled: false
imageVersion: 1.36.0
serviceName: <service>
logLevel: INFO
secret:
secretName: elastic-apm-server-credentials
serverUrlKey: elasticApmServerSchemeHostPort
secretTokenKey: elasticApmServerSecretToken
Whitelisted the 4WL (4 letter word) commands for Zookeeper. Both Solr monitoring of the Zookeeper status as well as Kubernetes health probes make use of these commands. The key change was adding a new Java "-D" runtime argument to the helm values.yaml
to enabled this feature:
heapOptions: "-Xms512M -Xmx512M -Dzookeeper.4lw.commands.whitelist=mntr,conf,ruok"
Added the ability to enable and define standard health probes. As with other probes, any valid probe definition can be added within a spec:
block. When enabled, defaults will be used unless specifically overridden. The following is the default probe definition. Note that the spec:
block should NOT define the probe type (eg. startupProbe
):
startupProbe:
enabled: false
spec:
exec:
command: ['/bin/bash', '-c', 'echo "ruok" | nc -w 2 -q 2 127.0.0.1 2181 | grep imok']
initialDelaySeconds: 15
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
livenessProbe:
enabled: false
spec:
exec:
command: ['/bin/bash', '-c', 'echo "ruok" | nc -w 2 -q 2 127.0.0.1 2181 | grep imok']
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
readinessProbe:
enabled: false
spec:
exec:
command: ['/bin/bash', '-c', 'echo "ruok" | nc -w 2 -q 2 127.0.0.1 2181 | grep imok']
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
Added an elasticAPM
block to all flex-*
charts, commerceweb
, and adminweb
. For the Java services, it adds a volume, volume mount, and initContainer
to load and run the Elastic APM Java agent. It also provided environment variables to do some needed configuration of the agent. For Node services, it only adds the environment variables and depends on the service image having the APM agent within it and set to run on application start. The new helm values and their defaults are below. Note that imageVersion
is only used by Java services and serviceName
defaults to the appropriate service name.
elasticAPM:
enabled: false
imageVersion: 1.36.0
serviceName: Asset
logLevel: INFO
secret:
secretName: elastic-apm-server-credentials
serverUrlKey: elasticApmServerSchemeHostPort
secretTokenKey: elasticApmServerSecretToken
Refactored startupProbe, readinessProbe, and livenessProbe to allow for complete definition of the specification of the probe. Any valid probe definition can be added within a spec:
block.
The following is an example of a specification for a given set of probes. The specification in this example uses httpGet
but any valid type can be specified. Note that the spec:
block should NOT define the probe type (eg. startupProbe
):
startupProbe:
enabled: false
spec:
httpGet:
path: /actuator/health/liveness
port: 10000
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 12
livenessProbe:
enabled: false
spec:
httpGet:
path: /actuator/health/liveness
port: 10000
initialDelaySeconds: 0
periodSeconds: 15
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 4
readinessProbe:
enabled: false
spec:
httpGet:
path: /actuator/health/readiness
port: 10000
initialDelaySeconds: 0
periodSeconds: 5
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 12
Added startupProbe, readinessProbe, and livenessProbe blocks to the primary application Chart for:
auth, admingateway, adminweb, commercegateway, commerceweb
flex-balanced-*, flex-granular-*
Probes have not been added to the min
flex package
The following defaults are provided for granular (standalone) services
startupProbe:
enabled: false
path: /actuator/health/liveness
port: 10000
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 12
livenessProbe:
enabled: false
path: /actuator/health/liveness
port: 10000
initialDelaySeconds: 0
periodSeconds: 15
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 4
readinessProbe:
enabled: false
path: /actuator/health/readiness
port: 10000
initialDelaySeconds: 0
periodSeconds: 5
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 12
The following defaults are provided for balanced services. The startupProbe.initialDelaySeconds
and the extended failureThreshold
provides support for the balanced flex package startup dependencies.
startupProbe:
enabled: false
path: /actuator/health/liveness
port: 10000
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 60
livenessProbe:
enabled: false
path: /actuator/health/liveness
port: 10000
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 6
readinessProbe:
enabled: false
path: /actuator/health/readiness
port: 10000
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 6
Set the "fsGroup": 1000
property in the zookeeper
chart as the default image defaults to a user and group 1000
(#81)
Fix the BROADLEAF_AUTH_SECURITY_PKCE_ENABLED
ENV variable in the auth
chart as it was incorrectly gated by the .Values.seedDataLoad.enabled
property (#87)
Move the deploymentStrategy
value from the pod specification block to the deployment specification block. Add deploymentStrategy
to admingateway
, adminweb
, auth
, commercegateway
, and commerceweb
(#88)
Removed the previously required BROADLEAF_TENANT_LIQUIBASE_CHANGELOG
ENV property in the flex-balanced-supporting
, flex-granular-tenant
, and flex-min-core
charts. Going forward this should be specified in your extraEnvs
if applicable for your implementation (#62)
Fix the flex-granular-content
chart to apply default chart configurability options: i.e. podSecurityContext
and jaegerAgentSidecar
etc…
Ability to override default Chart name by specifying appContainerName
- applies to most charts (#66)
Enable PKCE by default on the adminweb
chart. This introduces the following ENV properties: VITE_ENABLE_DEV_SETTINGS
and VITE_ENABLE_PKCE
(#73)
Enable PKCE by default on the auth
chart. This introduces the following ENV property: BROADLEAF_AUTH_SECURITY_PKCE_ENABLED
which defaults to true. (#73)
Ability to optionally enable/disable the SPRING_DATASOURCE_URL
and SPRING_DATASOURCE_USERNAME
ENV properties in the auth
chart using the property datasource.enabled
Add support for SSL database connections using the property: db.sslCertificates.enabled
and db.sslCertificates.secretName
which mounts the following default path: /etc/db-ssl-certs
. This applies to all charts with DB connectivity (e.g. auth
and all flex-*
charts). (#58)
Make the following ENV properties optional: BROADLEAF_COMPOSITE_DATASOURCE_URL
, BROADLEAF_COMPOSITE_DATASOURCE_USERNAME
, and BROADLEAF_COMPOSITE_DATASOURCE_PASSWORD
- this applies to all flex-*
charts using the property: compositedatasource.enabled
which is defaulted to true
Optionally enable a debug port using the debug.enabled
and debug.port
properties. Debug is set to false
by default and is applicable to the flex-*
charts.
The introduction of the following ENV properties: BROADLEAF_CARTOPERATION_CREDITACCOUNTPROVIDER_URL
, BROADLEAF_STORE_CREDIT_CREDITACCOUNTPROVIDER_URL
, BROADLEAF_CARTOPERATION_CUSTOMERPROVIDER_URL
, and SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_STORECREDITCLIENT_TOKENURI
in the flex-balanced-cart
chart (CreditAccountServices#9)
The introduction of the following ENV properties: BROADLEAF_CARTOPERATION_CREDITACCOUNTPROVIDER_URL
and BROADLEAF_CARTOPERATION_CUSTOMERPROVIDER_URL
in the flex-granular-cartops
chart (CreditAccountServices#9)
The introduction of the following ENV properties: SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_STORECREDITCLIENT_TOKENURI
and BROADLEAF_STORE_CREDIT_CREDITACCOUNTPROVIDER_URL
in the flex-granular-paymenttransaction
chart (CreditAccountServices#9)
The introduction of the following ENV properties: BROADLEAF_CARTOPERATION_CREDITACCOUNTPROVIDER_URL
, BROADLEAF_STORE_CREDIT_CREDITACCOUNTPROVIDER_URL
, BROADLEAF_CARTOPERATION_CUSTOMERPROVIDER_URL
and SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_STORECREDITCLIENT_TOKENURI
to the flex-min-core
chart (CreditAccountServices#9)
Introduce the flex-granular-creditaccount
chart (CreditAccountServices#9)
Adding the default BROADLEAF_GATEWAY_PROXYURLS_CREDITACCOUNT
value to flex-balanced-routes-env
, flex-granular-routes-env
, and flex-min-routes-env
charts (CreditAccountServices#9)
Adding an optional block where users can specify a "strategy" for the deployment using the deploymentStrategy
property. Applies to all flex-*
charts. (MPM#3509)
Enhanced the Zookeeper cart to enable 4WL commands and added standard health probes. See Version 1.8.4 Release Notes for details for this feature.
Refactored startupProbe, readinessProbe, and livenessProbe to allow for complete definition of the specification of the probe. Any valid probe definition can be added within a spec:
block. See Version 1.8.3 Release Notes for details for this feature.
Added startupProbe, readinessProbe, and livenessProbe blocks to the primary application Chart - See Version 1.8.2 Release Notes for details for this feature.
Added an elasticAPM
block to all flex-*
charts, commerceweb
, and adminweb
. For the Java services, it adds a volume, volume mount, and initContainer
to load and run the Elastic APM Java agent. It also provided environment variables to do some needed configuration of the agent. For Node services, it only adds the environment variables and depends on the service image having the APM agent within it and set to run on application start. The new helm values and their defaults are below. Note that imageVersion
is only used by Java services and serviceName
defaults to the appropriate service name.
elasticAPM:
enabled: false
imageVersion: 1.36.0
serviceName: Asset
logLevel: INFO
secret:
secretName: elastic-apm-server-credentials
serverUrlKey: elasticApmServerSchemeHostPort
secretTokenKey: elasticApmServerSecretToken
Update MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE
to include the health
actuator endpoint. This impacts the flex-balanced-common-env
, flex-granular-common-env
, and flex-min-common-env
Config Map charts. (MPM#3544)
Remove BROADLEAFDEMO_ACTUATOR_ANONYMOUS
from being explicitly defined in the base Config Map charts - impacts flex-balanced-common-env
, flex-granular-common-env
, and flex-min-common-env
. Going forward, you can specify your implementation-specific property override in your umbrella chart or specifically in your application. (#85)
Ability to override resources definitions (e.g. cpu
and memory
requests and limits) on several supporting service charts: job-zk-upconfig
, kafka
, openapi-ui
, solr
, zookeeper
, job-solr-create-collections
(#54)
Ability to override resources definitions (e.g. cpu
and memory
requests and limits) for kafka - checkZK (initContainer)
and zookeeper - jmx (init container)
Updated the default confluentinc/cp-zookeeper
image tag from 5.5.0
to 7.2.1
Updated the default confluentinc/cp-kafka
image tag from 7.0.0
to 7.2.1
Move the BROADLEAF_ADMINUSER_AUTHPROPERTIES_AUTHURI
ENV property out of the .Values.seedDataLoad.enabled
check - impacts flex-balanced-supporting
and flex-granular-adminuser
charts (#43)
Update the BROADLEAF_CARTOPERATION_CATALOGPROVIDER_URL
ENV property default value to be BROADLEAF_GATEWAY_PROXYURLS_CATALOGBROWSE
- impacts flex-min-core
, flex-granular-cartops
, and flex-balanced-cart
charts (#50)
Broadleaf’s Zookeeper and Kafka Helm charts have been updated to use version 7.2.1
of Confluent’s images.
The default Zookeeper image user has changed from root to UID 1000, and this requires a migration step for file permissions on existing Zookeeper volumes.
A DevOps engineer will have to do the following to migrate an existing deployment from the prior version to the new version.
On the existing (prior-version) deployment, temporarily eliminate the securityContext
setup on Kafka and Zookeeper.
This is necessary to allow chown
commands to work.
For example, this means setting the following values in blc-supporting-stack
:
zookeeper:
podSecurityContextCustom:
enabled: true
value: {
"securityContext": {}
}
containerSecurityContextCustom:
enabled: true
value: {
"securityContext": {}
}
# Only necessary if Kafka is on a version prior to 7.0.0 (unlikely)
kafka:
podSecurityContextCustom:
enabled: true
value: {
"securityContext": {}
}
containerSecurityContextCustom:
enabled: true
value: {
"securityContext": {}
}
Note
|
If the cluster has Pod Security Policies enabled, then pods without this securityContext may be rejected for violating those policies.
You may need to temporarily disable them on your cluster before applying this change.
|
Apply the updates to the blc-supporting-stack
chart by running a Helm upgrade
Now, you should be able to change ownership/permissions of files in each of the affected volumes. The containers will still be running as root at this point.
Use kubectl
or an equivalent tool to get a shell terminal open on each of the zookeeper-*
pods and run chown -R 1000:0 /var/lib/zookeeper/data && chown -R 1000:0 /var/lib/zookeeper/log
(only if Kafka is currently on a version prior to 7.0.0 and is currently running as root, which is unlikely) Use kubectl
or an equivalent tool to get a shell terminal open on each of the kafkacluster-*
pods and run chown -R 1000:0 /opt/kafka/data
Now that you have updated the file permissions, you can start the process of moving to the new versions of Zookeeper and Kafka charts.
Go back to your Helm chart configuration and undo the pod and container security context value overrides you wrote in the previous step
Update the references in blc-supporting-stack
to the new 2.0.0
charts which will bring in the new Kafka/Zookeeper images
Run a Helm upgrade on the blc-supporting-stack
chart to force all pods to be recreated with the new versions and have the security context settings re-applied.
Start up should succeed with this change.
You can now shell into the Zookeeper pods again and examine the /var/lib/zookeeper
contents to confirm the mounted volume ownership and permissions are correct:
[appuser@zookeeper-0 zookeeper]$ ls -lah
total 16K
drwxr-x--- 4 appuser root 4.0K Jul 15 00:50 .
drwxr-xr-x 1 root root 4.0K Jul 15 00:50 ..
drwxrwsr-x 4 root appuser 4.0K Jan 11 17:29 data
drwxrwsr-x 4 root appuser 4.0K Jan 11 17:29 log
[appuser@zookeeper-0 zookeeper]$ cd data/
[appuser@zookeeper-0 data]$ ls -lah
total 32K
drwxrwsr-x 4 root appuser 4.0K Jan 11 17:29 .
drwxr-x--- 4 appuser root 4.0K Jul 15 00:50 ..
drwxrws--- 2 root appuser 16K Jan 11 17:29 lost+found
-rw-r--r-- 1 appuser appuser 1 Jan 11 17:29 myid
drwxr-sr-x 2 appuser appuser 4.0K Jan 11 17:29 version-2
Ensure to re-enable your Pod Security Policy configuration as well if you disabled it in a previous step.
Add ContentServices
to support the 1.7.2 RT (MPM#2679)
Update default docker image tag for adminweb
(MPM#2819)
Remove default user security contexts in the admingateway
and commercegateway
charts to match the defaults in the flex-package
charts
Remove the cors
spring active profile as a default for the commercegateway
(#40)
Update the default tag for admingateway
and commercegateway
Allow overriding of default busybox and curl image for the check-zk and check-solr initContainer (#23)
Allow Security Context Overrides. If you would like to override the default podSecurityContext
or the containerSecurityContext
, you can define the respective overrides: podSecurityContextCustom
, and containerSecurityContextCustom
(#27)
Ability to override image
, imageTag
, and imagePullSecrets
for solr-exporter
(#32)
Ability to specify additional annotations for the blc-assets-pvc
resource (#32)
Ability to override imagePullSecretes
for kafka
and zookeeper
(#36)
Ability to override backOffLimit
for job-zk-upconfig
and job-solr-create-collections
(#36)
Ability to specify podAnnotations
for the flex-balanced-processing
and flex-balanced-supporting
charts (#38)