Upgrade fluentd to use new upstream helm and image
This commit is contained in:
parent
b0d0bc61b4
commit
54fad23983
|
|
@ -23,7 +23,7 @@ dependencies:
|
|||
repository: https://helm.elastic.co
|
||||
condition: eck-operator.enabled
|
||||
- name: fluentd
|
||||
version: 2.5.3
|
||||
version: 0.2.2
|
||||
condition: fluentd.enabled
|
||||
- name: fluent-bit
|
||||
version: 0.12.3
|
||||
|
|
|
|||
|
|
@ -14,8 +14,10 @@
|
|||
*.swp
|
||||
*.bak
|
||||
*.tmp
|
||||
*.orig
|
||||
*~
|
||||
# Various IDEs
|
||||
.project
|
||||
.idea/
|
||||
*.tmproj
|
||||
.vscode/
|
||||
|
|
|
|||
|
|
@ -1,12 +1,13 @@
|
|||
apiVersion: v1
|
||||
description: DEPRECATED A Fluentd Elasticsearch Helm chart for Kubernetes.
|
||||
icon: https://raw.githubusercontent.com/fluent/fluentd-docs/master/public/logo/Fluentd_square.png
|
||||
name: fluentd
|
||||
version: 2.5.3
|
||||
appVersion: v2.4.0
|
||||
apiVersion: v2
|
||||
appVersion: v1.12.0
|
||||
description: A Helm chart for Kubernetes
|
||||
home: https://www.fluentd.org/
|
||||
icon: https://www.fluentd.org/assets/img/miscellany/fluentd-logo_2x.png
|
||||
maintainers:
|
||||
- email: eduardo@treasure-data.com
|
||||
name: edsiper
|
||||
name: fluentd
|
||||
sources:
|
||||
- https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image
|
||||
- https://quay.io/repository/coreos/fluentd-kubernetes
|
||||
- https://github.com/coreos/fluentd-kubernetes-daemonset
|
||||
- https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
|
||||
- https://github.com/fluent/fluentd/
|
||||
- https://github.com/fluent/fluentd-kubernetes-daemonset
|
||||
version: 0.2.2
|
||||
|
|
|
|||
|
|
@ -1,121 +1,176 @@
|
|||
# ⚠️ Repo Archive Notice
|
||||
# Fluentd Helm Chart
|
||||
|
||||
As of Nov 13, 2020, charts in this repo will no longer be updated.
|
||||
For more information, see the Helm Charts [Deprecation and Archive Notice](https://github.com/helm/charts#%EF%B8%8F-deprecation-and-archive-notice), and [Update](https://helm.sh/blog/charts-repo-deprecation/).
|
||||
[Fluentd](https://www.fluentd.org/) is an open source data collector for unified logging layer. Fluentd allows you to unify data collection and consumption for a better use and understanding of data.
|
||||
|
||||
# fluentd
|
||||
## Installation
|
||||
|
||||
[Fluentd](https://www.fluentd.org/) collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure (Learn more about the Unified Logging Layer).
|
||||
To add the `fluent` helm repo, run:
|
||||
|
||||
## DEPRECATION NOTICE
|
||||
|
||||
This chart is deprecated and no longer supported.
|
||||
|
||||
## TL;DR;
|
||||
|
||||
```console
|
||||
$ helm install stable/fluentd
|
||||
```sh
|
||||
helm repo add fluent https://fluent.github.io/helm-charts
|
||||
helm repo update
|
||||
```
|
||||
|
||||
## Introduction
|
||||
To install a release named `fluentd`, run:
|
||||
|
||||
This chart bootstraps an fluentd deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
|
||||
|
||||
## Installing the Chart
|
||||
|
||||
To install the chart with the release name `my-release`:
|
||||
|
||||
```console
|
||||
$ helm install stable/fluentd --name my-release
|
||||
```sh
|
||||
helm install fluentd fluent/fluentd
|
||||
```
|
||||
|
||||
The command deploys fluentd on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
|
||||
## Chart Values
|
||||
|
||||
## Uninstalling the Chart
|
||||
|
||||
To uninstall/delete the `my-release` deployment:
|
||||
|
||||
```console
|
||||
$ helm delete my-release
|
||||
```sh
|
||||
helm show values fluent/fluentd
|
||||
```
|
||||
|
||||
The command removes all the Kubernetes components associated with the chart and deletes the release.
|
||||
## Value Details
|
||||
|
||||
## Autoscaling
|
||||
### default-volumes
|
||||
|
||||
By enabling autoscaling the chart will use statefulset with hpa instead of deployment with PVC.
|
||||
Please be noted to [statefulset limitation](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#limitations)
|
||||
The autoscaling is disabled by default for backward compatibility
|
||||
The default configurations bellow are required for the fluentd pod to be able to read the hosts container logs. The second section is responsible for allowing the user to load the "extra" configMaps either defined by the `fileConfigs` contained objects or, in addition, loaded externally and indicated by `configMapConfigs`.
|
||||
|
||||
## Configuration
|
||||
|
||||
The following table lists the configurable parameters of the fluentd chart and their default values.
|
||||
|
||||
Parameter | Description | Default
|
||||
--- | --- | ---
|
||||
`useStatefulSet` | Deploy as a StatefulSet regardless of whether autoscaling is enabled | `nil`
|
||||
`affinity` | node/pod affinities | `{}`
|
||||
`configMaps` | Fluentd configuration | See [values.yaml](values.yaml)
|
||||
`output.host` | output host | `elasticsearch-client.default.svc.cluster.local`
|
||||
`output.port` | output port | `9200`
|
||||
`output.scheme` | output scheme | `http`
|
||||
`output.sslVersion` | output ssl version | `TLSv1`
|
||||
`output.buffer_chunk_limit` | output buffer chunk limit | `2M`
|
||||
`output.buffer_queue_limit` | output buffer queue limit | `8`
|
||||
`deployment.labels` | Additional labels for pods | `{}`
|
||||
`image.pullPolicy` | Image pull policy | `IfNotPresent`
|
||||
`image.repository` | Image repository | `gcr.io/google-containers/fluentd-elasticsearch`
|
||||
`image.tag` | Image tag | `v2.4.0`
|
||||
`imagePullSecrets` | Specify image pull secrets | `nil` (does not add image pull secrets to deployed pods)
|
||||
`extraEnvVars` | Adds additional environment variables to the deployment (in yaml syntax) | `{}` See [values.yaml](values.yaml)
|
||||
`extraVolumeMounts` | Mount extra volumes (in yaml syntax) | `` See [values.yaml](values.yaml)
|
||||
`extraVolumes` | Extra volumes (in yaml syntax) | `` See [values.yaml](values.yaml)
|
||||
`ingress.enabled` | enable ingress | `false`
|
||||
`ingress.labels` | list of labels for the ingress rule | See [values.yaml](values.yaml)
|
||||
`ingress.annotations` | list of annotations for the ingress rule | `kubernetes.io/ingress.class: nginx` See [values.yaml](values.yaml)
|
||||
`ingress.hosts` | host definition for ingress | See [values.yaml](values.yaml)
|
||||
`ingress.tls` | tls rules for ingress | See [values.yaml](values.yaml)
|
||||
`nodeSelector` | node labels for pod assignment | `{}`
|
||||
`replicaCount` | desired number of pods | `1` ???
|
||||
`resources` | pod resource requests & limits | `{}`
|
||||
`plugins.enabled` | Enable Plugins Installation | `false`
|
||||
`plugins.pluginsList` | List of plugins to install | `[]`
|
||||
`rbac.create` | Specifies whether RBAC resources should be created | `true`
|
||||
`serviceAccount.create` | Specifies whether a service account should be created. | `true`
|
||||
`serviceAccount.name` | Name of the service account.
|
||||
`priorityClassName` | priorityClassName | `nil`
|
||||
`service.loadBalancerIP` | If `service.type` is `LoadBalancer` set custom IP load balancer IP address | `nil`
|
||||
`service.ports` | port definition for the service | See [values.yaml](values.yaml)
|
||||
`service.type` | type of service | `ClusterIP`
|
||||
`service.annotations` | list of annotations for the service | `{}`
|
||||
`tolerations` | List of node taints to tolerate | `[]`
|
||||
`persistence.enabled` | Enable buffer persistence | `false`
|
||||
`persistence.accessMode` | Access mode for buffer persistence | `ReadWriteOnce`
|
||||
`persistence.size` | Volume size for buffer persistence | `10Gi`
|
||||
`autoscaling.enabled` | Set this to `true` to enable autoscaling | `false`
|
||||
`autoscaling.minReplicas` | Set minimum number of replicas | `2`
|
||||
`autoscaling.maxReplicas` | Set maximum number of replicas | `5`
|
||||
`autoscaling.metrics` | metrics used for autoscaling | See [values.yaml](values.yaml)
|
||||
`terminationGracePeriodSeconds` | Optional duration in seconds the pod needs to terminate gracefully | `30`
|
||||
`metrics.enabled` | Set this to `true` to enable Prometheus metrics HTTP endpoint | `false`
|
||||
`metrics.service.port` | Prometheus metrics HTTP endpoint port | `24231`
|
||||
`metrics.serviceMonitor.enabled` | Set this to `true` to create ServiceMonitor for Prometheus operator | `false`
|
||||
`metrics.serviceMonitor.additionalLabels` | Additional labels that can be used so ServiceMonitor will be discovered by Prometheus | `{}`
|
||||
`metrics.serviceMonitor.namespace` | Optional namespace in which to create ServiceMonitor | `nil`
|
||||
`metrics.serviceMonitor.interval` | Scrape interval. If not set, the Prometheus default scrape interval is used | `nil`
|
||||
`metrics.serviceMonitor.scrapeTimeout` | Scrape timeout. If not set, the Prometheus default scrape timeout is used | `nil`
|
||||
|
||||
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
|
||||
|
||||
```console
|
||||
$ helm install stable/fluentd --name my-release \
|
||||
--set=image.tag=v0.0.2,resources.limits.cpu=200m
|
||||
```yaml
|
||||
- name: varlog
|
||||
hostPath:
|
||||
path: /var/log
|
||||
- name: varlibdockercontainers
|
||||
hostPath:
|
||||
path: /var/lib/docker/containers
|
||||
---
|
||||
- name: etcfluentd-main
|
||||
configMap:
|
||||
name: fluentd-main
|
||||
defaultMode: 0777
|
||||
- name: etcfluentd-config
|
||||
configMap:
|
||||
name: fluentd-config
|
||||
defaultMode: 0777
|
||||
```
|
||||
|
||||
Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example,
|
||||
### default-volumeMounts
|
||||
|
||||
```console
|
||||
$ helm install stable/fluentd --name my-release -f values.yaml
|
||||
The default configurations bellow are required for the fluentd pod to be able to read the hosts container logs. They should not be removed unless for some reason your container logs are accessible through a different path
|
||||
|
||||
```yaml
|
||||
- name: varlog
|
||||
mountPath: /var/log
|
||||
- name: varlibdockercontainers
|
||||
mountPath: /var/lib/docker/containers
|
||||
readOnly: true
|
||||
```
|
||||
|
||||
> **Tip**: You can use the default [values.yaml](values.yaml)
|
||||
The section bellow is responsible for allowing the user to load the "extra" configMaps either defined by the `fileConfigs` contained objects or otherwise load externally and indicated by `configMapConfigs`.
|
||||
|
||||
```yaml
|
||||
- name: etcfluentd-main
|
||||
mountPath: /etc/fluent
|
||||
- name: etcfluentd-config
|
||||
mountPath: /etc/fluent/config.d/
|
||||
```
|
||||
|
||||
### default-fluentdConfig
|
||||
|
||||
The `fileConfigs` section is organized by sources -> filters -> destinations. Flow control must be configured using fluentd routing with tags or labels to guarantee that the configurations are executed as intended. Alternatively you can use numeration on your files to control the configurations loading order.
|
||||
|
||||
```yaml
|
||||
01_sources.conf: |-
|
||||
<source>
|
||||
@type tail
|
||||
@id in_tail_container_logs
|
||||
@label @KUBERNETES
|
||||
path /var/log/containers/*.log
|
||||
pos_file /var/log/fluentd-containers.log.pos
|
||||
tag kubernetes.*
|
||||
read_from_head true
|
||||
<parse>
|
||||
@type multi_format
|
||||
<pattern>
|
||||
format json
|
||||
time_key time
|
||||
time_type string
|
||||
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
|
||||
keep_time_key false
|
||||
</pattern>
|
||||
<pattern>
|
||||
format regexp
|
||||
expression /^(?<time>.+) (?<stream>stdout|stderr)( (.))? (?<log>.*)$/
|
||||
time_format '%Y-%m-%dT%H:%M:%S.%NZ'
|
||||
keep_time_key false
|
||||
</pattern>
|
||||
</parse>
|
||||
emit_unmatched_lines true
|
||||
</source>
|
||||
|
||||
02_filters.conf: |-
|
||||
<label @KUBERNETES>
|
||||
<match kubernetes.var.log.containers.fluentd**>
|
||||
@type relabel
|
||||
@label @FLUENT_LOG
|
||||
</match>
|
||||
|
||||
# <match kubernetes.var.log.containers.**_kube-system_**>
|
||||
# @type null
|
||||
# @id ignore_kube_system_logs
|
||||
# </match>
|
||||
|
||||
<filter kubernetes.**>
|
||||
@type record_transformer
|
||||
enable_ruby
|
||||
<record>
|
||||
hostname ${record["kubernetes"]["host"]}
|
||||
raw ${record["log"]}
|
||||
</record>
|
||||
remove_keys $.kubernetes.host,log
|
||||
</filter>
|
||||
|
||||
<match **>
|
||||
@type relabel
|
||||
@label @DISPATCH
|
||||
</match>
|
||||
</label>
|
||||
|
||||
03_dispatch.conf: |-
|
||||
<label @DISPATCH>
|
||||
<filter **>
|
||||
@type prometheus
|
||||
<metric>
|
||||
name fluentd_input_status_num_records_total
|
||||
type counter
|
||||
desc The total number of incoming records
|
||||
<labels>
|
||||
tag ${tag}
|
||||
hostname ${hostname}
|
||||
</labels>
|
||||
</metric>
|
||||
</filter>
|
||||
|
||||
<match **>
|
||||
@type relabel
|
||||
@label @OUTPUT
|
||||
</match>
|
||||
</label>
|
||||
|
||||
04_outputs.conf: |-
|
||||
<label @OUTPUT>
|
||||
<match **>
|
||||
@type elasticsearch
|
||||
host "elasticsearch-master"
|
||||
port 9200
|
||||
path ""
|
||||
user elastic
|
||||
password changeme
|
||||
</match>
|
||||
</label>
|
||||
```
|
||||
|
||||
## Backwards Compatibility - v0.1.x
|
||||
|
||||
The old fluentd chart used the ENV variables and the default fluentd container definitions to set-up automatically many aspects of fluentd. It is still possible to trigger this behaviour by removing this charts current `.Values.env` configuration and replace by:
|
||||
|
||||
```yaml
|
||||
env:
|
||||
- name: FLUENT_ELASTICSEARCH_HOST
|
||||
value: "elasticsearch-master"
|
||||
- name: FLUENT_ELASTICSEARCH_PORT
|
||||
value: "9200"
|
||||
```
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,6 +1,5 @@
|
|||
To verify that Fluentd Elasticsearch has started, run:
|
||||
Get Fluentd build information by running these commands:
|
||||
|
||||
kubectl --namespace={{ .Release.Namespace }} get all -l "app={{ template "fluentd.name" . }},release={{ .Release.Name }}"
|
||||
|
||||
THIS APPLICATION CAPTURES ALL CONSOLE OUTPUT AND FORWARDS IT TO Elasticsearch. Anything that might be identifying,
|
||||
including things like IP addresses, container images, and object names will NOT be anonymized.
|
||||
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "fluentd.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
|
||||
echo "curl http://127.0.0.1:24231/metrics for Fluentd metrics"
|
||||
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 24231:24231
|
||||
|
|
|
|||
|
|
@ -31,6 +31,26 @@ Create chart name and version as used by the chart label.
|
|||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Common labels
|
||||
*/}}
|
||||
{{- define "fluentd.labels" -}}
|
||||
helm.sh/chart: {{ include "fluentd.chart" . }}
|
||||
{{ include "fluentd.selectorLabels" . }}
|
||||
{{- if .Chart.AppVersion }}
|
||||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
|
||||
{{- end }}
|
||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Selector labels
|
||||
*/}}
|
||||
{{- define "fluentd.selectorLabels" -}}
|
||||
app.kubernetes.io/name: {{ include "fluentd.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Create the name of the service account to use
|
||||
*/}}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,82 @@
|
|||
{{- define "fluentd.pod" -}}
|
||||
{{- $defaultTag := printf "%s-debian-elasticsearch7-1.0" (.Chart.AppVersion) -}}
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- if .Values.priorityClassName }}
|
||||
priorityClassName: {{ .Values.priorityClassName }}
|
||||
{{- end }}
|
||||
serviceAccountName: {{ include "fluentd.serviceAccountName" . }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.podSecurityContext | nindent 2 }}
|
||||
containers:
|
||||
- name: {{ .Chart.Name }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.securityContext | nindent 6 }}
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default $defaultTag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
{{- if .Values.plugins }}
|
||||
command:
|
||||
- "/bin/sh"
|
||||
- "-c"
|
||||
- |
|
||||
{{- range $plugin := .Values.plugins }}
|
||||
{{- print "fluent-gem install " $plugin | nindent 6 }}
|
||||
{{- end }}
|
||||
exec /fluentd/entrypoint.sh
|
||||
{{- end }}
|
||||
{{- if .Values.env }}
|
||||
env:
|
||||
{{- toYaml .Values.env | nindent 6 }}
|
||||
{{- end }}
|
||||
{{- if .Values.envFrom }}
|
||||
envFrom:
|
||||
{{- toYaml .Values.envFrom | nindent 6 }}
|
||||
{{- end }}
|
||||
ports:
|
||||
- name: metrics
|
||||
containerPort: 24231
|
||||
protocol: TCP
|
||||
{{- range $port := .Values.service.ports }}
|
||||
- name: {{ $port.name }}
|
||||
containerPort: {{ $port.containerPort }}
|
||||
protocol: {{ $port.protocol }}
|
||||
{{- end }}
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /metrics
|
||||
port: metrics
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /metrics
|
||||
port: metrics
|
||||
resources:
|
||||
{{- toYaml .Values.resources | nindent 8 }}
|
||||
volumeMounts:
|
||||
{{- toYaml .Values.volumeMounts | nindent 6 }}
|
||||
{{- range $key := .Values.configMapConfigs }}
|
||||
{{- print "- name: fluentd-custom-cm-" $key | nindent 6 }}
|
||||
{{- print "mountPath: /etc/fluent/" $key ".d" | nindent 8 }}
|
||||
{{- end }}
|
||||
volumes:
|
||||
{{- toYaml .Values.volumes | nindent 2 }}
|
||||
{{- range $key := .Values.configMapConfigs }}
|
||||
{{- print "- name: fluentd-custom-cm-" $key | nindent 2 }}
|
||||
configMap:
|
||||
{{- print "name: " . | nindent 6 }}
|
||||
defaultMode: 0777
|
||||
{{- end }}
|
||||
{{- with .Values.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{- toYaml . | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- with .Values.affinity }}
|
||||
affinity:
|
||||
{{- toYaml . | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- with .Values.tolerations }}
|
||||
tolerations:
|
||||
{{- toYaml . | nindent 2 }}
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
|
@ -1,21 +1,26 @@
|
|||
{{- if .Values.rbac.create -}}
|
||||
kind: ClusterRole
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- "namespaces"
|
||||
- "pods"
|
||||
verbs:
|
||||
- "get"
|
||||
- "watch"
|
||||
- "list"
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- pods
|
||||
- namespaces
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- policy
|
||||
resourceNames:
|
||||
- {{ include "fluentd.fullname" . }}
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -1,19 +1,16 @@
|
|||
{{- if .Values.rbac.create -}}
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: {{ include "fluentd.serviceAccountName" . }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,17 @@
|
|||
{{- if .Values.dashboards.enabled -}}
|
||||
{{- range $path, $_ := .Files.Glob "dashboards/*.json" }}
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: dashboard-{{ trimSuffix ".json" (base $path) }}
|
||||
labels:
|
||||
{{- include "fluentd.labels" $ | nindent 4 }}
|
||||
{{- range $key, $val := $.Values.dashboards.labels }}
|
||||
{{ $key }}: {{ $val }}
|
||||
{{- end }}
|
||||
data:
|
||||
{{ base $path }}: |-
|
||||
{{- $.Files.Get $path | nindent 4 }}
|
||||
---
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
|
@ -1,37 +0,0 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
data:
|
||||
{{- range $key, $value := .Values.configMaps }}
|
||||
{{ $key }}: |-
|
||||
{{ $value | indent 4 }}
|
||||
{{- end }}
|
||||
{{- if .Values.metrics.enabled }}
|
||||
metrics.conf: |
|
||||
<source>
|
||||
@type prometheus
|
||||
port {{ .Values.metrics.service.port }}
|
||||
</source>
|
||||
|
||||
<source>
|
||||
@type prometheus_monitor
|
||||
</source>
|
||||
|
||||
<source>
|
||||
@type prometheus_output_monitor
|
||||
</source>
|
||||
{{- end }}
|
||||
{{- if and (.Values.plugins.enabled) (gt (len .Values.plugins.pluginsList) 0) }}
|
||||
install-plugins.sh: |-
|
||||
#!/bin/sh
|
||||
{{- range $plugin := .Values.plugins.pluginsList }}
|
||||
fluent-gem install {{ $plugin }}
|
||||
{{- end }}
|
||||
exec /run.sh
|
||||
{{- end }}
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
{{- if eq .Values.kind "DaemonSet" }}
|
||||
apiVersion: apps/v1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "fluentd.selectorLabels" . | nindent 6 }}
|
||||
template:
|
||||
metadata:
|
||||
{{- with .Values.podAnnotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
labels:
|
||||
{{- include "fluentd.selectorLabels" . | nindent 8 }}
|
||||
{{- with .Values.podLabels }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- include "fluentd.pod" . | nindent 6 }}
|
||||
{{- end }}
|
||||
|
|
@ -1,151 +1,27 @@
|
|||
{{- $statefulSet := or (.Values.autoscaling.enabled) (.Values.useStatefulSet) -}}
|
||||
{{- if eq .Values.kind "Deployment" }}
|
||||
apiVersion: apps/v1
|
||||
{{- if $statefulSet }}
|
||||
kind: StatefulSet
|
||||
{{- else}}
|
||||
kind: Deployment
|
||||
{{- end}}
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
spec:
|
||||
{{- if not .Values.autoscaling.enabled }}
|
||||
replicas: {{ .Values.replicaCount }}
|
||||
{{- end }}
|
||||
{{- if .Values.autoscaling.enabled }}
|
||||
serviceName: {{ template "fluentd.name" . }}
|
||||
{{- end }}
|
||||
selector:
|
||||
matchLabels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
release: {{ .Release.Name }}
|
||||
{{- if and .Values.persistence.enabled (not .Values.autoscaling.enabled) }}
|
||||
strategy:
|
||||
type: Recreate
|
||||
{{- end }}
|
||||
{{- include "fluentd.selectorLabels" . | nindent 6 }}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
release: {{ .Release.Name }}
|
||||
{{- with .Values.deployment.labels }}
|
||||
{{ toYaml . | indent 8 }}
|
||||
{{- end }}
|
||||
annotations:
|
||||
checksum/configmap: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
|
||||
{{- if .Values.annotations }}
|
||||
{{- toYaml .Values.annotations | nindent 8 }}
|
||||
checksum/config: {{ include (print $.Template.BasePath "/fluentd-configurations-cm.yaml") . | sha256sum }}
|
||||
{{- with .Values.podAnnotations }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
labels:
|
||||
{{- include "fluentd.selectorLabels" . | nindent 8 }}
|
||||
{{- with .Values.podLabels }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.image.pullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- range $pullSecret := .Values.image.pullSecrets }}
|
||||
- name: {{ $pullSecret }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
|
||||
containers:
|
||||
- name: {{ .Chart.Name }}
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
{{- if and (.Values.plugins.enabled) (gt (len .Values.plugins.pluginsList) 0) }}
|
||||
command: ["/bin/sh", "-c", "/etc/fluent/config.d/install-plugins.sh"]
|
||||
{{- end }}
|
||||
env:
|
||||
- name: OUTPUT_HOST
|
||||
value: {{ .Values.output.host | quote }}
|
||||
- name: OUTPUT_PORT
|
||||
value: {{ .Values.output.port | quote }}
|
||||
- name: OUTPUT_SCHEME
|
||||
value: {{ .Values.output.scheme | quote }}
|
||||
- name: OUTPUT_SSL_VERSION
|
||||
value: {{ .Values.output.sslVersion | quote }}
|
||||
- name: OUTPUT_BUFFER_CHUNK_LIMIT
|
||||
value: {{ .Values.output.buffer_chunk_limit | quote }}
|
||||
- name: OUTPUT_BUFFER_QUEUE_LIMIT
|
||||
value: {{ .Values.output.buffer_queue_limit | quote }}
|
||||
{{- range $key, $value := .Values.env }}
|
||||
- name: {{ $key | quote }}
|
||||
value: {{ $value | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.extraEnvVars }}
|
||||
{{ toYaml .Values.extraEnvVars | indent 10 }}
|
||||
{{- end }}
|
||||
resources:
|
||||
{{ toYaml .Values.resources | indent 12 }}
|
||||
ports:
|
||||
{{- range $port := .Values.service.ports }}
|
||||
- name: {{ $port.name }}
|
||||
containerPort: {{ $port.containerPort }}
|
||||
protocol: {{ $port.protocol }}
|
||||
{{- end }}
|
||||
{{- if .Values.metrics.enabled }}
|
||||
- name: metrics
|
||||
containerPort: {{ .Values.metrics.service.port }}
|
||||
protocol: TCP
|
||||
{{- end }}
|
||||
- name: http-input
|
||||
containerPort: 9880
|
||||
protocol: TCP
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
# Use percent encoding for query param.
|
||||
# The value is {"log": "health check"}.
|
||||
# the endpoint itself results in a new fluentd
|
||||
# tag 'fluentd.pod-healthcheck'
|
||||
path: /fluentd.pod.healthcheck?json=%7B%22log%22%3A+%22health+check%22%7D
|
||||
port: 9880
|
||||
initialDelaySeconds: 5
|
||||
timeoutSeconds: 1
|
||||
volumeMounts:
|
||||
- name: config-volume-{{ template "fluentd.fullname" . }}
|
||||
mountPath: /etc/fluent/config.d
|
||||
- name: buffer
|
||||
mountPath: "/var/log/fluentd-buffers"
|
||||
{{- if .Values.extraVolumeMounts }}
|
||||
{{ toYaml .Values.extraVolumeMounts | indent 8 }}
|
||||
{{- end }}
|
||||
serviceAccountName: {{ template "fluentd.fullname" . }}
|
||||
volumes:
|
||||
- name: config-volume-{{ template "fluentd.fullname" . }}
|
||||
configMap:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
defaultMode: 0777
|
||||
{{- if and .Values.persistence.enabled (not $statefulSet) }}
|
||||
- name: buffer
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ template "fluentd.fullname" . }}
|
||||
{{- else if (not .Values.persistence.enabled) }}
|
||||
- name: buffer
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
{{- if .Values.extraVolumes }}
|
||||
{{ toYaml .Values.extraVolumes | indent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{ toYaml . | indent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.affinity }}
|
||||
affinity:
|
||||
{{ toYaml . | indent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.tolerations }}
|
||||
tolerations:
|
||||
{{ toYaml . | indent 8 }}
|
||||
{{- end }}
|
||||
{{- if and .Values.persistence.enabled ($statefulSet) }}
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: buffer
|
||||
spec:
|
||||
accessModes: [{{ .Values.persistence.accessMode }}]
|
||||
storageClassName: {{ .Values.persistence.storageClass }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.persistence.size }}
|
||||
{{- include "fluentd.pod" . | nindent 6 }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,25 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
name: fluentd-prometheus-conf
|
||||
data:
|
||||
prometheus.conf: |-
|
||||
<source>
|
||||
@type prometheus
|
||||
@id in_prometheus
|
||||
bind "0.0.0.0"
|
||||
port 24231
|
||||
metrics_path "/metrics"
|
||||
</source>
|
||||
|
||||
<source>
|
||||
@type prometheus_monitor
|
||||
@id in_prometheus_monitor
|
||||
</source>
|
||||
|
||||
<source>
|
||||
@type prometheus_output_monitor
|
||||
@id in_prometheus_output_monitor
|
||||
</source>
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: fluentd-config
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
data:
|
||||
{{- range $key, $value := .Values.fileConfigs }}
|
||||
{{$key }}: |-
|
||||
{{- (tpl $value $) | nindent 4 }}
|
||||
{{- end }}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: fluentd-main
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
data:
|
||||
fluent.conf: |-
|
||||
# do not collect fluentd logs to avoid infinite loops.
|
||||
<label @FLUENT_LOG>
|
||||
<match **>
|
||||
@type null
|
||||
@id ignore_fluent_logs
|
||||
</match>
|
||||
</label>
|
||||
|
||||
@include config.d/*.conf
|
||||
{{- range $key := .Values.configMapConfigs }}
|
||||
{{- print "@include " $key ".d/*" | nindent 4 }}
|
||||
{{- end }}
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
{{- if and .Values.autoscaling.enabled}}
|
||||
apiVersion: autoscaling/v2beta2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
minReplicas: {{ .Values.autoscaling.minReplicas }}
|
||||
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
|
||||
metrics:
|
||||
{{- toYaml .Values.autoscaling.metrics | nindent 4 }}
|
||||
{{- end }}
|
||||
|
|
@ -1,36 +0,0 @@
|
|||
{{- if .Values.ingress.enabled -}}
|
||||
{{- $serviceName := include "fluentd.fullname" . -}}
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- if .Values.ingress.labels }}
|
||||
{{ toYaml .Values.ingress.labels | indent 4 }}
|
||||
{{- end }}
|
||||
{{- if .Values.ingress.annotations }}
|
||||
annotations:
|
||||
{{ tpl ( toYaml .Values.ingress.annotations | indent 4 ) . }}
|
||||
{{- end }}
|
||||
spec:
|
||||
rules:
|
||||
{{- range $host := .Values.ingress.hosts }}
|
||||
- http:
|
||||
paths:
|
||||
- path: {{ $host.path | default "/" }}
|
||||
backend:
|
||||
serviceName: {{ $serviceName }}
|
||||
servicePort: {{ $host.servicePort }}
|
||||
{{- if (not (empty $host.name)) }}
|
||||
host: {{ $host.name }}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
{{- if .Values.ingress.tls }}
|
||||
tls:
|
||||
{{ toYaml .Values.ingress.tls | indent 4 }}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
{{- if .Values.podSecurityPolicy.enabled }}
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
{{- if .Values.podSecurityPolicy.annotations }}
|
||||
annotations:
|
||||
{{ toYaml .Values.podSecurityPolicy.annotations | indent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
requiredDropCapabilities:
|
||||
- ALL
|
||||
hostNetwork: false
|
||||
hostIPC: false
|
||||
hostPID: false
|
||||
volumes:
|
||||
- 'configMap'
|
||||
- 'secret'
|
||||
- 'hostPath'
|
||||
runAsUser:
|
||||
rule: 'RunAsAny'
|
||||
seLinux:
|
||||
rule: 'RunAsAny'
|
||||
supplementalGroups:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
- min: 1
|
||||
max: 65535
|
||||
fsGroup:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
- min: 1
|
||||
max: 65535
|
||||
readOnlyRootFilesystem: false
|
||||
{{- end }}
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
{{- if and ( .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" ) .Values.metrics.prometheusRule.enabled }}
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: PrometheusRule
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
{{- if .Values.metrics.prometheusRule.namespace }}
|
||||
namespace: {{ .Values.metrics.prometheusRule.namespace }}
|
||||
{{- end }}
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
{{- with .Values.metrics.prometheusRule.additionalLabels }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- with .Values.metrics.prometheusRule.rules }}
|
||||
groups:
|
||||
- name: {{ template "fluentd.fullname" . }}
|
||||
rules: {{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
|
@ -1,29 +0,0 @@
|
|||
{{- $statefulSet := or (.Values.autoscaling.enabled) (.Values.useStatefulSet) -}}
|
||||
{{- if and .Values.persistence.enabled (not $statefulSet) -}}
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- if .Values.persistence.annotations }}
|
||||
annotations:
|
||||
{{ toYaml .Values.persistence.annotations | indent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
accessModes:
|
||||
- {{ .Values.persistence.accessMode | quote }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.persistence.size | quote }}
|
||||
{{- if .Values.persistence.storageClass }}
|
||||
{{- if (eq "-" .Values.persistence.storageClass) }}
|
||||
storageClassName: ""
|
||||
{{- else }}
|
||||
storageClassName: "{{ .Values.persistence.storageClass }}"
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
{{- if .Values.rbac.create }}
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
rules:
|
||||
- apiGroups: ['extensions']
|
||||
resources: ['podsecuritypolicies']
|
||||
verbs: ['use']
|
||||
resourceNames:
|
||||
- {{ template "fluentd.fullname" . }}
|
||||
{{- end }}
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
{{- if .Values.rbac.create }}
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
|
|
@ -1,35 +1,28 @@
|
|||
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
{{- with .Values.service.annotations }}
|
||||
annotations:
|
||||
{{ toYaml .Values.service.annotations | indent 4 }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
type: {{ .Values.service.type }}
|
||||
{{ if (and (eq .Values.service.type "LoadBalancer") (not (empty .Values.service.loadBalancerIP))) }}
|
||||
loadBalancerIP: {{ .Values.service.loadBalancerIP }}
|
||||
{{ end }}
|
||||
ports:
|
||||
- port: 24231
|
||||
targetPort: metrics
|
||||
protocol: TCP
|
||||
name: metrics
|
||||
{{- if .Values.service.ports }}
|
||||
{{- range $port := .Values.service.ports }}
|
||||
- name: {{ $port.name }}
|
||||
port: {{ $port.containerPort }}
|
||||
targetPort: {{ $port.containerPort }}
|
||||
protocol: {{ $port.protocol }}
|
||||
- name: {{ $port.name }}
|
||||
port: {{ $port.containerPort }}
|
||||
targetPort: {{ $port.containerPort }}
|
||||
protocol: {{ $port.protocol }}
|
||||
{{- end }}
|
||||
{{ if (and (eq .Values.service.type "NodePort") (not (empty .Values.service.nodePort))) }}
|
||||
nodePort: {{ .Values.service.nodePort }}
|
||||
{{ end }}
|
||||
{{- if .Values.metrics.enabled }}
|
||||
- name: metrics
|
||||
port: {{ .Values.metrics.service.port }}
|
||||
targetPort: metrics
|
||||
protocol: TCP
|
||||
{{- end }}
|
||||
selector:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
release: {{ .Release.Name }}
|
||||
{{- include "fluentd.selectorLabels" . | nindent 4 }}
|
||||
|
|
|
|||
|
|
@ -4,8 +4,9 @@ kind: ServiceAccount
|
|||
metadata:
|
||||
name: {{ include "fluentd.serviceAccountName" . }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
{{- with .Values.serviceAccount.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -1,33 +1,35 @@
|
|||
{{- if and .Values.metrics.enabled .Values.metrics.serviceMonitor.enabled }}
|
||||
{{- if and ( .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" ) .Values.metrics.serviceMonitor.enabled }}
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: ServiceMonitor
|
||||
metadata:
|
||||
name: {{ include "fluentd.fullname" . }}
|
||||
{{- if .Values.metrics.serviceMonitor.namespace }}
|
||||
namespace: {{ .Values.metrics.serviceMonitor.namespace }}
|
||||
name: {{ template "fluentd.fullname" . }}
|
||||
{{- with .Values.metrics.serviceMonitor.namespace }}
|
||||
namespace: {{ . }}
|
||||
{{- end }}
|
||||
labels:
|
||||
app: {{ template "fluentd.name" . }}
|
||||
chart: {{ template "fluentd.chart" . }}
|
||||
release: {{ .Release.Name }}
|
||||
heritage: {{ .Release.Service }}
|
||||
{{- if .Values.metrics.serviceMonitor.additionalLabels }}
|
||||
{{ toYaml .Values.metrics.serviceMonitor.additionalLabels | indent 4 }}
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
{{- with .Values.metrics.serviceMonitor.additionalLabels }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
endpoints:
|
||||
- port: metrics
|
||||
{{- if .Values.metrics.serviceMonitor.interval }}
|
||||
interval: {{ .Values.metrics.serviceMonitor.interval }}
|
||||
path: /metrics
|
||||
{{- with .Values.metrics.serviceMonitor.interval }}
|
||||
interval: {{ . }}
|
||||
{{- end }}
|
||||
{{- if .Values.metrics.serviceMonitor.scrapeTimeout }}
|
||||
scrapeTimeout: {{ .Values.metrics.serviceMonitor.scrapeTimeout }}
|
||||
{{- with .Values.metrics.serviceMonitor.scrapeTimeout }}
|
||||
scrapeTimeout: {{ . }}
|
||||
{{- end }}
|
||||
{{- if .Values.metrics.serviceMonitor.namespaceSelector }}
|
||||
namespaceSelector:
|
||||
{{ toYaml .Values.metrics.serviceMonitor.namespaceSelector | indent 4 -}}
|
||||
{{ else }}
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
- {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
selector:
|
||||
matchLabels:
|
||||
app: {{ include "fluentd.name" . }}
|
||||
release: {{ .Release.Name }}
|
||||
{{- include "fluentd.selectorLabels" . | nindent 6 }}
|
||||
{{- end }}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,17 @@
|
|||
{{- if .Values.testFramework.enabled }}
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: "{{ include "fluentd.fullname" . }}-test-connection"
|
||||
labels:
|
||||
{{- include "fluentd.labels" . | nindent 4 }}
|
||||
annotations:
|
||||
"helm.sh/hook": test-success
|
||||
spec:
|
||||
containers:
|
||||
- name: wget
|
||||
image: busybox
|
||||
command: ['wget']
|
||||
args: ['{{ include "fluentd.fullname" . }}:{{ .Values.service.port }}']
|
||||
restartPolicy: Never
|
||||
{{- end }}
|
||||
|
|
@ -1,224 +1,260 @@
|
|||
# Default values for fluentd.
|
||||
# This is a YAML-formatted file.
|
||||
# Declare variables to be passed into your templates.
|
||||
replicaCount: 1
|
||||
nameOverride: ""
|
||||
fullnameOverride: ""
|
||||
|
||||
# DaemonSet or Deployment
|
||||
kind: "DaemonSet"
|
||||
|
||||
# # Only applicable for Deployment
|
||||
# replicaCount: 1
|
||||
|
||||
image:
|
||||
repository: gcr.io/google-containers/fluentd-elasticsearch
|
||||
tag: v2.4.0
|
||||
pullPolicy: IfNotPresent
|
||||
# pullSecrets:
|
||||
# - secret1
|
||||
# - secret2
|
||||
repository: "fluent/fluentd-kubernetes-daemonset"
|
||||
pullPolicy: "IfNotPresent"
|
||||
tag: ""
|
||||
|
||||
output:
|
||||
host: elasticsearch-client.default.svc.cluster.local
|
||||
port: 9200
|
||||
scheme: http
|
||||
sslVersion: TLSv1
|
||||
buffer_chunk_limit: 2M
|
||||
buffer_queue_limit: 8
|
||||
|
||||
env: {}
|
||||
|
||||
# Extra Environment Values - allows yaml definitions
|
||||
extraEnvVars:
|
||||
# - name: VALUE_FROM_SECRET
|
||||
# valueFrom:
|
||||
# secretKeyRef:
|
||||
# name: secret_name
|
||||
# key: secret_key
|
||||
|
||||
# extraVolumes:
|
||||
# - name: es-certs
|
||||
# secret:
|
||||
# defaultMode: 420
|
||||
# secretName: es-certs
|
||||
# extraVolumeMounts:
|
||||
# - name: es-certs
|
||||
# mountPath: /certs
|
||||
# readOnly: true
|
||||
|
||||
plugins:
|
||||
testFramework:
|
||||
enabled: false
|
||||
pluginsList: []
|
||||
|
||||
service:
|
||||
annotations: {}
|
||||
type: ClusterIP
|
||||
# loadBalancerIP:
|
||||
# type: NodePort
|
||||
# nodePort:
|
||||
# Used to create Service records
|
||||
ports:
|
||||
- name: "monitor-agent"
|
||||
protocol: TCP
|
||||
containerPort: 24220
|
||||
|
||||
metrics:
|
||||
enabled: false
|
||||
service:
|
||||
port: 24231
|
||||
serviceMonitor:
|
||||
enabled: false
|
||||
additionalLabels: {}
|
||||
# namespace: monitoring
|
||||
# interval: 30s
|
||||
# scrapeTimeout: 10s
|
||||
|
||||
annotations: {}
|
||||
# prometheus.io/scrape: "true"
|
||||
# prometheus.io/port: "24231"
|
||||
|
||||
# Pod Labels
|
||||
deployment:
|
||||
labels: {}
|
||||
|
||||
ingress:
|
||||
enabled: false
|
||||
annotations:
|
||||
kubernetes.io/ingress.class: nginx
|
||||
# kubernetes.io/tls-acme: "true"
|
||||
# # Depending on which version of ingress controller you may need to configure properly - https://kubernetes.github.io/ingress-nginx/examples/rewrite/#rewrite-target
|
||||
# nginx.ingress.kubernetes.io/rewrite-target: /
|
||||
labels: []
|
||||
# If doing TCP or UDP ingress rule don't forget to update your Ingress Controller to accept TCP connections - https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
|
||||
hosts:
|
||||
# - name: "http-input.local"
|
||||
# protocol: TCP
|
||||
# servicePort: 9880
|
||||
# path: /
|
||||
tls: {}
|
||||
# Secrets must be manually created in the namespace.
|
||||
# - secretName: http-input-tls
|
||||
# hosts:
|
||||
# - http-input.local
|
||||
|
||||
configMaps:
|
||||
general.conf: |
|
||||
# Prevent fluentd from handling records containing its own logs. Otherwise
|
||||
# it can lead to an infinite loop, when error in sending one message generates
|
||||
# another message which also fails to be sent and so on.
|
||||
<match fluentd.**>
|
||||
@type null
|
||||
</match>
|
||||
|
||||
# Used for health checking
|
||||
<source>
|
||||
@type http
|
||||
port 9880
|
||||
bind 0.0.0.0
|
||||
</source>
|
||||
|
||||
# Emits internal metrics to every minute, and also exposes them on port
|
||||
# 24220. Useful for determining if an output plugin is retryring/erroring,
|
||||
# or determining the buffer queue length.
|
||||
<source>
|
||||
@type monitor_agent
|
||||
bind 0.0.0.0
|
||||
port 24220
|
||||
tag fluentd.monitor.metrics
|
||||
</source>
|
||||
system.conf: |-
|
||||
<system>
|
||||
root_dir /tmp/fluentd-buffers/
|
||||
</system>
|
||||
forward-input.conf: |
|
||||
<source>
|
||||
@type forward
|
||||
port 24224
|
||||
bind 0.0.0.0
|
||||
</source>
|
||||
output.conf: |
|
||||
<match **>
|
||||
@id elasticsearch
|
||||
@type elasticsearch
|
||||
@log_level info
|
||||
include_tag_key true
|
||||
# Replace with the host/port to your Elasticsearch cluster.
|
||||
host "#{ENV['OUTPUT_HOST']}"
|
||||
port "#{ENV['OUTPUT_PORT']}"
|
||||
scheme "#{ENV['OUTPUT_SCHEME']}"
|
||||
ssl_version "#{ENV['OUTPUT_SSL_VERSION']}"
|
||||
logstash_format true
|
||||
<buffer>
|
||||
@type file
|
||||
path /var/log/fluentd-buffers/kubernetes.system.buffer
|
||||
flush_mode interval
|
||||
retry_type exponential_backoff
|
||||
flush_thread_count 2
|
||||
flush_interval 5s
|
||||
retry_forever
|
||||
retry_max_interval 30
|
||||
chunk_limit_size "#{ENV['OUTPUT_BUFFER_CHUNK_LIMIT']}"
|
||||
queue_limit_length "#{ENV['OUTPUT_BUFFER_QUEUE_LIMIT']}"
|
||||
overflow_action block
|
||||
</buffer>
|
||||
</match>
|
||||
|
||||
resources: {}
|
||||
# We usually recommend not to specify default resources and to leave this as a conscious
|
||||
# choice for the user. This also increases chances charts run on environments with little
|
||||
# resources, such as Minikube. If you do want to specify resources, uncomment the following
|
||||
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
|
||||
# limits:
|
||||
# cpu: 500m
|
||||
# memory: 200Mi
|
||||
# requests:
|
||||
# cpu: 500m
|
||||
# memory: 200Mi
|
||||
|
||||
rbac:
|
||||
# Specifies whether RBAC resources should be created
|
||||
create: true
|
||||
## Optional array of imagePullSecrets containing private registry credentials
|
||||
## Ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
|
||||
imagePullSecrets: []
|
||||
|
||||
serviceAccount:
|
||||
# Specifies whether a ServiceAccount should be created
|
||||
create: true
|
||||
# The name of the ServiceAccount to use.
|
||||
# If not set and create is true, a name is generated using the fullname template
|
||||
name:
|
||||
annotations: {}
|
||||
name: null
|
||||
|
||||
## Persist data to a persistent volume
|
||||
persistence:
|
||||
enabled: false
|
||||
rbac:
|
||||
create: true
|
||||
|
||||
## If defined, storageClassName: <storageClass>
|
||||
## If set to "-", storageClassName: "", which disables dynamic provisioning
|
||||
## If undefined (the default) or set to null, no storageClassName spec is
|
||||
## set, choosing the default provisioner. (gp2 on AWS, standard on
|
||||
## GKE, AWS & OpenStack)
|
||||
##
|
||||
# storageClass: "-"
|
||||
# annotations: {}
|
||||
accessMode: ReadWriteOnce
|
||||
size: 10Gi
|
||||
# Configure podsecuritypolicy
|
||||
# Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/
|
||||
podSecurityPolicy:
|
||||
enabled: true
|
||||
annotations: {}
|
||||
|
||||
## Security Context policies for controller pods
|
||||
## See https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/ for
|
||||
## notes on enabling and using sysctls
|
||||
##
|
||||
podSecurityContext: {}
|
||||
# seLinuxOptions:
|
||||
# type: "spc_t"
|
||||
|
||||
securityContext: {}
|
||||
# capabilities:
|
||||
# drop:
|
||||
# - ALL
|
||||
# readOnlyRootFilesystem: true
|
||||
# runAsNonRoot: true
|
||||
# runAsUser: 1000
|
||||
|
||||
resources: {}
|
||||
# requests:
|
||||
# cpu: 10m
|
||||
# memory: 128Mi
|
||||
# limits:
|
||||
# memory: 128Mi
|
||||
|
||||
# priorityClassName: "system-node-critical"
|
||||
|
||||
nodeSelector: {}
|
||||
|
||||
## Node tolerations for server scheduling to nodes with taints
|
||||
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
|
||||
##
|
||||
tolerations: []
|
||||
# - key: null
|
||||
# operator: Exists
|
||||
# effect: "NoSchedule"
|
||||
|
||||
## Affinity and anti-affinity
|
||||
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
|
||||
##
|
||||
affinity: {}
|
||||
|
||||
# Enable autoscaling using HorizontalPodAutoscaler
|
||||
autoscaling:
|
||||
enabled: false
|
||||
minReplicas: 2
|
||||
maxReplicas: 5
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 60
|
||||
- type: Resource
|
||||
resource:
|
||||
name: memory
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 60
|
||||
## Annotations to be added to fluentd pods
|
||||
##
|
||||
podAnnotations: {}
|
||||
|
||||
# Consider to set higher value when using in conjuction with autoscaling
|
||||
# Full description about this field: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.15/#pod-v1-core
|
||||
terminationGracePeriodSeconds: 30
|
||||
## Labels to be added to fluentd pods
|
||||
##
|
||||
podLabels: {}
|
||||
|
||||
## Additional environment variables to set for fluentd pods
|
||||
env:
|
||||
- name: "FLUENTD_CONF"
|
||||
value: "../../etc/fluent/fluent.conf"
|
||||
# - name: FLUENT_ELASTICSEARCH_HOST
|
||||
# value: "elasticsearch-master"
|
||||
# - name: FLUENT_ELASTICSEARCH_PORT
|
||||
# value: "9200"
|
||||
|
||||
envFrom: []
|
||||
|
||||
volumes:
|
||||
- name: varlog
|
||||
hostPath:
|
||||
path: /var/log
|
||||
- name: varlibdockercontainers
|
||||
hostPath:
|
||||
path: /var/lib/docker/containers
|
||||
- name: etcfluentd-main
|
||||
configMap:
|
||||
name: fluentd-main
|
||||
defaultMode: 0777
|
||||
- name: etcfluentd-config
|
||||
configMap:
|
||||
name: fluentd-config
|
||||
defaultMode: 0777
|
||||
|
||||
volumeMounts:
|
||||
- name: varlog
|
||||
mountPath: /var/log
|
||||
- name: varlibdockercontainers
|
||||
mountPath: /var/lib/docker/containers
|
||||
readOnly: true
|
||||
- name: etcfluentd-main
|
||||
mountPath: /etc/fluent
|
||||
- name: etcfluentd-config
|
||||
mountPath: /etc/fluent/config.d/
|
||||
|
||||
## Fluentd service
|
||||
##
|
||||
service:
|
||||
type: "ClusterIP"
|
||||
annotations: {}
|
||||
ports: []
|
||||
# - name: "forwarder"
|
||||
# protocol: TCP
|
||||
# containerPort: 24224
|
||||
|
||||
## Prometheus Monitoring
|
||||
##
|
||||
metrics:
|
||||
serviceMonitor:
|
||||
enabled: false
|
||||
additionalLabels:
|
||||
release: prometheus-operator
|
||||
namespace: ""
|
||||
namespaceSelector: {}
|
||||
# scrapeInterval: 30s
|
||||
# scrapeTimeout: 5s
|
||||
# honorLabels: true
|
||||
|
||||
prometheusRule:
|
||||
enabled: false
|
||||
additionalLabels: {}
|
||||
namespace: ""
|
||||
rules: []
|
||||
|
||||
## Grafana Monitoring Dashboard
|
||||
##
|
||||
dashboards:
|
||||
enabled: "true"
|
||||
namespace: ""
|
||||
labels:
|
||||
grafana_dashboard: '"1"'
|
||||
|
||||
## Fluentd list of plugins to install
|
||||
##
|
||||
plugins: []
|
||||
# - fluent-plugin-out-http
|
||||
|
||||
## Add fluentd config files from K8s configMaps
|
||||
##
|
||||
configMapConfigs:
|
||||
- fluentd-prometheus-conf
|
||||
# - fluentd-systemd-conf
|
||||
|
||||
## Fluentd configurations:
|
||||
##
|
||||
fileConfigs:
|
||||
01_sources.conf: |-
|
||||
## logs from podman
|
||||
<source>
|
||||
@type tail
|
||||
@id in_tail_container_logs
|
||||
@label @KUBERNETES
|
||||
path /var/log/containers/*.log
|
||||
pos_file /var/log/fluentd-containers.log.pos
|
||||
tag kubernetes.*
|
||||
read_from_head true
|
||||
<parse>
|
||||
@type multi_format
|
||||
<pattern>
|
||||
format json
|
||||
time_key time
|
||||
time_type string
|
||||
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
|
||||
keep_time_key false
|
||||
</pattern>
|
||||
<pattern>
|
||||
format regexp
|
||||
expression /^(?<time>.+) (?<stream>stdout|stderr)( (.))? (?<log>.*)$/
|
||||
time_format '%Y-%m-%dT%H:%M:%S.%NZ'
|
||||
keep_time_key false
|
||||
</pattern>
|
||||
</parse>
|
||||
emit_unmatched_lines true
|
||||
</source>
|
||||
|
||||
02_filters.conf: |-
|
||||
<label @KUBERNETES>
|
||||
<match kubernetes.var.log.containers.fluentd**>
|
||||
@type relabel
|
||||
@label @FLUENT_LOG
|
||||
</match>
|
||||
|
||||
# <match kubernetes.var.log.containers.**_kube-system_**>
|
||||
# @type null
|
||||
# @id ignore_kube_system_logs
|
||||
# </match>
|
||||
|
||||
<filter kubernetes.**>
|
||||
@type kubernetes_metadata
|
||||
@id filter_kube_metadata
|
||||
skip_labels false
|
||||
skip_container_metadata false
|
||||
skip_namespace_metadata true
|
||||
skip_master_url true
|
||||
</filter>
|
||||
|
||||
<match **>
|
||||
@type relabel
|
||||
@label @DISPATCH
|
||||
</match>
|
||||
</label>
|
||||
|
||||
03_dispatch.conf: |-
|
||||
<label @DISPATCH>
|
||||
<filter **>
|
||||
@type prometheus
|
||||
<metric>
|
||||
name fluentd_input_status_num_records_total
|
||||
type counter
|
||||
desc The total number of incoming records
|
||||
<labels>
|
||||
tag ${tag}
|
||||
hostname ${hostname}
|
||||
</labels>
|
||||
</metric>
|
||||
</filter>
|
||||
|
||||
<match **>
|
||||
@type relabel
|
||||
@label @OUTPUT
|
||||
</match>
|
||||
</label>
|
||||
|
||||
04_outputs.conf: |-
|
||||
<label @OUTPUT>
|
||||
<match **>
|
||||
@type elasticsearch
|
||||
host "elasticsearch-master"
|
||||
port 9200
|
||||
path ""
|
||||
user elastic
|
||||
password changeme
|
||||
</match>
|
||||
</label>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,125 @@
|
|||
diff -tubrN charts/fluentd/templates/files.conf/systemd.yaml charts/fluentd.zdt/templates/files.conf/systemd.yaml
|
||||
--- charts/fluentd/templates/files.conf/systemd.yaml 2021-02-12 18:13:04.000000000 +0100
|
||||
+++ charts/fluentd.zdt/templates/files.conf/systemd.yaml 1970-01-01 01:00:00.000000000 +0100
|
||||
@@ -1,83 +0,0 @@
|
||||
-apiVersion: v1
|
||||
-kind: ConfigMap
|
||||
-metadata:
|
||||
- labels:
|
||||
- {{- include "fluentd.labels" . | nindent 4 }}
|
||||
- name: fluentd-systemd-conf
|
||||
-data:
|
||||
- systemd.conf: |-
|
||||
- <source>
|
||||
- @type systemd
|
||||
- @id in_systemd_internal_kubernetes
|
||||
- @label @KUBERNETES_SYSTEM
|
||||
- matches [{"_SYSTEMD_UNIT":"kubelet.service"},{"_SYSTEMD_UNIT":"kube-apiserver.service"},{"_SYSTEMD_UNIT":"kube-controller-manager.service"},{"_SYSTEMD_UNIT":"kube-proxy.service"},{"_SYSTEMD_UNIT":"kube-scheduler.service"}]
|
||||
- read_from_head true
|
||||
- tag "internal-kubernetes.systemd"
|
||||
- <storage>
|
||||
- @type "local"
|
||||
- persistent true
|
||||
- path "/var/log/fluentd-journald-internal_kubernetes-cursor.json"
|
||||
- </storage>
|
||||
- <entry>
|
||||
- fields_strip_underscores true
|
||||
- field_map {"MESSAGE": "message", "_TRANSPORT": "stream", "_SYSTEMD_UNIT": "systemd_unit", "_HOSTNAME": "hostname"}
|
||||
- field_map_strict true
|
||||
- </entry>
|
||||
- </source>
|
||||
-
|
||||
- <source>
|
||||
- @type systemd
|
||||
- @id in_systemd_etcd
|
||||
- @label @KUBERNETES_SYSTEM
|
||||
- matches [{"_SYSTEMD_UNIT":"etcd.service"}]
|
||||
- read_from_head true
|
||||
- tag "etcd.systemd"
|
||||
- <storage>
|
||||
- @type "local"
|
||||
- persistent true
|
||||
- path "/var/log/fluentd-journald-internal_etcd-cursor.json"
|
||||
- </storage>
|
||||
- <entry>
|
||||
- fields_strip_underscores true
|
||||
- field_map {"MESSAGE": "message", "_TRANSPORT": "stream", "_SYSTEMD_UNIT": "systemd_unit", "_HOSTNAME": "hostname"}
|
||||
- field_map_strict true
|
||||
- </entry>
|
||||
- </source>
|
||||
-
|
||||
- <label @KUBERNETES_SYSTEM>
|
||||
- <filter internal-kubernetes.systemd>
|
||||
- @type parser
|
||||
- key_name message
|
||||
- <parse>
|
||||
- @type regexp
|
||||
- expression /^(?<level>[a-zA-Z])[0-9]* ([\d:.]+)\s+\d+ (?<file>[a-zA-Z-_.]+):(?<line>[\d]+)\]\s+(?<log>.*)$/
|
||||
- </parse>
|
||||
- reserve_data true
|
||||
- reserve_time true
|
||||
- </filter>
|
||||
-
|
||||
- <filter etcd.systemd>
|
||||
- @type parser
|
||||
- key_name message
|
||||
- <parse>
|
||||
- @type regexp
|
||||
- expression /^([^ ]+\s[^ ]+) (?<level>[A-Z]) \| (?<component>[a-zA-Z-_.]+): (?<log>.*)$/
|
||||
- </parse>
|
||||
- reserve_data true
|
||||
- reserve_time true
|
||||
- </filter>
|
||||
-
|
||||
- <filter **>
|
||||
- @type record_transformer
|
||||
- enable_ruby
|
||||
- <record>
|
||||
- raw ${record["message"]}
|
||||
- </record>
|
||||
- remove_keys message
|
||||
- </filter>
|
||||
-
|
||||
- <match **>
|
||||
- @type relabel
|
||||
- @label @DISPATCH
|
||||
- </match>
|
||||
- </label>
|
||||
diff -tubrN charts/fluentd/templates/fluentd-configurations-cm.yaml charts/fluentd.zdt/templates/fluentd-configurations-cm.yaml
|
||||
--- charts/fluentd/templates/fluentd-configurations-cm.yaml 2021-02-12 18:13:04.000000000 +0100
|
||||
+++ charts/fluentd.zdt/templates/fluentd-configurations-cm.yaml 2021-03-09 17:00:50.318180593 +0100
|
||||
@@ -7,7 +7,7 @@
|
||||
data:
|
||||
{{- range $key, $value := .Values.fileConfigs }}
|
||||
{{$key }}: |-
|
||||
- {{- $value | nindent 4 }}
|
||||
+ {{- (tpl $value $) | nindent 4 }}
|
||||
{{- end }}
|
||||
|
||||
---
|
||||
diff -tubrN charts/fluentd/templates/tests/test-connection.yaml charts/fluentd.zdt/templates/tests/test-connection.yaml
|
||||
--- charts/fluentd/templates/tests/test-connection.yaml 2021-02-12 18:13:04.000000000 +0100
|
||||
+++ charts/fluentd.zdt/templates/tests/test-connection.yaml 2021-03-09 17:02:56.061519591 +0100
|
||||
@@ -1,3 +1,4 @@
|
||||
+{{- if .Values.testFramework.enabled }}
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
@@ -13,3 +14,4 @@
|
||||
command: ['wget']
|
||||
args: ['{{ include "fluentd.fullname" . }}:{{ .Values.service.port }}']
|
||||
restartPolicy: Never
|
||||
+{{- end }}
|
||||
diff -tubrN charts/fluentd/values.yaml charts/fluentd.zdt/values.yaml
|
||||
--- charts/fluentd/values.yaml 2021-02-12 18:13:04.000000000 +0100
|
||||
+++ charts/fluentd.zdt/values.yaml 2021-03-09 17:04:51.438191456 +0100
|
||||
@@ -12,6 +12,9 @@
|
||||
pullPolicy: "IfNotPresent"
|
||||
tag: ""
|
||||
|
||||
+testFramework:
|
||||
+ enabled: false
|
||||
+
|
||||
## Optional array of imagePullSecrets containing private registry credentials
|
||||
## Ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
|
||||
imagePullSecrets: []
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
{{- if .Values.fluentd.enabled }}
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
type: Opaque
|
||||
metadata:
|
||||
name: {{ template "kubezero-lib.fullname" $ }}-fluentd-secret
|
||||
namespace: {{ .Release.Namespace }}
|
||||
labels:
|
||||
{{ include "kubezero-lib.labels" . | indent 4 }}
|
||||
data:
|
||||
shared_key: {{ .Values.fluentd.shared_key | b64enc | quote }}
|
||||
{{- end }}
|
||||
|
|
@ -1,8 +1,17 @@
|
|||
#!/bin/bash
|
||||
|
||||
FLUENT_BIT_VERSION=0.12.3
|
||||
FLUENTD_VERSION=0.2.2
|
||||
|
||||
# Fluent Bit
|
||||
rm -rf charts/fluent-bit
|
||||
curl -L -s -o - https://github.com/fluent/helm-charts/releases/download/fluent-bit-${FLUENT_BIT_VERSION}/fluent-bit-${FLUENT_BIT_VERSION}.tgz | tar xfz - -C charts
|
||||
|
||||
patch -i fluent-bit.patch -p0 --no-backup-if-mismatch
|
||||
|
||||
|
||||
# FluentD
|
||||
rm -rf charts/fluentd
|
||||
curl -L -s -o - https://github.com/fluent/helm-charts/releases/download/fluentd-${FLUENTD_VERSION}/fluentd-${FLUENTD_VERSION}.tgz | tar xfz - -C charts
|
||||
|
||||
patch -i fluentd.patch -p0 --no-backup-if-mismatch
|
||||
|
|
|
|||
|
|
@ -39,27 +39,16 @@ kibana:
|
|||
fluentd:
|
||||
enabled: false
|
||||
image:
|
||||
repository: quay.io/fluentd_elasticsearch/fluentd
|
||||
tag: v2.9.0
|
||||
repository: fluent/fluentd-kubernetes-daemonset
|
||||
tag: v1.12-debian-elasticsearch7-1
|
||||
istio:
|
||||
enabled: false
|
||||
|
||||
# broken as of 2.5.1 ;-(
|
||||
# useStatefulSet: true
|
||||
replicaCount: 2
|
||||
kind: Deployment
|
||||
replicaCount: 1
|
||||
|
||||
plugins:
|
||||
enabled: false
|
||||
pluginsList:
|
||||
#- fluent-plugin-detect-exceptions
|
||||
#- fluent-plugin-s3
|
||||
#- fluent-plugin-grok-parser
|
||||
|
||||
#persistence:
|
||||
# enabled: true
|
||||
# storageClass: "ebs-sc-gp2-xfs"
|
||||
# accessMode: ReadWriteOnce
|
||||
# size: 4Gi
|
||||
#plugins:
|
||||
#- fluent-plugin-s3
|
||||
|
||||
service:
|
||||
ports:
|
||||
|
|
@ -71,124 +60,140 @@ fluentd:
|
|||
containerPort: 9880
|
||||
|
||||
metrics:
|
||||
enabled: false
|
||||
serviceMonitor:
|
||||
enabled: true
|
||||
enabled: false
|
||||
additionalLabels:
|
||||
release: metrics
|
||||
|
||||
dashboards:
|
||||
enabled: false
|
||||
|
||||
# No need for docker nor /var/log
|
||||
volumes:
|
||||
- name: etcfluentd-main
|
||||
configMap:
|
||||
name: fluentd-main
|
||||
defaultMode: 0777
|
||||
- name: etcfluentd-config
|
||||
configMap:
|
||||
name: fluentd-config
|
||||
defaultMode: 0777
|
||||
volumeMounts:
|
||||
- name: etcfluentd-main
|
||||
mountPath: /etc/fluent
|
||||
- name: etcfluentd-config
|
||||
mountPath: /etc/fluent/config.d/
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
memory: 1024Mi
|
||||
|
||||
output:
|
||||
# Default should be "logging-kubezero-logging-es-http" if fullnameOverride is NOT used
|
||||
host: logging-es-http
|
||||
|
||||
shared_key: "cloudbender"
|
||||
|
||||
env:
|
||||
OUTPUT_USER: elastic
|
||||
OUTPUT_SSL_VERIFY: "false"
|
||||
# OUTPUT_USER: elastic
|
||||
# OUTPUT_SSL_VERIFY: "false"
|
||||
|
||||
env:
|
||||
- name: "FLUENTD_CONF"
|
||||
value: "../../etc/fluent/fluent.conf"
|
||||
# Same here the secret names change if fullnameOverride is not used !!
|
||||
extraEnvVars:
|
||||
- name: OUTPUT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: logging-es-elastic-user
|
||||
key: elastic
|
||||
- name: FLUENTD_SHARED_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: logging-fluentd-secret
|
||||
key: shared_key
|
||||
|
||||
configMaps:
|
||||
general.conf: |
|
||||
<label @FLUENT_LOG>
|
||||
<match **>
|
||||
@type null
|
||||
</match>
|
||||
</label>
|
||||
fileConfigs:
|
||||
00_system.conf: |-
|
||||
<system>
|
||||
workers 2
|
||||
</system>
|
||||
01_sources.conf: |-
|
||||
<source>
|
||||
@type http
|
||||
@label @KUBERNETES
|
||||
port 9880
|
||||
bind 0.0.0.0
|
||||
keepalive_timeout 30
|
||||
</source>
|
||||
<source>
|
||||
@type monitor_agent
|
||||
bind 0.0.0.0
|
||||
port 24220
|
||||
tag fluentd.monitor.metrics
|
||||
</source>
|
||||
|
||||
forward-input.conf: |
|
||||
<source>
|
||||
@type forward
|
||||
@label @KUBERNETES
|
||||
port 24224
|
||||
bind 0.0.0.0
|
||||
skip_invalid_event true
|
||||
# skip_invalid_event true
|
||||
send_keepalive_packet true
|
||||
<security>
|
||||
self_hostname "#{ENV['HOSTNAME']}"
|
||||
shared_key "#{ENV['FLUENTD_SHARED_KEY']}"
|
||||
shared_key {{ .Values.shared_key }}
|
||||
</security>
|
||||
</source>
|
||||
|
||||
output.conf: |
|
||||
<match **>
|
||||
@id elasticsearch
|
||||
@type elasticsearch
|
||||
@log_level info
|
||||
include_tag_key true
|
||||
id_key id
|
||||
remove_keys id
|
||||
02_filters.conf: |-
|
||||
<label @KUBERNETES>
|
||||
# prevent log feedback loops eg. ES has issues etc.
|
||||
# discard logs from our own pods
|
||||
<match kube.logging.fluentd>
|
||||
@type relabel
|
||||
@label @FLUENT_LOG
|
||||
</match>
|
||||
|
||||
# KubeZero pipeline incl. GeoIP etc.
|
||||
pipeline fluentd
|
||||
<match **>
|
||||
@type relabel
|
||||
@label @DISPATCH
|
||||
</match>
|
||||
</label>
|
||||
|
||||
hosts "#{ENV['OUTPUT_HOST']}"
|
||||
port "#{ENV['OUTPUT_PORT']}"
|
||||
scheme "#{ENV['OUTPUT_SCHEME']}"
|
||||
ssl_version "#{ENV['OUTPUT_SSL_VERSION']}"
|
||||
ssl_verify "#{ENV['OUTPUT_SSL_VERIFY']}"
|
||||
user "#{ENV['OUTPUT_USER']}"
|
||||
password "#{ENV['OUTPUT_PASSWORD']}"
|
||||
04_outputs.conf: |-
|
||||
<label @OUTPUT>
|
||||
<match **>
|
||||
@id elasticsearch
|
||||
@type elasticsearch
|
||||
@log_level info
|
||||
include_tag_key true
|
||||
id_key id
|
||||
remove_keys id
|
||||
|
||||
log_es_400_reason
|
||||
logstash_format true
|
||||
reconnect_on_error true
|
||||
reload_on_failure true
|
||||
request_timeout 60s
|
||||
suppress_type_name true
|
||||
# KubeZero pipeline incl. GeoIP etc.
|
||||
pipeline fluentd
|
||||
|
||||
<buffer tag>
|
||||
@type file_single
|
||||
path /var/log/fluentd-buffers/kubernetes.system.buffer
|
||||
chunk_limit_size 16MB
|
||||
total_limit_size 4GB
|
||||
flush_mode interval
|
||||
flush_thread_count 4
|
||||
flush_interval 5s
|
||||
flush_at_shutdown true
|
||||
retry_type exponential_backoff
|
||||
retry_timeout 60m
|
||||
overflow_action drop_oldest_chunk
|
||||
disable_chunk_backup true
|
||||
</buffer>
|
||||
</match>
|
||||
hosts "{{ .Values.output.host }}"
|
||||
port 9200
|
||||
scheme http
|
||||
user elastic
|
||||
password "#{ENV['OUTPUT_PASSWORD']}"
|
||||
|
||||
filter.conf: |
|
||||
<filter disabled.kube.**>
|
||||
@type parser
|
||||
key_name message
|
||||
remove_key_name_field true
|
||||
reserve_data true
|
||||
reserve_time true
|
||||
# inject_key_prefix message_json.
|
||||
emit_invalid_record_to_error false
|
||||
<parse>
|
||||
@type json
|
||||
</parse>
|
||||
</filter>
|
||||
log_es_400_reason
|
||||
logstash_format true
|
||||
reconnect_on_error true
|
||||
reload_on_failure true
|
||||
request_timeout 60s
|
||||
suppress_type_name true
|
||||
|
||||
<buffer tag>
|
||||
@type file_single
|
||||
path /var/log/fluentd-buffers/kubernetes.system.buffer
|
||||
chunk_limit_size 16MB
|
||||
total_limit_size 4GB
|
||||
flush_mode interval
|
||||
flush_thread_count 4
|
||||
flush_interval 5s
|
||||
flush_at_shutdown true
|
||||
retry_type exponential_backoff
|
||||
retry_timeout 60m
|
||||
overflow_action drop_oldest_chunk
|
||||
disable_chunk_backup true
|
||||
</buffer>
|
||||
</match>
|
||||
</label>
|
||||
|
||||
fluent-bit:
|
||||
enabled: false
|
||||
|
|
@ -213,7 +218,7 @@ fluent-bit:
|
|||
refreshInterval: 10
|
||||
|
||||
logLevel: warn
|
||||
flushInterval: 1
|
||||
flushInterval: 5
|
||||
|
||||
#extraRecords:
|
||||
# source.clustername: MyKubeCluster
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ apiVersion: v2
|
|||
name: kubezero
|
||||
description: KubeZero - Bootstrap and ArgoCD Root App of Apps chart
|
||||
type: application
|
||||
version: 0.5.0
|
||||
version: 0.5.1
|
||||
home: https://kubezero.com
|
||||
icon: https://cdn.zero-downtime.net/assets/kubezero/logo-small-64.png
|
||||
keywords:
|
||||
|
|
@ -15,4 +15,4 @@ dependencies:
|
|||
- name: kubezero-lib
|
||||
version: ">= 0.1.3"
|
||||
repository: https://zero-down-time.github.io/kubezero/
|
||||
kubeVersion: ">= 1.16.0"
|
||||
kubeVersion: ">= 1.18.0"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# kubezero
|
||||
|
||||
 
|
||||
 
|
||||
|
||||
KubeZero - Bootstrap and ArgoCD Root App of Apps chart
|
||||
|
||||
|
|
@ -14,7 +14,7 @@ KubeZero - Bootstrap and ArgoCD Root App of Apps chart
|
|||
|
||||
## Requirements
|
||||
|
||||
Kubernetes: `>= 1.16.0`
|
||||
Kubernetes: `>= 1.18.0`
|
||||
|
||||
| Repository | Name | Version |
|
||||
|------------|------|---------|
|
||||
|
|
@ -32,6 +32,7 @@ Kubernetes: `>= 1.16.0`
|
|||
| argocd.namespace | string | `"argocd"` | |
|
||||
| aws-ebs-csi-driver.enabled | bool | `false` | |
|
||||
| aws-efs-csi-driver.enabled | bool | `false` | |
|
||||
| aws-node-termination-handler.enabled | bool | `false` | |
|
||||
| calico.crds | bool | `true` | |
|
||||
| calico.enabled | bool | `false` | |
|
||||
| calico.retain | bool | `true` | |
|
||||
|
|
|
|||
|
|
@ -47,8 +47,10 @@ kibana:
|
|||
{{- if .Values.logging.fluentd }}
|
||||
fluentd:
|
||||
enabled: {{ .Values.logging.fluentd.enabled }}
|
||||
replicaCount: {{ default 1 .Values.logging.fluentd.replicaCount }}
|
||||
metrics:
|
||||
enabled: {{ .Values.metrics.enabled }}
|
||||
serviceMonitor:
|
||||
enabled: {{ .Values.metrics.enabled }}
|
||||
url: {{ .Values.logging.fluentd.url }}
|
||||
{{- if .Values.logging.fluentd.output }}
|
||||
output:
|
||||
|
|
|
|||
|
|
@ -1,5 +1,3 @@
|
|||
argo: {}
|
||||
|
||||
global:
|
||||
kubezero:
|
||||
server: https://kubernetes.default.svc
|
||||
|
|
@ -65,3 +63,5 @@ argocd:
|
|||
namespace: argocd
|
||||
istio:
|
||||
enabled: false
|
||||
|
||||
argo: {}
|
||||
|
|
|
|||
Loading…
Reference in New Issue