ZeroDownTime/KubeZero
3
0
Fork 0
Code Issues 1 Pull Requests 21 Packages Projects Releases Wiki Activity
2783d6da21
KubeZero/charts/kubezero-logging/README.md

124 lines
6.8 KiB
Markdown
Raw Normal View History

Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
# kubezero-logging
Version bump for aws-ebs-csi and kiam, ES bugfix bump, fluentd tuning
2021-01-21 12:31:06 +00:00
![Version: 0.5.3](https://img.shields.io/badge/Version-0.5.3-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.3.1](https://img.shields.io/badge/AppVersion-1.3.1-informational?style=flat-square)
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
KubeZero Umbrella Chart for complete EFK stack
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
**Homepage:** <https://kubezero.com>
## Maintainers
| Name | Email | Url |
| ---- | ------ | --- |
| Quarky9 | | |
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
## Requirements
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
Kubernetes: `>= 1.16.0`
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
| Repository | Name | Version |
|------------|------|---------|
Version bump for aws-ebs-csi and kiam, ES bugfix bump, fluentd tuning
2021-01-21 12:31:06 +00:00
| | fluent-bit | 0.7.14 |
| | fluentd | 2.5.3 |
More logging tuning
2020-12-10 14:36:26 +00:00
| https://helm.elastic.co | eck-operator | 1.3.1 |
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
| https://zero-down-time.github.io/kubezero/ | kubezero-lib | >= 0.1.3 |
## Changes from upstream
### ECK
- Operator mapped to controller nodes
Bugfix release for Calico, README updates
2020-08-14 16:05:25 +00:00
### ES
- SSL disabled ( Todo: provide cluster certs and setup Kibana/Fluentd to use https incl. client certs )
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
- Installed Plugins:
Bugfix release for Calico, README updates
2020-08-14 16:05:25 +00:00
- repository-s3
- elasticsearch-prometheus-exporter
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
- [Cross AZ Zone awareness](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html#k8s-availability-zone-awareness) is implemented via nodeSets
Bugfix release for Calico, README updates
2020-08-14 16:05:25 +00:00
### Kibana
- increased timeout to ES to 3 minutes
Fluent-bit version bump and support for api audit logs
2020-09-08 11:40:28 +00:00
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
### FluentD
Fluent-bit version bump and support for api audit logs
2020-09-08 11:40:28 +00:00
### Fluent-bit
- support for dedot Lua filter to replace "." with "_" for all annotations and labels
- support for api audit log
Bugfix release for Calico, README updates
2020-08-14 16:05:25 +00:00
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
## Manual tasks ATM
Fluent-bit version bump and support for api audit logs
2020-09-08 11:40:28 +00:00
- install index template
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
- setup Kibana
- create `logstash-*` Index Pattern
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
## Values
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
| Key | Type | Default | Description |
|-----|------|---------|-------------|
Integrare fluent-bit into logging to allow better config
2020-12-08 15:05:25 +00:00
| eck-operator.enabled | bool | `false` | |
| eck-operator.installCRDs | bool | `false` | |
First try adding argoCD day 2
2020-11-24 14:44:57 +00:00
| eck-operator.nodeSelector."node-role.kubernetes.io/master" | string | `""` | |
| eck-operator.tolerations[0].effect | string | `"NoSchedule"` | |
| eck-operator.tolerations[0].key | string | `"node-role.kubernetes.io/master"` | |
EBS-CSI version bump, reduce ArgoCD concurrency to reduce load spikes, sync from 180s to 300s
2020-08-15 22:37:45 +00:00
| elastic_password | string | `""` | |
Bugfix release for Calico, README updates
2020-08-14 16:05:25 +00:00
| es.nodeSets | list | `[]` | |
| es.prometheus | bool | `false` | |
| es.s3Snapshot.enabled | bool | `false` | |
| es.s3Snapshot.iamrole | string | `""` | |
Integrare fluent-bit into logging to allow better config
2020-12-08 15:05:25 +00:00
| fluent-bit.config.flushInterval | int | `1` | |
| fluent-bit.config.input.memBufLimit | string | `"16MB"` | |
| fluent-bit.config.input.refreshInterval | int | `10` | |
| fluent-bit.config.logLevel | string | `"warn"` | |
| fluent-bit.config.output.host | string | `"logging-fluentd"` | |
| fluent-bit.config.output.sharedKey | string | `"cloudbender"` | |
| fluent-bit.config.output.tls | bool | `false` | |
Move scrape username to its own secret as eck operator cleans up otherwise
2020-09-04 00:13:39 +00:00
| fluent-bit.enabled | bool | `false` | |
Update docs
2020-08-22 17:27:31 +00:00
| fluent-bit.serviceMonitor.enabled | bool | `true` | |
| fluent-bit.serviceMonitor.selector.release | string | `"metrics"` | |
| fluent-bit.test.enabled | bool | `false` | |
| fluent-bit.tolerations[0].effect | string | `"NoSchedule"` | |
| fluent-bit.tolerations[0].key | string | `"node-role.kubernetes.io/master"` | |
Update docs, bump argo-cd parallel jobs
2020-10-27 11:54:44 +00:00
| fluentd.configMaps."filter.conf" | string | `"<filter disabled.kube.**>\n @type parser\n key_name message\n remove_key_name_field true\n reserve_data true\n reserve_time true\n # inject_key_prefix message_json.\n emit_invalid_record_to_error false\n <parse>\n @type json\n </parse>\n</filter>\n"` | |
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
| fluentd.configMaps."forward-input.conf" | string | `"<source>\n @type forward\n port 24224\n bind 0.0.0.0\n skip_invalid_event true\n send_keepalive_packet true\n <security>\n self_hostname \"#{ENV['HOSTNAME']}\"\n shared_key \"#{ENV['FLUENTD_SHARED_KEY']}\"\n </security>\n</source>\n"` | |
| fluentd.configMaps."general.conf" | string | `"<label @FLUENT_LOG>\n <match **>\n @type null\n </match>\n</label>\n<source>\n @type http\n port 9880\n bind 0.0.0.0\n keepalive_timeout 30\n</source>\n<source>\n @type monitor_agent\n bind 0.0.0.0\n port 24220\n tag fluentd.monitor.metrics\n</source>\n"` | |
Version bump for aws-ebs-csi and kiam, ES bugfix bump, fluentd tuning
2021-01-21 12:31:06 +00:00
| fluentd.configMaps."output.conf" | string | `"<match **>\n @id elasticsearch\n @type elasticsearch\n @log_level info\n include_tag_key true\n id_key id\n remove_keys id\n\n # KubeZero pipeline incl. GeoIP etc.\n pipeline fluentd\n\n hosts \"#{ENV['OUTPUT_HOST']}\"\n port \"#{ENV['OUTPUT_PORT']}\"\n scheme \"#{ENV['OUTPUT_SCHEME']}\"\n ssl_version \"#{ENV['OUTPUT_SSL_VERSION']}\"\n ssl_verify \"#{ENV['OUTPUT_SSL_VERIFY']}\"\n user \"#{ENV['OUTPUT_USER']}\"\n password \"#{ENV['OUTPUT_PASSWORD']}\"\n\n log_es_400_reason\n logstash_format true\n reconnect_on_error true\n reload_on_failure true\n request_timeout 60s\n suppress_type_name true\n\n <buffer tag>\n @type file_single\n path /var/log/fluentd-buffers/kubernetes.system.buffer\n chunk_limit_records 8192\n total_limit_size 4GB\n flush_mode interval\n flush_thread_count 2\n flush_interval 30s\n flush_at_shutdown true\n retry_type exponential_backoff\n retry_timeout 60m\n overflow_action drop_oldest_chunk\n disable_chunk_backup true\n </buffer>\n</match>\n"` | |
Disable default syncPolicy, use values instead
2020-08-20 10:40:08 +00:00
| fluentd.enabled | bool | `false` | |
| fluentd.env.OUTPUT_SSL_VERIFY | string | `"false"` | |
| fluentd.env.OUTPUT_USER | string | `"elastic"` | |
| fluentd.extraEnvVars[0].name | string | `"OUTPUT_PASSWORD"` | |
| fluentd.extraEnvVars[0].valueFrom.secretKeyRef.key | string | `"elastic"` | |
| fluentd.extraEnvVars[0].valueFrom.secretKeyRef.name | string | `"logging-es-elastic-user"` | |
| fluentd.extraEnvVars[1].name | string | `"FLUENTD_SHARED_KEY"` | |
| fluentd.extraEnvVars[1].valueFrom.secretKeyRef.key | string | `"shared_key"` | |
| fluentd.extraEnvVars[1].valueFrom.secretKeyRef.name | string | `"logging-fluentd-secret"` | |
Move scrape username to its own secret as eck operator cleans up otherwise
2020-09-04 00:13:39 +00:00
| fluentd.image.repository | string | `"quay.io/fluentd_elasticsearch/fluentd"` | |
Updated helm-docs, fluentd SSL handled by Istio, ES&Istio tuning
2020-10-05 10:50:23 +00:00
| fluentd.image.tag | string | `"v2.9.0"` | |
Disable default syncPolicy, use values instead
2020-08-20 10:40:08 +00:00
| fluentd.istio.enabled | bool | `false` | |
| fluentd.metrics.enabled | bool | `false` | |
| fluentd.metrics.serviceMonitor.additionalLabels.release | string | `"metrics"` | |
| fluentd.metrics.serviceMonitor.enabled | bool | `true` | |
| fluentd.output.host | string | `"logging-es-http"` | |
| fluentd.plugins.enabled | bool | `false` | |
| fluentd.plugins.pluginsList | string | `nil` | |
| fluentd.replicaCount | int | `2` | |
| fluentd.service.ports[0].containerPort | int | `24224` | |
| fluentd.service.ports[0].name | string | `"tcp-forward"` | |
| fluentd.service.ports[0].protocol | string | `"TCP"` | |
| fluentd.service.ports[1].containerPort | int | `9880` | |
| fluentd.service.ports[1].name | string | `"http-fluentd"` | |
| fluentd.service.ports[1].protocol | string | `"TCP"` | |
| fluentd.shared_key | string | `"cloudbender"` | |
Bugfix release for Calico, README updates
2020-08-14 16:05:25 +00:00
| kibana.count | int | `1` | |
| kibana.istio.enabled | bool | `false` | |
| kibana.istio.gateway | string | `"istio-system/ingressgateway"` | |
| kibana.istio.url | string | `""` | |
Version bump for aws-ebs-csi and kiam, ES bugfix bump, fluentd tuning
2021-01-21 12:31:06 +00:00
| version | string | `"7.10.2"` | |
Add rabbitmq ingress gateway def
2020-08-11 14:09:48 +00:00
## Resources:
- https://www.elastic.co/downloads/elastic-cloud-kubernetes
- https://github.com/elastic/cloud-on-k8s
Calico version bump to 3.16.1
2020-09-09 13:17:02 +00:00
- https://grafana.com/grafana/dashboards/7752
Reference in New Issue Copy Permalink