ZeroDownTime/KubeZero
3
0
Fork 0
Code Issues 1 Pull Requests 21 Packages Projects Releases Wiki Activity

feat: new wrapper chart for manticoresearch

Browse Source
This commit is contained in:
Stefan Reimer 2022-08-10 14:01:00 +00:00
parent 5d3fae12cf
commit 69d1778492
15 changed files with 671 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
charts/manticore/.helmignore Normal file
View File

@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

21
charts/manticore/Chart.yaml Normal file
View File

@ -0,0 +1,21 @@
apiVersion: v2
name: manticore
description: Chart for Manticore
type: application
version: 5.0.02
appVersion: 5.0.02
home: https://kubezero.com
icon: https://cdn.zero-downtime.net/assets/kubezero/logo-small-64.png
keywords:
- kubezero
- manticore
maintainers:
- name: Stefan Reimer
email: stefan@zero-downtime.net
dependencies:
- name: kubezero-lib
version: ">= 0.1.4"
repository: https://cdn.zero-downtime.net/charts
- name: manticoresearch
version: "=5.0.02"
kubeVersion: ">= 1.20.0"

42
charts/manticore/README.md Normal file
View File

@ -0,0 +1,42 @@
# clamav
![Version: 0.1.1](https://img.shields.io/badge/Version-0.1.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.104.0](https://img.shields.io/badge/AppVersion-0.104.0-informational?style=flat-square)
Chart for deploying a ClamavD on kubernetes as statfulSet
**Homepage:** <https://kubezero.com>
## Maintainers
| Name | Email | Url |
| ---- | ------ | --- |
| Quarky9 | | |
## Requirements
Kubernetes: `>= 1.18.0`
| Repository | Name | Version |
|------------|------|---------|
| https://cdn.zero-downtime.net/charts/ | kubezero-lib | >= 0.1.4 |
## Values
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| clamav.freshclam.mirrors | string | `"database.clamav.net"` | A list of clamav mirrors to be used by the clamav service |
| clamav.image | string | `"clamav/clamav"` | The clamav docker image |
| clamav.limits.connectionQueueLength | int | `100` | Maximum length the queue of pending connections may grow to |
| clamav.limits.fileSize | int | `20` | The largest file size scanable by clamav, in MB |
| clamav.limits.maxThreads | int | `4` | Maximum number of threads running at the same time. |
| clamav.limits.scanSize | int | `100` | The largest scan size permitted in clamav, in MB |
| clamav.limits.sendBufTimeout | int | `500` | |
| clamav.replicaCount | int | `1` | |
| clamav.resources | object | `{"requests":{"cpu":"300m","memory":"1300M"}}` | The resource requests and limits for the clamav service |
| clamav.version | string | `"unstable"` | The clamav docker image version - defaults to .Chart.appVersion |
| fullnameOverride | string | `""` | override the full name of the clamav chart |
| nameOverride | string | `""` | override the name of the clamav chart |
| service.port | int | `3310` | The port to be used by the clamav service |
----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.9.1](https://github.com/norwoodj/helm-docs/releases/v1.9.1)

23
charts/manticore/charts/manticoresearch/Chart.yaml Normal file
View File

@ -0,0 +1,23 @@
apiVersion: v2
name: manticoresearch
description: A Helm chart for Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 5.0.02
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 5.0.0.2

37
charts/manticore/charts/manticoresearch/templates/NOTES.txt Normal file
View File

@ -0,0 +1,37 @@
1) Wait until all pods are moved to running state:
kubectl --namespace {{ .Release.Namespace }} get po
2) Forward worker port to your local machine:
export WORKER_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "manticoresearch.name" . }},app.kubernetes.io/instance={{ .Release.Name }},label=manticore-worker" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace {{ .Release.Namespace }} port-forward $WORKER_NAME 7306:{{ .Values.worker.service.ql.port }}
3) Connect to the worker:
mysql -h0 -P7306
4) Create your first table:
CREATE TABLE idx(title text);
5) Add table to cluster:
ALTER CLUSTER {{ .Values.worker.clusterName }} ADD idx;
6) Add some data to your table:
INSERT INTO {{ .Values.worker.clusterName }}:idx (title) VALUES ('dog is brown'), ('mouse is small');
7) Forward worker port to your local machine:
export BALANCER_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "manticoresearch.name" . }},app.kubernetes.io/instance={{ .Release.Name }},name={{ include "manticoresearch.fullname" . }}-balancer" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace {{ .Release.Namespace }} port-forward $BALANCER_NAME 6306:{{ .Values.balancer.service.ql.port }}
8) Search through the balancer:
mysql -h0 -P6306 -e "SELECT * FROM idx WHERE match('dog')"
9) Scale the cluster:
kubectl scale statefulsets manticore-manticoresearch-worker -n {{ .Release.Namespace }} --replicas=5

58
charts/manticore/charts/manticoresearch/templates/_helpers.tpl Normal file
View File

@ -0,0 +1,58 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "manticoresearch.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "manticoresearch.fullname" -}}
{{- if .Values.fullNameOverride }}
{{- .Values.fullNameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "manticoresearch.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "manticoresearch.labels" -}}
helm.sh/chart: {{ include "manticoresearch.chart" . }}
{{ include "manticoresearch.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "manticoresearch.selectorLabels" -}}
app.kubernetes.io/name: {{ include "manticoresearch.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "manticoresearch.serviceAccountName" -}}
{{- default (include "manticoresearch.fullname" .) .Values.serviceAccount.name }}
{{- end }}

10
charts/manticore/charts/manticoresearch/templates/balancer-config-map.yaml Normal file
View File

@ -0,0 +1,10 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "manticoresearch.fullname" . }}-balancer-config
labels:
{{- include "manticoresearch.labels" . | nindent 4 }}
data:
manticore.conf:
{{ toYaml .Values.balancer.config.content | indent 4 }}

111
charts/manticore/charts/manticoresearch/templates/manticore-balancer.yaml Normal file
View File

@ -0,0 +1,111 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "manticoresearch.fullname" . }}-balancer
labels:
name: {{ include "manticoresearch.fullname" . }}-balancer
{{- include "manticoresearch.labels" . | nindent 4 }}
spec:
replicas: 1
selector:
matchLabels:
name: {{ include "manticoresearch.fullname" . }}-balancer
template:
metadata:
{{- if or .Values.exporter.enabled .Values.podAnnotations }}
annotations:
{{- if .Values.exporter.enabled }}
{{- toYaml .Values.exporter.annotations | nindent 8 }}
{{- end }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
labels:
name: {{ include "manticoresearch.fullname" . }}-balancer
{{- include "manticoresearch.selectorLabels" . | nindent 8 }}
spec:
serviceAccountName: {{ include "manticoresearch.serviceAccountName" . }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}-balancer
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.balancer.image.repository }}:{{ .Values.balancer.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.balancer.image.pullPolicy }}
env:
- name: OBSERVER_RUN_INTERVAL
value: "{{ .Values.balancer.runInterval }}"
{{- if .Values.optimize.enabled }}
- name: OPTIMIZE_RUN_INTERVAL
value: "{{ .Values.optimize.interval }}"
- name: CHUNKS_COEFFICIENT
value: "{{ .Values.optimize.coefficient }}"
{{- end }}
- name: WORKER_LABEL
value: "manticore-worker"
- name: CONFIGMAP_PATH
value: {{ .Values.balancer.config.path }}
- name: BALANCER_PORT
value: "{{ .Values.balancer.service.ql.port }}"
- name: WORKER_PORT
value: "{{ .Values.worker.service.ql.port }}"
readinessProbe:
tcpSocket:
port: {{ .Values.balancer.service.ql.port }}
initialDelaySeconds: 5
periodSeconds: 3
livenessProbe:
tcpSocket:
port: {{ .Values.balancer.service.ql.port }}
initialDelaySeconds: 5
periodSeconds: 3
volumeMounts:
- name: config-volume
mountPath: {{ .Values.balancer.config.path }}
subPath: manticore.conf
resources:
{{- toYaml .Values.resources | nindent 10 }}
{{- if .Values.exporter.enabled }}
- name: {{ .Chart.Name }}-exporter
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.exporter.image.repository }}:{{ .Values.exporter.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.exporter.image.pullPolicy }}
env:
- name: MANTICORE_HOST
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: MANTICORE_PORT
value: "{{ .Values.balancer.service.ql.targetPort }}"
livenessProbe:
httpGet:
path: /health
port: {{ index .Values.exporter.annotations "prometheus.io/port" }}
initialDelaySeconds: 3
periodSeconds: 3
resources:
{{- toYaml .Values.resources | nindent 10 }}
{{- end }}
volumes:
- name: config-volume
configMap:
name: {{ include "manticoresearch.fullname" . }}-balancer-config
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

126
charts/manticore/charts/manticoresearch/templates/manticore-worker.yaml Normal file
View File

@ -0,0 +1,126 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "manticoresearch.fullname" . }}-worker
labels:
name: {{ include "manticoresearch.fullname" . }}-worker
{{- include "manticoresearch.labels" . | nindent 4 }}
label: "{{ include "manticoresearch.fullname" . }}-worker"
spec:
replicas: {{ .Values.worker.replicaCount }}
selector:
matchLabels:
name: {{ include "manticoresearch.fullname" . }}-worker
serviceName: {{ include "manticoresearch.fullname" . }}-worker-svc
template:
metadata:
{{- if or .Values.exporter.enabled .Values.podAnnotations }}
annotations:
{{- if .Values.exporter.enabled }}
{{- toYaml .Values.exporter.annotations | nindent 8 }}
{{- end }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
labels:
name: "{{ include "manticoresearch.fullname" . }}-worker"
label: "manticore-worker"
{{- include "manticoresearch.selectorLabels" . | nindent 8 }}
spec:
serviceAccountName: {{ include "manticoresearch.serviceAccountName" . }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}-worker
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: {{ .Values.worker.image.repository }}:{{ .Values.worker.image.tag | default .Chart.AppVersion }}
imagePullPolicy: {{ .Values.worker.image.pullPolicy }}
lifecycle:
preStop:
exec:
command: [ "/bin/sh", "-c", "./shutdown.sh" ]
env:
- name: WORKER_LABEL
value: "manticore-worker"
- name: AUTO_ADD_TABLES_IN_CLUSTER
value: "{{ .Values.worker.autoAddTablesInCluster }}"
- name: CONFIGMAP_PATH
value: {{ .Values.worker.config.path }}
- name: MANTICORE_PORT
value: "{{ .Values.worker.service.ql.port }}"
- name: CLUSTER_NAME
value: {{ .Values.worker.clusterName }}
- name: BALANCER_URL
value: {{ include "manticoresearch.fullname" . }}-balancer-svc:{{ .Values.balancer.service.ql.targetPort }}
- name: WORKER_SERVICE
value: {{ include "manticoresearch.fullname" . }}-worker-svc
readinessProbe:
tcpSocket:
port: {{ .Values.worker.service.ql.port }}
initialDelaySeconds: 5
periodSeconds: 3
livenessProbe:
tcpSocket:
port: {{ .Values.worker.service.ql.port }}
initialDelaySeconds: 5
periodSeconds: 3
volumeMounts:
- name: data
mountPath: /var/lib/manticore/
- name: config-volume
mountPath: {{ .Values.worker.config.path }}
subPath: manticore.conf
resources:
{{- toYaml .Values.resources | nindent 10 }}
{{- if .Values.exporter.enabled }}
- name: {{ .Chart.Name }}-exporter
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.exporter.image.repository }}:{{ .Values.exporter.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.exporter.image.pullPolicy }}
env:
- name: MANTICORE_HOST
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: MANTICORE_PORT
value: "{{ .Values.balancer.service.ql.targetPort }}"
livenessProbe:
httpGet:
path: /health
port: {{ index .Values.exporter.annotations "prometheus.io/port" }}
initialDelaySeconds: 3
periodSeconds: 3
resources:
{{- toYaml .Values.resources | nindent 10 }}
{{- end }}
volumes:
- name: config-volume
configMap:
name: {{ include "manticoresearch.fullname" . }}-worker-config
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: {{ .Values.worker.volume.size }}

35
charts/manticore/charts/manticoresearch/templates/rolebinding.yaml Normal file
View File

@ -0,0 +1,35 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "manticoresearch.serviceAccountName" . }}
labels:
{{- include "manticoresearch.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "manticoresearch.fullname" . }}-{{ .Release.Namespace }}
rules:
- apiGroups: [ '' ]
resources: [ 'nodes', 'configmaps', 'pods', 'services' ]
verbs: [ "get", "watch", "list" ]
- apiGroups: [ 'apps' ]
resources: [ 'deployments', 'statefulsets' ]
verbs: [ "get", "watch", "list" ]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "manticoresearch.fullname" . }}-{{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "manticoresearch.fullname" . }}-{{ .Release.Namespace }}
subjects:
- kind: ServiceAccount
namespace: {{ .Release.Namespace }}
name: {{ include "manticoresearch.serviceAccountName" . }}

20
charts/manticore/charts/manticoresearch/templates/service-balancer.yaml Normal file
View File

@ -0,0 +1,20 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "manticoresearch.fullname" . }}-balancer-svc
labels:
name: {{ include "manticoresearch.fullname" . }}-balancer-svc
{{- include "manticoresearch.labels" . | nindent 4 }}
spec:
ports:
- port: {{ .Values.balancer.service.ql.port }}
targetPort: {{ .Values.balancer.service.ql.targetPort }}
name: balancer
- port: {{ .Values.balancer.service.observer.port }}
targetPort: {{ .Values.balancer.service.observer.targetPort }}
name: observer
- port: {{ .Values.balancer.service.http.port }}
targetPort: {{ .Values.balancer.service.http.targetPort }}
name: http
selector:
name: {{ include "manticoresearch.fullname" . }}-balancer

18
charts/manticore/charts/manticoresearch/templates/service-worker.yaml Normal file
View File

@ -0,0 +1,18 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "manticoresearch.fullname" . }}-worker-svc
labels:
name: {{ include "manticoresearch.fullname" . }}-worker-svc
{{- include "manticoresearch.labels" . | nindent 4 }}
spec:
clusterIP: "None"
ports:
- port: {{ .Values.worker.service.ql.port }}
targetPort: {{ .Values.worker.service.ql.targetPort }}
name: ql
- port: {{ .Values.worker.service.http.port }}
targetPort: {{ .Values.worker.service.http.targetPort }}
name: http
selector:
name: {{ include "manticoresearch.fullname" . }}-worker

10
charts/manticore/charts/manticoresearch/templates/worker-config-map.yaml Normal file
View File

@ -0,0 +1,10 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "manticoresearch.fullname" . }}-worker-config
labels:
{{- include "manticoresearch.labels" . | nindent 4 }}
data:
manticore.conf:
{{ toYaml .Values.worker.config.content | indent 4 }}

132
charts/manticore/charts/manticoresearch/values.yaml Normal file
View File

@ -0,0 +1,132 @@
# Default values for manticoresearch.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
balancer:
runInterval: 5
image:
repository: manticoresearch/helm-balancer
# tag: 3.6.0.0
pullPolicy: IfNotPresent
service:
ql:
port: 9306
targetPort: 9306
observer:
port: 8080
targetPort: 8080
http:
port: 9308
targetPort: 9308
config:
path: /etc/manticoresearch/configmap.conf
content: |
searchd
{
listen = /var/run/mysqld/mysqld.sock:mysql
listen = 9306:mysql
listen = 9308:http
log = /dev/stdout
query_log = /dev/stdout
query_log_format = sphinxql
pid_file = /var/run/manticore/searchd.pid
binlog_path = /var/lib/manticore/data
}
worker:
replicaCount: 3
clusterName: manticore
autoAddTablesInCluster: true
image:
repository: manticoresearch/helm-worker
# tag: 3.6.0.0
pullPolicy: IfNotPresent
service:
ql:
port: 9306
targetPort: 9306
http:
port: 9308
targetPort: 9308
volume:
size: 1Gi
config:
path: /etc/manticoresearch/configmap.conf
content: |
searchd
{
listen = /var/run/mysqld/mysqld.sock:mysql
listen = 9306:mysql
listen = 9308:http
listen = 9301:mysql_vip
listen = $ip:9312
listen = $ip:9315-9415:replication
binlog_path = /var/lib/manticore/data
log = /dev/stdout
query_log = /dev/stdout
query_log_format = sphinxql
pid_file = /var/run/manticore/searchd.pid
data_dir = /var/lib/manticore
shutdown_timeout = 25s
auto_optimize = 0
}
exporter:
enabled: false
image:
repository: manticoresearch/prometheus-exporter
pullPolicy: IfNotPresent
# tag: 3.6.0.0
annotations:
prometheus.io/path: /metrics
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
optimize:
enabled: true
interval: "30"
coefficient: "2"
imagePullSecrets: []
nameOverride: ""
fullNameOverride: ""
serviceAccount:
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: "manticore-sa"
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
nodeSelector: {}
tolerations: []
affinity: {}

5
charts/manticore/values.yaml Normal file
View File

@ -0,0 +1,5 @@
manticoresearch:
worker:
replicaCount: 1
volume:
size: 4Gi