什么是MongoDB
MongoDB是一種非關系型數據庫管理系統(NoSQL),它被設計用于存儲和檢索大量數據,特別適用于處理大數據和實時應用程序。MongoDB的名稱源自"humongous"(巨大的)這個詞,它強調了MongoDB的優勢,即能夠輕松處理大規模的數據。以下是MongoDB的一些關鍵特點和概念:
- 非關系型數據存儲:與傳統的關系型數據庫不同,MongoDB不使用表格結構,而是采用文檔存儲模型。數據以文檔的形式存儲,通常使用JSON或類似的格式,這使得MongoDB更加靈活,可以存儲各種不同結構的數據。
- 高度可擴展:MongoDB是一個分布式數據庫系統,可以在多臺服務器上運行。它支持水平擴展,可以通過添加更多的服務器來增加存儲容量和處理能力,以應對大規模數據和高負載的需求。
- 強大的查詢語言:MongoDB提供了豐富的查詢和索引功能,使您可以輕松地檢索數據、篩選數據和執行聚合操作。
- 高性能:MongoDB的設計目標之一是提供高性能的讀寫操作。它使用內存映射文件來加速數據訪問,同時支持垂直分割和水平分割以提高性能。
- 復制和故障容忍性:MongoDB支持數據復制,可以在多個服務器之間復制數據,以提供冗余和故障容忍性。如果一個服務器出現故障,系統可以自動切換到備用服務器。
- 開源和活躍的社區:MongoDB是開源的,擁有龐大的社區支持和活躍的開發團隊。這意味著它不僅免費,還有許多可用的工具和擴展,以滿足各種不同的需求。
MongoDB廣泛用于各種應用程序,包括Web應用程序、大數據分析、物聯網(IoT)應用程序、日志管理和許多其他領域,因為它的靈活性和可擴展性使其適用于各種不同類型的數據存儲需求。
實現背景
在實際企業環境中,MongoDB需要多節點部署組成一個集群,可用于確保高可用性、容量擴展、負載均衡和數據備份。此時需要對整個MongoDB集群進行監控,持續關注集群的健康狀況。因此我們使用MongoDB Exporter將指標轉換為Prometheus的數據類型。最后通過Prometheus進行收集Mongodb集群的監控指標,并采用Grafana可視化。
創建StorageClasses
基于NFS創建StorageClasses,作為Mongodb集群持久化存儲。
# 創建StorageClasses
$ kubectl apply -f sc.yml
storageclass.storage.k8s.io/kubesre-nfs created
# 查看StorageClasses
$ kubectl get storageclasses.storage.k8s.io
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
kubesre-nfs example.com/external-nfs Delete Immediate false 63s
standard (default) rancher.io/local-path Delete WaitForFirstConsumer false 13d
MaongoDB集群部署
通過Helm方式進行部署MongoDB集群
# 添加Helm倉庫
$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories
# 搜索 MangoDB
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "higress.io" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
Update Complete. ?Happy Helming!?
$ helm search repo mongodb
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/mongodb 13.18.4 6.0.10 MongoDB(R) is a relational open source NoSQL da...
bitnami/mongodb-sharded 6.6.6 6.0.10 MongoDB(R) is an open source NoSQL database tha...
# 將MongoDB Chat下載到本地
$ mkdir mongodb && cd mongodb
$ helm pull bitnami/mongodb
$ tar zxf mongodb-13.18.4.tgz
$ cp mongodb/values.yaml ./values-test.yaml
# 修改values-test.yaml
$ cat values-test.yaml
## 配置文件中定義 storageClass: "",會使用集群配置的 openebs 提供的 storageClass,
## 使用此文檔部署,需要自行解決 storageClass 問題 (ceph, nfs, 公有云提供的 nfs)
global:
# 定義 storageClass 使用的類型
storageClass: "nfs-client"
# 定義 mongodb 集群為副本集模式
architecture: replicaset
# 啟動集群認證功能,設置超級管理員賬戶密碼
auth:
enabled: true
rootUser: root
rootPassword: "root"
# 設置集群數量,3個
replicaCount: 3
# 啟用持久化存儲,使用 global.storageClass 自動創建 pvc
persistence:
enabled: true
size: 20Gi
# 安裝MongoDB集群
helm install mongodb-cluster mongodb -f ./values-test.yaml
NAME: mongodb-cluster
LAST DEPLOYED: Tue Sep 19 15:54:36 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mongodb
CHART VERSION: 13.18.4
APP VERSION: 6.0.10
** Please be patient while the chart is being deployed **
MongoDB? can be accessed on the following DNS name(s) and ports from within your cluster:
mongodb-cluster-0.mongodb-cluster-headless.default.svc.cluster.local:27017
mongodb-cluster-1.mongodb-cluster-headless.default.svc.cluster.local:27017
mongodb-cluster-2.mongodb-cluster-headless.default.svc.cluster.local:27017
To get the root password run:
export MONGODB_ROOT_PASSWORD=$(kubectl get secret --namespace default mongodb-cluster -o jsonpath="{.data.mongodb-root-password}" | base64 -d)
To connect to your database, create a MongoDB? client container:
kubectl run --namespace default mongodb-cluster-client --rm --tty -i --restart='Never' --env="MONGODB_ROOT_PASSWORD=$MONGODB_ROOT_PASSWORD" --image docker.io/bitnami/mongodb:6.0.10-debian-11-r0 --command -- bash
Then, run the following command:
mongosh admin --host "mongodb-cluster-0.mongodb-cluster-headless.default.svc.cluster.local:27017,mongodb-cluster-1.mongodb-cluster-headless.default.svc.cluster.local:27017,mongodb-cluster-2.mongodb-cluster-headless.default.svc.cluster.local:27017" --authenticationDatabase admin -u $MONGODB_ROOT_USER -p $MONGODB_ROOT_PASSWORD
# 查看運行狀態
$ kubectl get pods | grep mongo
mongodb-cluster-0 1/1 Running 0 17m
mongodb-cluster-1 1/1 Running 0 6m42s
mongodb-cluster-2 1/1 Running 0 4m29s
mongodb-cluster-arbiter-0 1/1 Running 4 (7m51s ago) 20m
MongoDB Exporter 部署
接下來部署Mongodb Exporter:
# 部署Mongodb Exporter
$ cat exporter.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongo-expoter
spec:
template:
metadata:
labels:
app: mongo-expoter
spec:
containers:
- args:
- '--web.listen-address=:9104'
- '--mongodb.uri'
- >-
mongodb://mongodb-cluster-0.mongodb-cluster-headless.default.svc.cluster.local:27017,mongodb-cluster-1.mongodb-cluster-headless.default.svc.cluster.local:27017,mongodb-cluster-2.mongodb-cluster-headless.default.svc.cluster.local:27017/admin?authSource=admin
image: 'percona/mongodb_exporter:0.39.0'
imagePullPolicy: Always
name: mongo-expoter
resources:
requests:
cpu: 250m
memory: 512Mi
$ kubectl apply -f exporter.yml
# 部署Mongodb Exporter Service
$ cat service.yml
apiVersion: v1
kind: Service
metadata:
name: mongo-exporter
spec:
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- port: 9104
protocol: TCP
targetPort: 9104
selector:
app: mongo-exporter
type: ClusterIP
$ kubectl apply -f service.yml
到此Exporter部署完畢!
Prometheus安裝
接下來需要創建一個Configmap 存儲Prometheus 的配置映射:
$ cat prometheus-cm.yml
apiVersion: v1
data:
prometheus.yml: |-
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: "exporter"
static_configs:
- targets: ["mongo-exporter:9104"]
kind: ConfigMap
name: prometheus
$ kubectl apply -f prometheus-cm.yml
# 部署Prometheus
$ cat prometheus.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus
spec:
template:
metadata:
labels:
app: prometheus
name: prometheus-pod
spec:
containers:
- image: prom/prometheus
imagePullPolicy: IfNotPresent
name: prometheus
ports:
- containerPort: 9090
name: metrics
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/prometheus
name: prometheus-config
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 15
volumes:
- configMap:
defaultMode: 420
items:
- key: prometheus.yml
path: prometheus.yml
name: prometheus
name: prometheus-config
# 創建Prometheus Service
$ cat prometheus-service.yml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: '2023-09-19T14:12:15Z'
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
'f:spec':
'f:internalTrafficPolicy': {}
'f:ipFamilyPolicy': {}
'f:ports':
.: {}
'k:{"port":9090,"protocol":"TCP"}':
.: {}
'f:port': {}
'f:protocol': {}
'f:targetPort': {}
'f:selector': {}
'f:sessionAffinity': {}
'f:type': {}
manager: ACK-Console Apache-HttpClient
operation: Update
time: '2023-09-19T14:12:15Z'
name: prometheues
namespace: default
resourceVersion: '531698594'
uid: 0187137d-6805-4179-9981-dfa5481b8d5e
spec:
clusterIP: 172.25.7.25
clusterIPs:
- 172.25.7.25
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- port: 9090
protocol: TCP
targetPort: 9090
selector:
app: prometheus
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
$ kubectl apply -f prometheus-service.yml
此時可以訪問Prometheus控制臺了:
Grafana 部署
開始部署Grafana嘍:
# 部署Grafana
$ cat grafana.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
spec:
template:
metadata:
labels:
tool: grafana
name: grafana-pod
spec:
containers:
- image: grafana/grafana
imagePullPolicy: IfNotPresent
name: grafana
ports:
- containerPort: 3000
protocol: TCP
$ kubectl apply -f grafana.yml
# 創建Service
$ cat grafana-service.yml
apiVersion: v1
kind: Service
metadata:
name: grafana
spec:
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- port: 3000
protocol: TCP
targetPort: 3000
selector:
tool: grafana
sessionAffinity: None
type: ClusterIP
$ kubectl apply -f grafana-service.yml
可視化展示
訪問Grafana:默認賬戶/密碼:admin/admin
配置Prometheus數據源:
導入模版查看數據: