一、Docker啟動異常表現：
1.狀態反復restaring，用命令查看
|
1
2
3
|
$docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES21c09be88c11 docker.xxxx.cn:5000/xxx-tes/xxx_tes:1.0.6 "/usr/local/tomcat..." 9 days ago Restarting (1) Less than a second ago xxx10 |
2.Docker日志有明顯問題：
|
1
|
$docker logs [容器名/容器ID] |
二、Docker啟動異常的可能原因：
2.1.內存不夠
Docker 啟動至少需要2G內存，首先執行free -mh命令查看剩余內存是否足夠
直接查看內存
|
1
2
3
4
|
$free -mh total used free shared buff/cache availableMem: 15G 14G 627M 195M 636M 726MSwap: 0B 0B 0B |
分析日志
有時候一瞬間內存過載溢出，導致部分進程被殺死，看起來內存也是夠用的，事實上docker還是會反復重啟，就需要通過docker日志和系統日志信的息來進一步分析：
分析docker日志
查看docker日志看到內存溢出的信息，要仔細翻閱才能找到信息，并不是在最下面
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
$docker logs [容器名/容器ID]|lessJava HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12)## There is insufficient memory for the Java Runtime Environment to continue.# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.# An error report file with more information is saved as:# //hs_err_pid1.logJava HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12)## There is insufficient memory for the Java Runtime Environment to continue.# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.# An error report file with more information is saved as:# /tmp/hs_err_pid1.logJava HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12)## There is insufficient memory for the Java Runtime Environment to continue.# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.# Can not save log file, dump to screen..## There is insufficient memory for the Java Runtime Environment to continue.# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.# Possible reasons:# The system is out of physical RAM or swap space# In 32 bit mode, the process size limit was hit# Possible solutions:# Reduce memory load on the system# Increase physical memory or swap space# Check if swap backing store is full# Use 64 bit Java on a 64 bit OS# Decrease Java heap size (-Xmx/-Xms)# Decrease number of Java threads# Decrease Java thread stack sizes (-Xss)# Set larger code cache with -XX:ReservedCodeCacheSize=# This output file may be truncated or incomplete.## Out of Memory Error (os_linux.cpp:2756), pid=1, tid=140325689620224## JRE version: (7.0_79-b15) (build )# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode linux-amd64 compressed oops)# Core dump written. Default location: //core or core.1# |
分析系統日志
查看系統日志，發現有大量由于內存溢出，進程被殺死的記錄
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
$grep -i 'Out of Memory' /var/log/messagesApr 7 10:04:02 centos106 kernel: Out of memory: Kill process 1192 (java) score 54 or sacrifice childApr 7 10:08:00 centos106 kernel: Out of memory: Kill process 2301 (java) score 54 or sacrifice childApr 7 10:09:59 centos106 kernel: Out of memory: Kill process 28145 (java) score 52 or sacrifice childApr 7 10:20:40 centos106 kernel: Out of memory: Kill process 2976 (java) score 54 or sacrifice childApr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3577 (java) score 47 or sacrifice childApr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3631 (java) score 47 or sacrifice childApr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3634 (java) score 47 or sacrifice childApr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3640 (java) score 47 or sacrifice childApr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3654 (java) score 47 or sacrifice childApr 7 10:27:27 centos106 kernel: Out of memory: Kill process 6998 (java) score 51 or sacrifice childApr 7 10:27:28 centos106 kernel: Out of memory: Kill process 7027 (java) score 52 or sacrifice childApr 7 10:28:10 centos106 kernel: Out of memory: Kill process 7571 (java) score 42 or sacrifice childApr 7 10:28:10 centos106 kernel: Out of memory: Kill process 7586 (java) score 42 or sacrifice child |
2.2.端口沖突
該docker監聽端口已經被其他進程占用，一般此種問題容易出現在新部署的服務，或在原有機器上部署新的后臺服務，所以在部署之前應該執行命令檢查端口是否已經被占用，如果上線后發現占有則應改為可用端口再重啟之。
檢查命令： $netstat -nltp|grep [規劃的端口號]
三、對策
3.1.內存不夠的對策：
對策1:
3.1.1 saltstack的minion在運行過久之后，可能占用大量內存，需要將其重啟。重啟命令可能有時并不起作用。主要檢查運行狀態，如果未成功停止，則重新重啟；
對策2:
3.2.2 ELK日志收集程序或者其他java進程占用過高，用top和ps命令排查，謹慎確定進程的作用，在確保不影響業務的情況下，停止相關進程；
對策3:
釋放被占用的內存（buff/cache）：
$sync #將內存數據寫入磁盤
$echo 3 > /proc/sys/vm/drop_caches #釋放被占用的內存
對策4:
有時候并不是buff/cache過高導致內存不夠用，確實是被很多必要的進程消耗掉了內存，那就需要從機器資源分配使用的層面去考慮和解決了。
3.2 端口沖突的對策
對策1:
一般此種問題容易出現在新部署的服務，或在原有機器上部署新的后臺服務，所以在部署之前應該執行命令檢查端口是否已經被占用，如果上線后發現占有則應改為可用端口再重啟之。
檢查命令： $netstat -nltp|grep [規劃的端口號]
以上就是本文的全部內容，希望對大家的學習有所幫助，也希望大家多多支持服務器之家。
原文鏈接：https://blog.csdn.net/u010716706/article/details/69524863
