[Docker] Tomcat 컨테이너 Heap 모니터링

개요

docker, kubernetes cluster 환경에서 java heap 모니터링 방법

구성환경

docker-ce 19.03.5, kubernetes v1.18.0, tomcat 8.5.53

 

톰캣 컨테이너의  힙 모니터링을 하기 위해서는 docker 실행시 --cap-add=SYS_PTRACE 옵션을 추가해야 한다.

예시 1) docker run 명령어에 --cap-add=SYS_PTRACE 옵션 추가 후 jmap 명령어를 이용하여 heap 상태체크및 덤프

shell> docker run -d --cap-add=SYS_PTRACE -e "JAVA_OPTS=-Xms512m -Xmx512m" --name mytomcat tomcat:8.5.53 54c29c557b2ff5afd3a4a65f709bd602eab4b3dbf0df0cb20d1cf708de7fd91e   shell> docker exec -it mytomcat sh  => 톰캣 컨테이너에 쉘로 접속 # jmap -heap 1 Attaching to process ID 1, please wait... Debugger attached successfully. Server compiler detected. JVM version is 25.242-b08 using thread-local object allocation. Parallel GC with 4 thread(s) Heap Configuration:    MinHeapFreeRatio         = 0    MaxHeapFreeRatio         = 100    MaxHeapSize              = 536870912 (512.0MB)    NewSize                  = 178782208 (170.5MB)    MaxNewSize               = 178782208 (170.5MB)    OldSize                  = 358088704 (341.5MB)    NewRatio                 = 2    SurvivorRatio            = 8    MetaspaceSize            = 21807104 (20.796875MB)    CompressedClassSpaceSize = 1073741824 (1024.0MB)    MaxMetaspaceSize         = 17592186044415 MB    G1HeapRegionSize         = 0 (0.0MB) Heap Usage: PS Young Generation Eden Space:    capacity = 134742016 (128.5MB)    used     = 43186848 (41.186187744140625MB)    free     = 91555168 (87.31381225585938MB)    32.05150797209387% used From Space:    capacity = 22020096 (21.0MB)    used     = 0 (0.0MB)    free     = 22020096 (21.0MB)    0.0% used To Space:    capacity = 22020096 (21.0MB)    used     = 0 (0.0MB)    free     = 22020096 (21.0MB)    0.0% used PS Old Generation    capacity = 358088704 (341.5MB)    used     = 0 (0.0MB)    free     = 358088704 (341.5MB)    0.0% used 7708 interned Strings occupying 725000 bytes. # jmap -dump:format=b,file=heap.hprof 1 => heap dump Dumping heap to /usr/local/tomcat/heap.hprof ... Heap dump file created #

 

예시 2) docker-compose.yaml 에 --cap-add=SYS_PTRACE 옵션 추가 방법

version: '3.1' services: # tomcat 서비스 정의    was:      image: tomcat:8.5.53      cap_add:      - SYS_PTRACE      environment:      - "JAVA_OPTS=-Xms512m -Xmx512m"      ports:      - "8080:8080"

 

예시 3) kubernetes deployment yaml examples

apiVersion: apps/v1 kind: Deployment metadata:   name: tomcat-deployment   labels:     app: tomcat spec:   replicas: 1   selector:     matchLabels:       app: tomcat   template:     metadata:       labels:         app: tomcat     spec:       containers:       - name: tomcat         image: tomcat:8.5.53         securityContext:           capabilities:             add:             - SYS_PTRACE         ports:         - containerPort: 8080         env:         - name: JAVA_OPTS           value: "-Xms512m -Xmx512m"

 

--cap-add=SYS_PTRACE 옵션 없이 톰캣 컨테이너를 실행했을 경우 아래와 같은 오류가 발생

shell> docker run -d -e "JAVA_OPTS=-Xms512m -Xmx512m" --name mytomcat tomcat:8.5.53  54c29c557b2ff5afd3a4a65f709bd602eab4b3dbf0df0cb20d1cf708de7fd91e  shell> docker exec -it mytomcat sh # jmap -heap 1 Attaching to process ID 1, please wait... ERROR: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permitted Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permitted sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permitted         at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:163)         at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach(LinuxDebuggerLocal.java:278)         at sun.jvm.hotspot.HotSpotAgent.attachDebugger(HotSpotAgent.java:671)         at sun.jvm.hotspot.HotSpotAgent.setupDebuggerLinux(HotSpotAgent.java:611)         at sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:337)         at sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:304)         at sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:140)         at sun.jvm.hotspot.tools.Tool.start(Tool.java:185)         at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)         at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:49)         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         at java.lang.reflect.Method.invoke(Method.java:498)         at sun.tools.jmap.JMap.runTool(JMap.java:201)         at sun.tools.jmap.JMap.main(JMap.java:130) Caused by: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 1: Operation not permitted         at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach0(Native Method)         at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$100(LinuxDebuggerLocal.java:62)         at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1AttachTask.doit(LinuxDebuggerLocal.java:269)         at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:138) #