前言

// 呵呵 果然运动才是最解压的方式, 感谢白*, 不知道手明天能用不, 今晚能睡个好觉吧 

最近看到了 这样的一篇文章 [讨论] Class.forName()被阻塞, 进而导出了 你假笨 的一篇文章 JDK的sql设计不合理导致的驱动类初始化死锁问题 

呵呵 这个还是挺有意思的, 我再 jdk7u40, jdk1.8.0_211 上面稳定复现 

但是 在 jdk9 上面, 就不会死锁了 

呵呵 对于这个问题, 这两天还是 尝试了很多的地方, 呵呵 麻蛋的 fastdebug 版本的 jdk 调试不了, 呵呵 可能编译记录的 源文件相关的信息 和我这里的场景 对不上吧 

文章 JDK的sql设计不合理导致的驱动类初始化死锁问题  中也使用 gdb 调试了一下 该场景, 但是 是从一个更加细节的突破口来进行调试的, 我这里的调试出发点 和 笨神 有一定的区别吧 

下面的 createOOM 是为了生成 堆转储文件 来进行分析, 我最初的思考是根据 运行时的寄存器的一些信息, 从堆转储文件中获取到对应的地址, inspeect 查看运行时的数据信息, 呵呵 但是 后来放弃了, 直接从 lldb 上面查看吧 

一下主要是分为了几个部分吧 : java 层面的调试, vm 层面的调试, 运行时汇编的调试(哎) 

一下的相关代码, 截图如果没有特殊说明 基于 jdk8 

测试用例

package com.hx.test06;

import java.util.ArrayList;
import java.util.List;

/**
 * Test12LoadDiffDriver
 *
 * @author Jerry.X.He 
 * @version 1.0
 * @date 2020-05-23 16:31
 */
public class Test12LoadDiffDriver {

  // Test12LoadDiffDriver
  // -Xmx32M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/java/heapdump_Test12LoadDiffDriver.hprof
  public static void main(String[] args) throws Exception {

    InnerMysqlThread it1 = new InnerMysqlThread();
    InnerPostgresqlThread it2 = new InnerPostgresqlThread();
    it1.start();
    it2.start();

    for(int i=10; i>0; i--) {
      Thread.sleep(1000);
      System.out.println("count down : " + i);
    }
    createOOM();

  }

  /**
   * createOOM
   *
   * @return
   * @date 2020-05-23 17:34
   */
  public static void createOOM() {
    List<Object> list = new ArrayList<>();
    while(true) {
      list.add(new byte[1000_000]);
    }
  }

  // InnerMysqlThread
  static class InnerMysqlThread extends Thread {
    public InnerMysqlThread() {
      this.setName("InnerMysqlThread");
    }

    @Override
    public void run() {
      try {
        Class.forName("com.mysql.jdbc.Driver", true, this.getClass().getClassLoader());
        System.out.println("com.mysql.jdbc.Driver");
      } catch (ClassNotFoundException e) {
        e.printStackTrace();
      }
    }

  }

  // InnerPostgresqlThread
  static class InnerPostgresqlThread extends Thread {
    public InnerPostgresqlThread() {
      this.setName("InnerPostgresqlThread");
    }

    @Override
    public void run() {
      try {
        Class.forName("org.postgresql.Driver", true, this.getClass().getClassLoader());
        System.out.println("org.postgresql.Driver");

      } catch (ClassNotFoundException e) {
        e.printStackTrace();
      }
    }
  }
}

运行时效果如下 

29 加载多个JdbcDriver造成死锁_driver

java层面的调试

我们先来看下 加载 mysql driver 的这个线程

29 加载多个JdbcDriver造成死锁_sql_02

我们发现 他阻塞的地方, 居然是在 加载 org.postgresql.Driver, 并注意一个细节 现在还是在 DriverManager.class 初始化的阶段, 在调用 DriverManager.<clinit> 

我去 这个是什么套路??? 

DriverManager 初始化的时候, 会迭代一些(看后面) driver 注册到到 provider 里面

29 加载多个JdbcDriver造成死锁_sql_03

迭代那些 drvier 呢?, 扫classpath下面的包, 如果 /META-INF/services/java.sql.Driver 存在, 则进行加载 

就好比我这里 本来程序里面只 使用了 mysql, postgresql 的 driver, 但是 sqlite 的包也符合这里的条件, 因此 也进行了加载 

29 加载多个JdbcDriver造成死锁_driver_04

所以 现在 InnerMysqlThread 这个线程的情况就是 在初始化 DriverManager.class 的时候 尝试去初始化 org.postgresql.Driver.class 

我们再来看下 加载 postgresql driver 的这个线程

29 加载多个JdbcDriver造成死锁_driver_05

可以发现 加载 postgresql driver 的这个线程 目前的情况是 在初始化 org.postgresql.Driver.class 的时候, 尝试去 初始化 DriverManager.class 

然后 两个线程都在等 对方的资源, 也都在等对方 初始化 org.postgresql.Driver.class/DriverManager.class 结束(vm 初始化这里的处理是如果在初始化中, 并且不是当前线程初始化, 则挂起当前线程, 等待初始化线程初始化完成)

这个 就是死锁的原因了  

HotspotVM 的调试

我这里只有 jdk9, jdk7 的调试环境, 但是 jdk9 上面复现不了这个问题, 呵呵 只能使用 jdk7 了 

妈的 昨晚尝试 调试 jdk7, jdk8 真的是费劲了心思, 呵呵 最后还是不行, 就当在试错吧 

一下调试基于 jdk7u40 

我们先来看下 加载 mysql driver 的这个线程

29 加载多个JdbcDriver造成死锁_5e_06

阻塞 是在初始化阶段, 尝试初始化 org.postgresql.Driver.class 

29 加载多个JdbcDriver造成死锁_java_07

再往上看, 是在初始化 DriverManager.class 的时候, 触发了上面的 org.postgresql.Driver.class  的初始化  

但是 当前 DriverManager.class 尚未完成初始化 

我们再来看下 加载 postgresql driver 的这个线程

29 加载多个JdbcDriver造成死锁_sql_08

阻塞 是在初始化阶段, 尝试初始化 DriverManager.class 

29 加载多个JdbcDriver造成死锁_sql_09

再往上看, 是在初始化 org.postgresql.Driver.class 的时候, 触发了上面的 DriverManager.class 的初始化  

但是 当前 org.postgresql.Driver.class 尚未完成初始化 

这就造成了死锁, 这个就是 运行时的真正的情况, 我不放弃我的资源, 你不放弃你的资源, 你在等我加载完, 我在等你加载完 

运行时汇编的调试

呵呵 汇编这部分, 是我最开始 想到的思路, 因为 我本地 jdk9 复现不了这个问题 

然后 当时没有考虑到 我还有一个 jdk7 的调试环境, 呵呵 这个是 大意了, 但是 jdk7 的调试 说实话 也是一把辛酸一把泪 

netbeans 上面啥都看不到, gdb 调试, 打印 this_oop 里面的数据, 一打印 就 gg 

然后 换成 clion 来调试, 最开始是 gdb 也是 打印 this_oop 里面的数据, 一打印 就 gg, 后面 换成了 lldb 来调试 总算是 好点了 

另外 有一套环境来调试的优势, 也可以来帮助 理解汇编层面的东西, 比如 字段 相对于 类的偏移啊, 这些 可以在 clion 里面调试了, 然后 在汇编上面 在看看效果  

以下调试 基于 jdk1.7.0-internal-fastdebug 

(lldb) thread list
Process 49004 stopped
* thread #1: tid = 0x1b901, 0x00007fff6315b22a libsystem_kernel.dylib`mach_msg_trap + 10, queue = 'com.apple.main-thread', stop reason = signal SIGKILL
  thread #2: tid = 0x1b915, 0x00007fff6315d9de libsystem_kernel.dylib`__ulock_wait + 10
  thread #4: tid = 0x1b917, 0x00007fff6315cbfe libsystem_kernel.dylib`__workq_kernreturn + 10
  thread #5: tid = 0x1b918, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #6: tid = 0x1b919, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #7: tid = 0x1b91a, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #8: tid = 0x1b91b, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #9: tid = 0x1b91c, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #10: tid = 0x1b91d, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #11: tid = 0x1b91e, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #12: tid = 0x1b91f, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #13: tid = 0x1b920, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #14: tid = 0x1b921, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #15: tid = 0x1b922, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Reference Handler'
  thread #16: tid = 0x1b923, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Finalizer'
  thread #17: tid = 0x1b92a, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: InnerPostgresqlThread'
  thread #18: tid = 0x1b924, 0x00007fff6315b266 libsystem_kernel.dylib`semaphore_wait_trap + 10, name = 'Java: Signal Dispatcher'
  thread #19: tid = 0x1b925, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: C2 CompilerThread0'
  thread #20: tid = 0x1b926, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: C2 CompilerThread1'
  thread #21: tid = 0x1b927, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Service Thread'
  thread #22: tid = 0x1b928, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #23: tid = 0x1b929, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: InnerMysqlThread'
  thread #24: tid = 0x1b92b, 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Abandoned connection cleanup thread'
(lldb) thread select 23
* thread #23, name = 'Java: InnerMysqlThread'
    frame #0: 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
libsystem_kernel.dylib`__psynch_cvwait:
->  0x7fff6315e86a <+10>: jae    0x7fff6315e874            ; <+20>
    0x7fff6315e86c <+12>: movq   %rax, %rdi
    0x7fff6315e86f <+15>: jmp    0x7fff6315b457            ; cerror_nocancel
    0x7fff6315e874 <+20>: retq
(lldb) bt
* thread #23, name = 'Java: InnerMysqlThread'
  * frame #0: 0x00007fff6315e86a libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff6321756e libsystem_pthread.dylib`_pthread_cond_wait + 722
    frame #2: 0x0000000102e1e2f7 libjvm.dylib`os::PlatformEvent::park() + 337
    frame #3: 0x0000000102e0ceba libjvm.dylib`ObjectMonitor::wait(long long, bool, Thread*) + 876
    frame #4: 0x0000000102be7b3a libjvm.dylib`instanceKlass::initialize_impl(instanceKlassHandle, Thread*) + 242
    frame #5: 0x0000000102be7a05 libjvm.dylib`instanceKlass::initialize(Thread*) + 75
    frame #6: 0x0000000102e8bf1c libjvm.dylib`Reflection::invoke_constructor(oopDesc*, objArrayHandle, Thread*) + 254
    frame #7: 0x0000000102cb3d7d libjvm.dylib`JVM_NewInstanceFromConstructor + 723
    frame #8: 0x0000000104023d7b
    frame #9: 0x00000001040063b5
    frame #10: 0x00000001040063b5
    frame #11: 0x0000000104006a63
    frame #12: 0x00000001040063b5
    frame #13: 0x00000001040063b5
    frame #14: 0x00000001040063b5
    frame #15: 0x0000000104006a63
    frame #16: 0x00000001040063b5
    frame #17: 0x0000000104000671
    frame #18: 0x0000000102c30cce libjvm.dylib`JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 916
    frame #19: 0x0000000102c300ef libjvm.dylib`JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) + 97
    frame #20: 0x0000000102c9bab4 libjvm.dylib`JVM_DoPrivileged + 1998
    frame #21: 0x0000000104023d7b
    frame #22: 0x00000001040063b5
    frame #23: 0x0000000104006590
    frame #24: 0x0000000104000671
    frame #25: 0x0000000102c30cce libjvm.dylib`JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 916
    frame #26: 0x0000000102c300ef libjvm.dylib`JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) + 97
    frame #27: 0x0000000102be930c libjvm.dylib`instanceKlass::call_class_initializer_impl(instanceKlassHandle, Thread*) + 354
    frame #28: 0x0000000102be7dcf libjvm.dylib`instanceKlass::initialize_impl(instanceKlassHandle, Thread*) + 903
    frame #29: 0x0000000102be7a05 libjvm.dylib`instanceKlass::initialize(Thread*) + 75
    frame #30: 0x0000000102d50437 libjvm.dylib`LinkResolver::resolve_static_call(CallInfo&, KlassHandle&, Symbol*, Symbol*, KlassHandle, bool, bool, Thread*) + 189
    frame #31: 0x0000000102d51f0c libjvm.dylib`LinkResolver::resolve_invokestatic(CallInfo&, constantPoolHandle, int, Thread*) + 132
    frame #32: 0x0000000102c28953 libjvm.dylib`InterpreterRuntime::resolve_invoke(JavaThread*, Bytecodes::Code) + 1037
    frame #33: 0x000000010403baa7
    frame #34: 0x0000000104000671
    frame #35: 0x0000000102c30cce libjvm.dylib`JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 916
    frame #36: 0x0000000102c300ef libjvm.dylib`JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) + 97
    frame #37: 0x0000000102be930c libjvm.dylib`instanceKlass::call_class_initializer_impl(instanceKlassHandle, Thread*) + 354
    frame #38: 0x0000000102be7dcf libjvm.dylib`instanceKlass::initialize_impl(instanceKlassHandle, Thread*) + 903
    frame #39: 0x0000000102be7a05 libjvm.dylib`instanceKlass::initialize(Thread*) + 75
    frame #40: 0x0000000102c9800e libjvm.dylib`find_class_from_class_loader(JNIEnv_*, Symbol*, unsigned char, Handle, Handle, unsigned char, Thread*) + 146
    frame #41: 0x0000000102c97ed0 libjvm.dylib`JVM_FindClassFromClassLoader + 769
    frame #42: 0x000000010040f242 libjava.dylib`Java_java_lang_Class_forName0 + 296
    frame #43: 0x0000000104023d7b
    frame #44: 0x00000001040063b5
    frame #45: 0x00000001040063b5
    frame #46: 0x0000000104000671
    frame #47: 0x0000000102c30cce libjvm.dylib`JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 916
    frame #48: 0x0000000102c300ef libjvm.dylib`JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) + 97
    frame #49: 0x0000000102c302d3 libjvm.dylib`JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*) + 473
    frame #50: 0x0000000102c30368 libjvm.dylib`JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*) + 110
    frame #51: 0x0000000102ca8d84 libjvm.dylib`thread_entry(JavaThread*, Thread*) + 135
    frame #52: 0x0000000102f365b6 libjvm.dylib`JavaThread::thread_main_inner() + 272
    frame #53: 0x0000000102f363ea libjvm.dylib`JavaThread::run() + 556
    frame #54: 0x0000000102e18f31 libjvm.dylib`java_start(Thread*) + 289
    frame #55: 0x00007fff632142eb libsystem_pthread.dylib`_pthread_body + 126
    frame #56: 0x00007fff63217249 libsystem_pthread.dylib`_pthread_start + 66
    frame #57: 0x00007fff6321340d libsystem_pthread.dylib`thread_start + 13

这里可以看到 InnerMysqlThread, InnerPostgresqlThread 都在 wait 

instanceKlass::initialize 调用了 instanceKlass::initialize_impl 函数 

我们先来看下简单的 instanceKlass::initialize 

(lldb) dis -s 0x102be7a05-75 -c 0x40
libjvm.dylib`instanceKlass::initialize:
    0x102be79ba <+0>:   pushq  %rbp
    0x102be79bb <+1>:   movq   %rsp, %rbp
    0x102be79be <+4>:   pushq  %r15
    0x102be79c0 <+6>:   pushq  %r14
    0x102be79c2 <+8>:   pushq  %rbx
    0x102be79c3 <+9>:   subq   $0x48, %rsp
    0x102be79c7 <+13>:  movq   %rsi, %r14
    0x102be79ca <+16>:  movq   %rdi, %rbx
    0x102be79cd <+19>:  movq   (%rbx), %rax
    0x102be79d0 <+22>:  callq  *0x50(%rax)
    0x102be79d3 <+25>:  testb  %al, %al
    0x102be79d5 <+27>:  je     0x102be7a0f               ; <+85>
    0x102be79d7 <+29>:  leaq   -0x50(%rbp), %r15
    0x102be79db <+33>:  movq   %r15, %rdi
    0x102be79de <+36>:  movq   %r14, %rsi
    0x102be79e1 <+39>:  callq  0x102bc30fe               ; HandleMark::initialize(Thread*)
    0x102be79e6 <+44>:  addq   $-0x10, %rbx
    0x102be79ea <+48>:  leaq   -0x58(%rbp), %rdi
    0x102be79ee <+52>:  movq   %r14, %rsi
    0x102be79f1 <+55>:  movq   %rbx, %rdx
    0x102be79f4 <+58>:  callq  0x102f5ef20               ; instanceKlassHandle::instanceKlassHandle(Thread*, klassOopDesc*)
    0x102be79f9 <+63>:  movq   -0x58(%rbp), %rdi
    0x102be79fd <+67>:  movq   %r14, %rsi
    0x102be7a00 <+70>:  callq  0x102be7a48               ; instanceKlass::initialize_impl(instanceKlassHandle, Thread*)
->  0x102be7a05 <+75>:  movq   %r15, %rdi
    0x102be7a08 <+78>:  callq  0x102bc31d2               ; HandleMark::~HandleMark()
    0x102be7a0d <+83>:  jmp    0x102be7a3c               ; <+130>
    0x102be7a0f <+85>:  cmpb   $0x5, 0x1da(%rbx)
    0x102be7a16 <+92>:  je     0x102be7a3c               ; <+130>
    0x102be7a18 <+94>:  leaq   0x48989f(%rip), %rdi      ; "/Users/fanhua/develop/openjdk-home/openjdk7/hotspot/src/share/vm/oops/instanceKlass.cpp"
    0x102be7a1f <+101>: leaq   0x4898f0(%rip), %rdx      ; "assert(is_initialized()) failed"
    0x102be7a26 <+108>: leaq   0x3e8f25(%rip), %rcx      ; "sanity check"
    0x102be7a2d <+115>: movl   $0xf8, %esi
    0x102be7a32 <+120>: callq  0x102aeae47               ; report_vm_error(char const*, int, char const*, char const*)
    0x102be7a37 <+125>: callq  0x102e17f5e               ; breakpoint
    0x102be7a3c <+130>: addq   $0x48, %rsp
    0x102be7a40 <+134>: popq   %rbx
    0x102be7a41 <+135>: popq   %r14
    0x102be7a43 <+137>: popq   %r15
    0x102be7a45 <+139>: popq   %rbp
    0x102be7a46 <+140>: retq
    0x102be7a47 <+141>: nop

(lldb) fram select 5
frame #5: 0x0000000102be7a05 libjvm.dylib`instanceKlass::initialize(Thread*) + 75
libjvm.dylib`instanceKlass::initialize:
->  0x102be7a05 <+75>: movq   %r15, %rdi
    0x102be7a08 <+78>: callq  0x102bc31d2               ; HandleMark::~HandleMark()
    0x102be7a0d <+83>: jmp    0x102be7a3c               ; <+130>
    0x102be7a0f <+85>: cmpb   $0x5, 0x1da(%rbx)
(lldb) re r
General Purpose Registers:
       rbx = 0x00000007f90f0710
       rbp = 0x0000700004356a00
       rsp = 0x00007000043569a0
       r12 = 0x0000000100821000
       r13 = 0x000000000000000b
       r14 = 0x0000000100821000
       r15 = 0x00007000043569b0
       rip = 0x0000000102be7a05  libjvm.dylib`instanceKlass::initialize(Thread*) + 75
13 registers were unavailable.

(lldb) x 0x0000700004356a00-0x58
0x7000043569a8: 60 b1 f1 0b 01 00 00 00 00 10 82 00 01 00 00 00  `??.............
0x7000043569b8: e8 bd 12 00 01 00 00 00 18 b0 f1 0b 01 00 00 00  ?.......??.....

我们吧一些 函数调用的地方 提出来

0x102be79de <+36>:  movq   %r14, %rsi
	0x102be79e1 <+39>:  callq  0x102bc30fe               ; HandleMark::initialize(Thread*)

    0x102be79ea <+48>:  leaq   -0x58(%rbp), %rdi
    0x102be79ee <+52>:  movq   %r14, %rsi
    0x102be79f1 <+55>:  movq   %rbx, %rdx
    0x102be79f4 <+58>:  callq  0x102f5ef20               ; instanceKlassHandle::instanceKlassHandle(Thread*, klassOopDesc*)

    0x102be79f9 <+63>:  movq   -0x58(%rbp), %rdi
    0x102be79fd <+67>:  movq   %r14, %rsi
    0x102be7a00 <+70>:  callq  0x102be7a48               ; instanceKlass::initialize_impl(instanceKlassHandle, Thread*)

可以发现的东西是 r14 上面是 当前线程, -0x58(%rbp) 对应的是 instanceKlassHandle/klassOopDesc* 

另外就是 关于参数的传递, rsi 似乎是用于固定传递 Thread*, rdi 用于传递 handle 

可以得出的结论是 

Thread = 0x0000000100821000
instanceKlassHandle = 0x010bf1b160
instanceKlass = 0x07f90f0710

我们再来看下 instanceKlass::initialize_impl

(lldb) dis -s 0x102be7b3a-242 -c 0x60
libjvm.dylib`instanceKlass::initialize_impl:
    0x102be7a48 <+0>:   pushq  %rbp
    0x102be7a49 <+1>:   movq   %rsp, %rbp
    0x102be7a4c <+4>:   pushq  %r15
    0x102be7a4e <+6>:   pushq  %r14
    0x102be7a50 <+8>:   pushq  %r13
    0x102be7a52 <+10>:  pushq  %r12
    0x102be7a54 <+12>:  pushq  %rbx
    0x102be7a55 <+13>:  subq   $0xe8, %rsp
    0x102be7a5c <+20>:  movq   %rsi, %r15
    0x102be7a5f <+23>:  movq   %rdi, %rbx
    0x102be7a62 <+26>:  xorl   %edi, %edi
    0x102be7a64 <+28>:  testq  %rbx, %rbx
    0x102be7a67 <+31>:  movq   0x54e782(%rip), %r12      ; (void *)0x00007fff96077070: __stack_chk_guard
    0x102be7a6e <+38>:  movq   (%r12), %rax
    0x102be7a72 <+42>:  movq   %rax, -0x30(%rbp)
    0x102be7a76 <+46>:  je     0x102be7a7b               ; <+51>
    0x102be7a78 <+48>:  movq   (%rbx), %rdi
    0x102be7a7b <+51>:  addq   $0x10, %rdi
    0x102be7a7f <+55>:  movq   %r15, %rsi
    0x102be7a82 <+58>:  callq  0x102be82ee               ; instanceKlass::link_class(Thread*)
    0x102be7a87 <+63>:  cmpq   $0x0, 0x8(%r15)
    0x102be7a8c <+68>:  jne    0x102be8252               ; <+2058>
    0x102be7a92 <+74>:  xorl   %edi, %edi
    0x102be7a94 <+76>:  testq  %rbx, %rbx
    0x102be7a97 <+79>:  je     0x102be7a9c               ; <+84>
    0x102be7a99 <+81>:  movq   (%rbx), %rdi
    0x102be7a9c <+84>:  callq  0x102a8aff0               ; instanceKlass::cast(klassOopDesc*)
    0x102be7aa1 <+89>:  movq   0x10(%rax), %rax
    0x102be7aa5 <+93>:  xorl   %r12d, %r12d
    0x102be7aa8 <+96>:  testq  %rax, %rax
    0x102be7aab <+99>:  movl   $0x0, %r13d
    0x102be7ab1 <+105>: je     0x102be7abf               ; <+119>
    0x102be7ab3 <+107>: movzwl 0x20(%rax), %r12d
    0x102be7ab8 <+112>: addq   $0x22, %rax
    0x102be7abc <+116>: movq   %rax, %r13
    0x102be7abf <+119>: xorl   %r14d, %r14d
    0x102be7ac2 <+122>: testq  %rbx, %rbx
    0x102be7ac5 <+125>: movl   $0x0, %edi
    0x102be7aca <+130>: je     0x102be7acf               ; <+135>
    0x102be7acc <+132>: movq   (%rbx), %rdi
    0x102be7acf <+135>: callq  0x102a8aff0               ; instanceKlass::cast(klassOopDesc*)
    0x102be7ad4 <+140>: movq   (%rax), %rcx
    0x102be7ad7 <+143>: movq   %rax, %rdi
    0x102be7ada <+146>: callq  *0x70(%rcx)
    0x102be7add <+149>: movq   $-0x1, %rcx
    0x102be7ae4 <+156>: movq   %r13, %rdi
    0x102be7ae7 <+159>: movq   %r12, %rsi
    0x102be7aea <+162>: movq   %rax, %rdx
    0x102be7aed <+165>: nop
    0x102be7aee <+166>: nopl   (%rax)
    0x102be7af2 <+170>: leaq   -0x58(%rbp), %rdi
    0x102be7af6 <+174>: movl   $0x1, %ecx
    0x102be7afb <+179>: movq   %rbx, %rsi
    0x102be7afe <+182>: movq   %r15, %rdx
    0x102be7b01 <+185>: callq  0x102f04a58               ; ObjectLocker::ObjectLocker(Handle, Thread*, bool)
    0x102be7b06 <+190>: movq   (%rbx), %rdi
    0x102be7b09 <+193>: movb   0x1ea(%rdi), %al
    0x102be7b0f <+199>: cmpb   $0x6, %al
    0x102be7b11 <+201>: je     0x102be802f               ; <+1511>
    0x102be7b17 <+207>: cmpb   $0x5, %al
    0x102be7b19 <+209>: je     0x102be811c               ; <+1748>
    0x102be7b1f <+215>: cmpb   $0x4, %al
    0x102be7b21 <+217>: jne    0x102be7b59               ; <+273>
    0x102be7b23 <+219>: cmpq   %r15, 0x178(%rdi)
    0x102be7b2a <+226>: je     0x102be7b4c               ; <+260>
    0x102be7b2c <+228>: movq   -0x48(%rbp), %rdi
    0x102be7b30 <+232>: xorl   %esi, %esi
    0x102be7b32 <+234>: movq   %r15, %rdx
    0x102be7b35 <+237>: callq  0x102f04d36               ; ObjectSynchronizer::waitUninterruptibly(Handle, long long, Thread*)
->  0x102be7b3a <+242>: cmpq   $0x0, 0x8(%r15)
    0x102be7b3f <+247>: movl   $0x1, %r14d
    0x102be7b45 <+253>: je     0x102be7b06               ; <+190>
    0x102be7b47 <+255>: jmp    0x102be823f               ; <+2039>
    0x102be7b4c <+260>: cmpq   %r15, 0x178(%rdi)
    0x102be7b53 <+267>: je     0x102be7ef9               ; <+1201>
    0x102be7b59 <+273>: testq  %rbx, %rbx
    0x102be7b5c <+276>: je     0x102be8273               ; <+2091>
    0x102be7b62 <+282>: addq   $0x10, %rdi
    0x102be7b66 <+286>: movl   $0x4, %esi
    0x102be7b6b <+291>: callq  0x102be793c               ; instanceKlass::set_init_state(instanceKlass::ClassState)
    0x102be7b70 <+296>: movq   (%rbx), %rax
    0x102be7b73 <+299>: movq   %r15, 0x178(%rax)
    0x102be7b7a <+306>: leaq   -0x58(%rbp), %rdi
    0x102be7b7e <+310>: callq  0x102f04ad0               ; ObjectLocker::~ObjectLocker()
    0x102be7b83 <+315>: movq   (%rbx), %rax
    0x102be7b86 <+318>: movq   0x80(%rax), %r12
    0x102be7b8d <+325>: testq  %r12, %r12
    0x102be7b90 <+328>: je     0x102be7cb2               ; <+618>
    0x102be7b96 <+334>: testb  $0x2, 0x9d(%rax)
    0x102be7b9d <+341>: jne    0x102be7cb2               ; <+618>
    0x102be7ba3 <+347>: movq   %r12, %rdi
    0x102be7ba6 <+350>: callq  0x102f5d7e0               ; Klass::cast(klassOopDesc*)
    0x102be7bab <+355>: movq   (%rax), %rcx
    0x102be7bae <+358>: movq   %rax, %rdi
    0x102be7bb1 <+361>: callq  *0x50(%rcx)
    0x102be7bb4 <+364>: testb  %al, %al

我们把一些函数调用的地方提出来 

0x102be7a5c <+20>:  movq   %rsi, %r15
    0x102be7a5f <+23>:  movq   %rdi, %rbx
    
    0x102be7a7f <+55>:  movq   %r15, %rsi
    0x102be7a82 <+58>:  callq  0x102be82ee               ; instanceKlass::link_class(Thread*)
    
    0x102be7a99 <+81>:  movq   (%rbx), %rdi
    0x102be7a9c <+84>:  callq  0x102a8aff0               ; instanceKlass::cast(klassOopDesc*)
    
    0x102be7acc <+132>: movq   (%rbx), %rdi
    0x102be7acf <+135>: callq  0x102a8aff0               ; instanceKlass::cast(klassOopDesc*)
    
    0x102be7af6 <+174>: movl   $0x1, %ecx
    0x102be7afb <+179>: movq   %rbx, %rsi
    0x102be7afe <+182>: movq   %r15, %rdx
    0x102be7b01 <+185>: callq  0x102f04a58               ; ObjectLocker::ObjectLocker(Handle, Thread*, bool)
    
    0x102be7b2c <+228>: movq   -0x48(%rbp), %rdi
    0x102be7b30 <+232>: xorl   %esi, %esi
    0x102be7b32 <+234>: movq   %r15, %rdx
    0x102be7b35 <+237>: callq  0x102f04d36               ; ObjectSynchronizer::waitUninterruptibly(Handle, long long, Thread*)

r15 记录的是当前线程, rbx 记录的是 instanceKlassHandle

这里的函数调用方式, 除了 ObjectLocker::ObjectLocker 其他的基础调用 和上面的规则差不多 

ObjectSynchronizer::waitUninterruptibly 的调用 使用的 rdx 存放 Thread*, si 存放的第二个参数 0, di 存放的 instanceKlassHandle

然后 我们来尝试获取一些 运行时数据, this_oop->name 是什么呢? 

(lldb) frame select 4
frame #4: 0x0000000102be7b3a libjvm.dylib`instanceKlass::initialize_impl(instanceKlassHandle, Thread*) + 242
libjvm.dylib`instanceKlass::initialize_impl:
->  0x102be7b3a <+242>: cmpq   $0x0, 0x8(%r15)
    0x102be7b3f <+247>: movl   $0x1, %r14d
    0x102be7b45 <+253>: je     0x102be7b06               ; <+190>
    0x102be7b47 <+255>: jmp    0x102be823f               ; <+2039>
(lldb) re r
General Purpose Registers:
       rbx = 0x000000010bf1b160
       rbp = 0x0000700004356990
       rsp = 0x0000700004356880
       r12 = 0x0000000000000015
       r13 = 0x000000010021709a
       r14 = 0x0000000000000000
       r15 = 0x0000000100821000
       rip = 0x0000000102be7b3a  libjvm.dylib`instanceKlass::initialize_impl(instanceKlassHandle, Thread*) + 242
13 registers were unavailable.

(lldb) x 0x0000700004356880 -c 0x120
0x700004356880: 01 5b 26 03 01 00 00 00 00 10 82 00 01 00 00 00  .[&.............
0x700004356890: c0 68 35 04 00 70 00 00 83 2f bc 02 01 00 00 00  ?h5..p.../?.....
0x7000043568a0: 38 6a 35 04 00 70 00 00 00 10 82 00 01 00 00 00  8j5..p..........
0x7000043568b0: 50 e3 0e f9 07 00 00 00 70 70 07 96 ff 7f 00 00  P?.?....pp..?...
0x7000043568c0: 00 6a 35 04 00 70 00 00 ca 95 af 02 01 00 00 00  .j5..p..?.?.....
0x7000043568d0: 00 69 35 04 00 70 00 00 22 2c b3 02 01 00 00 00  .i5..p..",?.....
0x7000043568e0: d4 1c 24 03 01 00 00 00 10 07 0f f9 07 00 00 00  ?.$........?....
0x7000043568f0: a8 69 35 04 00 70 00 00 10 07 0f f9 07 00 00 00  ?i5..p.....?....
0x700004356900: 27 00 7c fb 92 d4 9c b4 a8 69 35 04 00 70 00 00  '.|?.?.??i5..p..
0x700004356910: 10 07 0f f9 07 00 00 00 b0 69 35 04 00 70 00 00  ...?....?i5..p..
0x700004356920: 30 69 35 04 00 70 00 00 b8 a6 a8 02 01 00 00 00  0i5..p..???.....
0x700004356930: 40 69 35 04 00 70 00 00 d0 00 1f 03 01 00 00 00  @i5..p..?.......
0x700004356940: 00 10 82 00 01 00 00 00 60 b1 f1 0b 01 00 00 00  ........`??.....
0x700004356950: 01 00 00 00 00 00 00 00 01 ef f5 02 01 00 00 00  .........??.....
0x700004356960: 27 00 7c fb 92 d4 9c b4 10 07 0f f9 07 00 00 00  '.|?.?.?...?....
0x700004356970: 00 10 82 00 01 00 00 00 0b 00 00 00 00 00 00 00  ................
0x700004356980: 00 10 82 00 01 00 00 00 b0 69 35 04 00 70 00 00  ........?i5..p..
0x700004356990: 00 6a 35 04 00 70 00 00 05 7a be 02 01 00 00 00  .j5..p...z?.....
(lldb) x 0x0000700004356990-0x48 -c 0x40
0x700004356948: 60 b1 f1 0b 01 00 00 00 01 00 00 00 00 00 00 00  `??.............
0x700004356958: 01 ef f5 02 01 00 00 00 27 00 7c fb 92 d4 9c b4  .??.....'.|?.?.?
0x700004356968: 10 07 0f f9 07 00 00 00 00 10 82 00 01 00 00 00  ...?............
0x700004356978: 0b 00 00 00 00 00 00 00 00 10 82 00 01 00 00 00  ................
(lldb) x 0x010bf1b160
0x10bf1b160: 10 07 0f f9 07 00 00 00 bc bc bc bc bc bc bc bc  ...?....????????
0x10bf1b170: bc bc bc bc bc bc bc bc bc bc bc bc bc bc bc bc  ????????????????
(lldb) x 0x07f90f0710 -c 0x40
0x7f90f0710: 0a 2c 00 0d 01 00 00 00 48 00 1c ff 00 00 00 00  .,......H..?....
0x7f90f0720: 00 3e 1b 03 01 00 00 00 10 00 00 00 40 00 00 00  .>..........@...
0x7f90f0730: 78 70 21 00 01 00 00 00 b0 c5 0d f9 07 00 00 00  xp!.....??.?....
0x7f90f0740: 48 d2 0e f9 07 00 00 00 78 2b e0 f8 07 00 00 00  H?.?....x+??....
(lldb) x 0x0100217078 -c 0x40
0x100217078: 10 e0 1b 03 01 00 00 00 85 8f de ff fe ff ff ff  .?........??????
0x100217088: 00 00 00 00 00 00 00 00 04 00 00 00 2a 50 2c 34  ............*P,4
0x100217098: 15 00 6f 72 67 2f 70 6f 73 74 67 72 65 73 71 6c  ..org/postgresql
0x1002170a8: 2f 44 72 69 76 65 72 f1 f1 f1 f1 f1 f1 f1 f1 f1  /Driver?????????

先获取 instanceKlassHandle, 然后根据 instanceKlassHandle 获取 instanceKlass, 然后 name 取偏移为 0x20 的地方 

然后 获取 name 的相关信息, 最后得出结论, InnerMysqlThread 线程 阻塞 是在初始化阶段, 尝试初始化 org.postgresql.Driver.class 

当然用类似的方式, 可以查看 InnerMysqlThread 是在初始化 那个类 的时候尝试加载 org.postgresql.Driver.class 

InnerPostgresqlThread 的情况类似 

jdk9 为什么没事?

29 加载多个JdbcDriver造成死锁_java_10

29 加载多个JdbcDriver造成死锁_5e_11

driver 的注册放到了, 需要使用 driver 的时候  

参考

[讨论] Class.forName()被阻塞

JDK的sql设计不合理导致的驱动类初始化死锁问题