简介:

Dynamic SQL Variation 可以认为是Db2内部的一种变量,存放在Dynamic SQL Cache(package cache的一部分)中,每个Variation对应一条编译的动态SQL语句,也就是说,每当Db2编译了一条动态SQL,SQL cache中就会多一个variation,在variation上的加的锁即是Variation Lock(简称V lock 或 VarLock)。为了确保variation有效,SQL执行期间需要在其对应的variation上加锁,详细的加锁方案如下:
如果待执行SQL已在SQL Cache中,应用会在其对应的variation上加一个S锁,这个S锁可以确保SQL执行期间的有效性(比如,删除一张表会使依赖这张表的所有variation失效)。
如果待执行SQL不在SQL Cache中,那么需要编译该SQL,编译期间为了把variation加载(loading)到cache中,会在编译开始前对该variation加一个X类型的锁,这个锁被称之为V loading lock,等编译完成之后,就会释放该锁。X类型V loading lock的目地是为了确保只有这一个应用编译该SQL(编译SQL的代价非常高,只需要编译一次即可,之后不管是同一个应用还是其他应用,要执行相同的SQL就不必再重新编译)。如果应用A拿到了X类型的 V Loading lock,在进行编译期间,有应用B也想编译,那应用B也会尝试获取同一个V loading lock,不过是S类型的。

db2pd工具

为了对Dynamic SQL Variations有一个直观的理解,可以使用db2pd的dynamic选项查看内存中有哪些Dynamic SQL Variation:

测试用到的命令如下,测试之前请确保数据库是未激活状态:

$ db2 +o terminate
$ db2 +o connect to sample
$ db2pd -d sample -dyn
$ db2 +o "list tables"
$ db2 +o "select * from t1 fetch first 1 rows only"
$ db2 +o "describe table t2"
$ db2  "insert into t2 values(10,'aaa')"
$ db2pd -d sample -dyn
$ db2  "insert into t2 values(10,'aaa')"
$ db2pd -d sample -dyn

1. 刚连接库, 未发出过动态SQL,发现没有Dynamic SQL Variations

inst105@node01:~$ db2 +o terminate
inst105@node01:~$ db2 +o connect to sample
inst105@node01:~$ db2pd -d sample -dyn

Database Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:03 -- Date 2019-04-19-05.54.19.052052

Dynamic Cache:
Current Memory Used           199632
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          0
Number of Statement Inserts   0
Number of Statement Deletes   0
Number of Variation Inserts   0
Number of Statements          0

Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text 

Dynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk

Dynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies

2. 发出4条语句之后,有4个Dynamic SQL Statements,可以根据AnchID和StmtUID找到对应的SQL语句(可以看到list tables和describe table命令也被Db2转化为动态SQL执行了):

inst105@node01:~$ db2 +o "list tables"
inst105@node01:~$ db2 +o "select * from t1 fetch first 1 rows only"
inst105@node01:~$ db2 +o "describe table t2"
inst105@node01:~$ db2 "insert into t2 values(10,'aaa')"
DB20000I  The SQL command completed successfully.
inst105@node01:~$ db2pd -d sample -dyn

Database Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:24 -- Date 2019-04-19-05.54.40.171977

Dynamic Cache:
Current Memory Used           643435
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          4
Number of Statement Inserts   8
Number of Statement Deletes   4
Number of Variation Inserts   4
Number of Statements          4

Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text 
0x00007F60ACF51EA0 116    1          1          1          1          1          SELECT TABNAME, TABSCHEMA, TYPE, CREATE_TIME FROM SYSCAT.TABLES WHERE TABSCHEMA = USER ORDER BY TABSCHEMA, TABNAME
0x00007F60AFD994E0 854    1          1          1          1          1          insert into t2 values(10,'aaa')
0x00007F60ACF5ABA0 904    1          1          1          1          1          select * from t1 fetch first 1 rows only
0x00007F60ACF5FD60 988    1          1          1          1          1          SELECT COLNAME, TYPESCHEMA, TYPENAME, LENGTH, SCALE, NULLS FROM SYSCAT.COLUMNS WHERE TABSCHEMA = CURRENT SCHEMA AND TABNAME = 'T2' ORDER BY COLNO

Dynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk
0x00007F60ACF52060 116    1          1          CS  5    B
0x00007F60AFD99640 854    1          1          CS  5    B
0x00007F60ACF5AD00 904    1          1          CS  5    B
0x00007F60AFD90080 988    1          1          CS  5    B

Dynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies
0x00007F60ACF523E0 116    1          1          1          1          6   01000000010000000100800ED6 Y   2019-04-19-05.54.23.368394 16888      0     
0x00007F60AFD999C0 854    1          1          1          1          4   01000000010000000100C06AD6 Y   2019-04-19-05.54.36.155828 5336       1     
0x00007F60ACF5B080 904    1          1          1          1          6   010000000100000001000071D6 Y   2019-04-19-05.54.27.488288 5560       1     
0x00007F60AFD90340 988    1          1          1          1          6   01000000010000000100807BD6 Y   2019-04-19-05.54.31.645792 18432      1

3. 再执行一次相同的SQL,发现还是用的原来的 Variation,只是NumRef增加了1.

inst105@node01:~$ db2  "insert into t2 values(10,'aaa')"
DB20000I  The SQL command completed successfully.
inst105@node01:~$ db2pd -d sample -dyn

Database Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:32 -- Date 2019-04-19-05.54.48.034430

Dynamic Cache:
Current Memory Used           643435
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          5
Number of Statement Inserts   8
Number of Statement Deletes   4
Number of Variation Inserts   4
Number of Statements          4

Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text 
0x00007F60ACF51EA0 116    1          1          1          1          1          SELECT TABNAME, TABSCHEMA, TYPE, CREATE_TIME FROM SYSCAT.TABLES WHERE TABSCHEMA = USER ORDER BY TABSCHEMA, TABNAME
0x00007F60AFD994E0 854    1          1          1          2          2          insert into t2 values(10,'aaa')
0x00007F60ACF5ABA0 904    1          1          1          1          1          select * from t1 fetch first 1 rows only
0x00007F60ACF5FD60 988    1          1          1          1          1          SELECT COLNAME, TYPESCHEMA, TYPENAME, LENGTH, SCALE, NULLS FROM SYSCAT.COLUMNS WHERE TABSCHEMA = CURRENT SCHEMA AND TABNAME = 'T2' ORDER BY COLNO

Dynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk
0x00007F60ACF52060 116    1          1          CS  5    B
0x00007F60AFD99640 854    1          1          CS  5    B
0x00007F60ACF5AD00 904    1          1          CS  5    B
0x00007F60AFD90080 988    1          1          CS  5    B

Dynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies
0x00007F60ACF523E0 116    1          1          1          1          6   01000000010000000100800ED6 Y   2019-04-19-05.54.23.368394 16888      0     
0x00007F60AFD999C0 854    1          1          1          2          4   01000000010000000100C06AD6 Y   2019-04-19-05.54.36.155828 5336       1     
0x00007F60ACF5B080 904    1          1          1          1          6   010000000100000001000071D6 Y   2019-04-19-05.54.27.488288 5560       1     
0x00007F60AFD90340 988    1          1          1          1          6   01000000010000000100807BD6 Y   2019-04-19-05.54.31.645792 18432      1

4. 如果有其他的应用,执行相同的SQL,也不会有新的Dynamic SQL Variations。

Trace工具

为了进一步验证上面的说法,可以抓取trace,做一个简单的分析。脚本如下:

db2stop force && db2start
db2 connect to sample
sleep 2

db2 -x 'values MON_GET_APPLICATION_HANDLE()' | awk '{print $1}' > currentAgentID
appHdl=`cat currentAgentID`
db2trc on -t -apphdl $appHdl -f db2trace.dmp 

for i in 1 2 3 4 5
do
        db2 +o "select * from t1 fetch first 1 rows only"
        sleep 2
done
                                      
db2trc off                                                                    
db2trc flw -t db2trace.dmp  db2trace.flw                          
db2trc fmt db2trace.dmp db2trace.fmt   
db2pd -d sample -dynamic > db2pd.dyn.out
db2 +o terminate
echo "Script ends"

输出中可以看到SQL执行了5次(NumRef):

inst105@node01:~/20190419$ cat db2pd.dyn.out 

Database Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:18 -- Date 2019-04-19-06.22.34.333059

Dynamic Cache:
Current Memory Used           568245
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          6
Number of Statement Inserts   2
Number of Statement Deletes   0
Number of Variation Inserts   2
Number of Statements          2

Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text 
0x00007FEE60F51040 651    1          1          1          1          1          values MON_GET_APPLICATION_HANDLE()
0x00007FEE60F56680 904    1          1          1          5          5          select * from t1 fetch first 1 rows only

Dynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk
0x00007FEE60F511A0 651    1          1          CS  5    B
0x00007FEE60F567E0 904    1          1          CS  5    B

Dynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies
0x00007FEE60F51520 651    1          1          1          1          6   010000000100000001006051D6 Y   2019-04-19-06.22.21.009028 6840       0     
0x00007FEE60F56B60 904    1          1          1          5          6   010000000100000001000071D6 Y   2019-04-19-06.22.22.333892 5560       1

查看Trace中锁的名子,loading = 1表示是V loading lock,可以看到V loading lock的lockname和其他基本一样(只差了一位):

inst105@node01:~/20190419$ grep lockname  db2trace.fmt | grep -i VARIATION
        lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )
        lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )
        lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )
        lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )

如果进一步分析trace文件,还可以验证很多结论,比如,加的V loading lock是X类型的:

1428    entry DB2 UDB lock manager sqlplrq fnc (1.3.35.22.0)
        pid 10054 tid 140663370802944 cpid 10075 node 0 sec 0 nsec 3299000
        eduid 22 eduname db2agent
        bytes 152

        Data1   (PD_TYPE_SQLP_LOCK_INFO,144) SQLP_LOCK_INFO:
        lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )
        pLRB (nil) prevIntent NON curIntent NON intent ..X duration 1
        rlInFlags 0x00000001 rlOutFlags 0x00000000 rlTimeout 0xFFFFFFFE cursorBitmap 0x40000000 rrIIDin 0 rrIIDout 0 priority 0 pEHLState 0000000000000000
        rlUserData.UNKNOWN 0000 0000 0000 0000                        ........

        dataPtr (nil)

1429    exit DB2 UDB lock manager sqlplrq fnc (2.3.35.22.0)
        pid 10054 tid 140663370802944 cpid 10075 node 0 sec 0 nsec 3302000
        rc = 0
        bytes 288

        Data1   (PD_TYPE_SQLP_LOCK_INFO,144) SQLP_LOCK_INFO:
        lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )
        pLRB 0x7fee44354f00 prevIntent NON curIntent ..X intent ..X duration 1
        rlInFlags 0x00000001 rlOutFlags 0x00000000 rlTimeout 0xFFFFFFFE cursorBitmap 0x40000000 rrIIDin 0 rrIIDout 0 priority 2 pEHLState 0000000000000000
        rlUserData.UNKNOWN 0000 0000 0000 0000                        ........

        dataPtr (nil)
        Data2   (PD_TYPE_SQLP_LRB,128) SQLP_LRB:
        state L (0x04) next (nil) rsInfoIdx 0
        prev (nil) tranChainPrev (nil)
        status G (0x01) mode ..X dur 1 convMode NON tran_handl 3
        holdcount 0 wantrrIID 0  rrIID 0  lrbFlag 0x0 chainNum 0 lsoFeedback 0
        tran_chain 0x7fee44354200 head_ptr 0x7fee44354e80 awb_ptr (nil)
        cursorBitmap 0x40000000 hashResult 1517 wantAttributes 00 priority 2 pad2 00

又比如:
第一次跑SQL,对V lock操作顺序如下: 加X类型的V loading lock (010000000100000001000171D6),加S类型的V lock(010000000100000001000071D6),释放X类型的V loading lock,释放S类型的V lock。
之后的4次SQL,因此SQL Cache中已经有variation, 所以再没有需要过V loading lock,只是简单的加V lock,释放V lock

再比如:
compile SQL的代价真的很高,从开始1436到结束的17737:

inst105@node01:~/20190419$ grep 'sqlra_compile_var entry' db2trace.flw 
1436           0.003305000   | | | | | | sqlra_compile_var entry [eduid 22 eduname db2agent]
inst105@node01:~/20190419$ grep 'sqlra_compile_var exit' db2trace.flw 
17737          0.182564000   | | | | | | sqlra_compile_var exit
而运行5次SQL才一共39039:
inst105@node01:~/20190419$ tail -n 2 db2trace.flw
39038          9.400739000   | | sqlccipcrecv data [probe 55]
39039          9.400740000   | | | sqloSSemP entry [eduid 22 eduname db2agent]

 

再比如:Db2是先获得了X类型的V loading lock(Trace中1428条目),然后才开始编译(Trace中1436条目),也就是说编码的前置条件是已经获得了X类型的V loading lock。