copy into from 是dremio 24 版本很不错的一个新功能，可以提升数据加载处理的速度（以前对于iceberg 数据的处理有insert into 以及通过ctas 模式）

简单机制说明

目前看到copy into from 并不象其他sql 命令基于sql SimpleDirectHandler 模式，而是通过sql 关系转换的
DremioSqlToRelConverter 中的

 


  private RelRoot convertOther(SqlNode query, boolean top, RelDataType targetRowType) {
    if (query instanceof SqlOptimize) {
      NamespaceKey path = ((SqlOptimize)query).getPath();
      Prepare.PreparingTable nsTable = catalogReader.getTable(path.getPathComponents());
      return RelRoot.of(new TableOptimizeCrel(cluster, cluster.traitSetOf(Convention.NONE), nsTable.toRel(createToRelContext()), nsTable, null, OptimizeOptions.DEFAULT), SqlKind.OTHER);
    } else if (query instanceof SqlCopyIntoTable) {
      NamespaceKey path = ((SqlCopyIntoTable)query).getPath();
      Prepare.PreparingTable nsTable = catalogReader.getTable(path.getPathComponents());
      CopyIntoTableContext  copyIntoTableContext= new CopyIntoTableContext((SqlCopyIntoTable)query);
      return RelRoot.of(new CopyIntoTableCrel(cluster, cluster.traitSetOf(Convention.NONE), nsTable,  copyIntoTableContext), SqlKind.OTHER);
    } else  {
      return super.convertQueryRecursive(query, top, targetRowType);
    }
  }

几个核心类

主要是CopyIntoTableContext，CopyIntoTableCrel ，CopyIntoTableRel ，CopyIntoTablePrule,CopyIntoTableRule
CopyIntoTableRule 主要是为PlannerPhase 逻辑阶段提供规则（CopyIntoTableRule 使用了CopyIntoTableContext，CopyIntoTableCrel ，CopyIntoTableRel ）
CopyIntoTablePrule 主要是为PlannerPhase 物理计划规则提供规则，实际物理规则的生成是通过了CopyIntoTablePlanBuilder ，里边包含了实际物理规则的处理，值得学习参考

说明

以上是简单的说明，内部好多细节是值得学习参考的，而且目前copy into from 属于append 模式的（也比较符合日常数据湖数据处理模式）

参考资料

sabot/kernel/src/main/java/com/dremio/exec/planner/sql/handlers/query/CopyIntoTableContext.java
sabot/kernel/src/main/java/com/dremio/exec/planner/sql/DremioSqlToRelConverter.java
sabot/kernel/src/main/java/com/dremio/exec/calcite/logical/CopyIntoTableCrel.java
sabot/kernel/src/main/java/com/dremio/exec/planner/physical/CopyIntoTablePrule.java
sabot/kernel/src/main/java/com/dremio/exec/planner/PlannerPhase.java
sabot/kernel/src/main/java/com/dremio/exec/planner/CopyIntoTablePlanBuilder.java
sabot/kernel/src/main/java/com/dremio/exec/planner/logical/CopyIntoTableRule.java
https://docs.dremio.com/software/data-formats/copying-data-into-tables/