ViewCreatorFactory 是dremio 中专门管理view 的，目前主要是对于space 以及home 中的处理，当然对于source 也是可以支持的（支持版本类型的，比如nessie，原始分布式文件系统的实际也是可以支持的，但是不太符合dremio的玩法，以及限定死了）

调用部分

实际上通过标准的sqlhandler，具体实现就是CreateViewHandler

  • 参考代码

CreateViewHandler.java

public List<SimpleCommandResult> toResult(String sql, SqlNode sqlNode) throws Exception {
    SqlCreateView createView = SqlNodeUtil.unwrap(sqlNode, SqlCreateView.class);
    ParserUtil.validateParsedViewQuery(createView.getQuery());
    final NamespaceKey resolvedViewPath = catalog.resolveSingle(createView.getPath());
    Preconditions.checkState(resolvedViewPath.size() > 1, "View path "+ resolvedViewPath + " is not valid ");
    String formattedViewSql = getViewSql(createView, sql);
    VersionContext statementSourceVersion =
      ReferenceTypeUtils.map(createView.getRefType(), createView.getRefValue(), null);
    final VersionContext sessionVersion = config.getContext().getSession().getSessionVersionForSource(resolvedViewPath.getRoot());
    ConvertedRelNode convertedRelNode = validateTablesAndVersionContext(createView.getQuery(), resolvedViewPath, statementSourceVersion.orElse(sessionVersion));
    catalog.validatePrivilege(resolvedViewPath, SqlGrant.Privilege.ALTER);
    if (isVersioned(resolvedViewPath)) {
      return createVersionedView(createView, formattedViewSql, convertedRelNode, resolvedViewPath);
    } else {
      return createView(createView, formattedViewSql, convertedRelNode, resolvedViewPath);
    }
  }
SqlCreateView createView = SqlNodeUtil.unwrap(sqlNode, SqlCreateView.class);

createView参考处理

  • 代码

CatalogImpl.java

 public void createView(final NamespaceKey key, View view, ViewOptions viewOptions, NamespaceAttribute... attributes) throws IOException {
    // 会判断不同的类型，比如source，space，home
    switch (getType(key, true)) {
      case SOURCE:
        asMutable(key, "does not support create view")
          .createOrUpdateView(key, options.getSchemaConfig(), view, viewOptions);
        break;
      case SPACE:
      case HOME:
        if (view.hasDeclaredFieldNames()) {
          throw UserException.unsupportedError()
              .message("Dremio doesn't support field aliases defined in view creation.")
              .buildSilently();
        }
        // 对于space 以及home 都使用了viewCreatorFactory 进行view 的操作，具体的实现是DACViewCreatorFactory
        viewCreatorFactory.get(userName)
          .createView(key.getPathComponents(), view.getSql(), view.getWorkspaceSchemaPath(), false, attributes);
        break;
      default:
        throw UserException.unsupportedError()
          .message("Cannot create view in %s.", key)
          .build(logger);
    }
  }
// 会判断不同的类型，比如source，space，home

DACViewCreatorFactory的处理

  • 初始化

DACDaemonModule 模块中
参考代码

registry.bind(
      ViewCreatorFactory.class,
      new DACViewCreatorFactory(
        registry.provider(InitializerRegistry.class),
        registry.provider(LegacyKVStoreProvider.class),
        registry.provider(JobsService.class),
        registry.provider(NamespaceService.Factory.class),
        registry.provider(CatalogService.class),
        registry.provider(ContextService.class),
        () -> bootstrap.getAllocator()
      )
);
ViewCreatorFactory.class,
  • createView 处理

DACViewCreatorFactory 实现了ViewCreatorFactory 的接口定义，我只简单说明下createView 的处理

public void createView(List<String> path, String sql, List<String> sqlContext, boolean isVersionedSource, NamespaceAttribute... attributes) {
      SecurityContext securityContext = getSecurityContext();
      QueryExecutor executor = new QueryExecutor(jobsService, null, securityContext);
      DatasetTool tool = newDatasetTool(securityContext, executor);
 
      try {
        DatasetVersion version = DatasetVersion.newVersion();
        DatasetPath datasetPath = new DatasetPath(path);
       // DatasetTool 会进行view dataset 的创建，只是默认title为空，同时加载预览数据
        InitialPreviewResponse response = tool.newUntitled(allocator, new FromSQL(sql), version, sqlContext, null,true, 0, true);
        DatasetPath tmpPath = new DatasetPath(response.getDataset().getFullPath());
        VirtualDatasetUI vds = datasetService.getVersion(tmpPath, response.getDataset().getDatasetVersion(), isVersionedSource);
        // TODO - Remove dependency on a Resource file.
       // 此处使用了一个DatasetVersionResource，不太好，上边的todo 也说明了
        newDatasetVersionResource(securityContext, tool, version, tmpPath, allocator).save(vds, datasetPath, null, null, isVersionedSource, attributes);
      } catch (Exception e) {
        throw Throwables.propagate(e);
      }
}
SecurityContext securityContext = getSecurityContext();

说明

dremio 推荐的玩法是基于view 的vds 处理，使用了ViewCreatorFactory，核心是space 以及home 类型的，当然对于支持多版本的source 也是支持的（nessie ），以上是一个简单说明，详细的可以参考个github 源码（比如CreateViewHandler 执行的）

参考资料

sabot/kernel/src/main/java/com/dremio/exec/catalog/ViewCreatorFactory.java
dac/backend/src/main/java/com/dremio/dac/service/datasets/DACViewCreatorFactory.java
sabot/kernel/src/main/java/com/dremio/exec/catalog/CatalogImpl.java
sabot/kernel/src/main/java/com/dremio/exec/planner/sql/parser/SqlCreateView.java
sabot/kernel/src/main/java/com/dremio/exec/server/SabotContext.java
sabot/kernel/src/main/java/com/dremio/exec/planner/sql/handlers/direct/CreateViewHandler.java
sabot/kernel/src/main/java/com/dremio/exec/planner/sql/handlers/commands/CommandCreator.java
sabot/kernel/src/main/java/com/dremio/exec/work/foreman/AttemptManager.java
dac/backend/src/main/java/com/dremio/dac/explore/QueryExecutor.java
dac/backend/src/main/java/com/dremio/dac/explore/DatasetTool.java
dac/backend/src/main/java/com/dremio/dac/service/datasets/DatasetVersionMutator.java