基因数据处理28之avocado运行

原创

KeepLearningAI 2023-01-04 10:55:42 博主文章分类：基因数据处理 ©著作权

文章标签 avocado hadoop spark SPARK 文章分类 运维

©著作权归作者所有：来自51CTO博客作者KeepLearningAI的原创作品，请联系作者获取转载授权，否则将追究法律责任

需要注意的是如果使用avocado的命令行，fs和fq为hdfs路径，properties为本地路径：

hadoop@Master:~/xubo/data/testTools/se$ avocado-submit /xubo/avocado/hs1.fq /xubo/avocado/hs38DH.fa /xubo/avocado/test20160527 /home/hadoop/cloud/avocado/basic.properties
Using SPARK_SUBMIT=/home/hadoop/cloud/spark-1.5.2//bin/spark-submit
Loading reads in from /xubo/avocado/hs1.fq
[Stage 8:>                                                          (0 + 2) / 4]SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
hadoop@Master:~/xubo/data/testTools/se$ hadoop fs -ls /xubo/avocado/test20160527
Found 7 items
-rw-r--r--   3 hadoop supergroup          0 2016-05-27 22:32 /xubo/avocado/test20160527/_SUCCESS
-rw-r--r--   3 hadoop supergroup      13367 2016-05-27 22:32 /xubo/avocado/test20160527/_common_metadata
-rw-r--r--   3 hadoop supergroup      13367 2016-05-27 22:32 /xubo/avocado/test20160527/_metadata
-rw-r--r--   3 hadoop supergroup      13367 2016-05-27 22:31 /xubo/avocado/test20160527/part-r-00000.gz.parquet
-rw-r--r--   3 hadoop supergroup      13367 2016-05-27 22:31 /xubo/avocado/test20160527/part-r-00001.gz.parquet
-rw-r--r--   3 hadoop supergroup      13367 2016-05-27 22:32 /xubo/avocado/test20160527/part-r-00002.gz.parquet
-rw-r--r--   3 hadoop supergroup      13367 2016-05-27 22:31 /xubo/avocado/test20160527/part-r-00003.gz.parquet

详细请见：
avocado：

hadoop@Master:~/xubo/data/testTools/se$ avocado-submit 
Using SPARK_SUBMIT=/home/hadoop/cloud/spark-1.5.2//bin/spark-submit
Argument "READS" is required
 READS                                                           : ADAM read-oriented data
 REFERENCE                                                       : ADAM or FASTA reference genome data
 VARIANTS                                                        : ADAM variant output
 CONFIG                                                          : avocado configuration file
 -debug                                                          : If set, prints a higher level of debug output.
 -fragment_length N                                              : Sets maximum fragment length. Default value is 10,000. Values greater than 1e9
                                                                   should be avoided.
 -h (-help, --help, -?)                                          : Print help
 -parquet_block_size N                                           : Parquet block size (default = 128mb)
 -parquet_compression_codec [UNCOMPRESSED | SNAPPY | GZIP | LZO] : Parquet compression codec
 -parquet_disable_dictionary                                     : Disable dictionary encoding
 -parquet_logging_level VAL                                      : Parquet logging level (default = severe)
 -parquet_page_size N                                            : Parquet page size (default = 1mb)
 -print_metrics                                                  : Print metrics to the log on completion

参考：
【1】https://github.com/bigdatagenomics/avocado/issues/152
【2】https://github.com/bigdatagenomics/avocado/