JaneliaSciComp/spark_stop

subworkflow

Stops Spark processing by writing a marker file which terminates the cluster processes.

spark bigdata infrastructure

Module Information

Repository: https://github.com/JaneliaSciComp/nextflow-modules/tree/main/subworkflows/janelia/spark_stop
Source: Janelia
Organization: JaneliaSciComp
Authors: @krokicki , @cgoina

This subworkflow uses the following modules:

Name	Type	Description
ch_meta	tuple	Channel of tuples where the first item is the meta map, and the second is a a spark context map. Structure: [ val(meta), val(spark) ]

Name	Type	Description
done	tuple	Same as the input tuple.

Include this subworkflow in your Nextflow pipeline:

include { SPARK_STOP } from 'https://github.com/JaneliaSciComp/nextflow-modules/tree/main/subworkflows/janelia/spark_stop'