BioImageTools/spark_stop

subworkflow

Stops Spark processing by writing a marker file which terminates the cluster processes.

spark bigdata infrastructure

Module Information

Repository: https://github.com/BioImageTools/nextflow-modules/tree/main/subworkflows/bits/spark_stop
Source: BioImageTools
Organization: BioImageTools
Authors: @krokicki , @cgoina

This subworkflow uses the following modules:

Name	Type	Description
ch_meta	tuple	Channel of tuples where the first item is the meta map, then a list of related files, and finally a spark_context map. Only the third item is used by this subworkflow. Structure: [ val(meta), [ files ], val(spark_context) ]

Name	Type	Description
done	tuple	The Spark work directory that was terminated.

Include this subworkflow in your Nextflow pipeline:

include { SPARK_STOP } from 'https://github.com/BioImageTools/nextflow-modules/tree/main/subworkflows/bits/spark_stop'