Introduction

Workflows are a powerful feature in Galaxy that allow you to chain multiple steps of an analysis together. To make a workflow reusable with slightly different settings you can define and use workflow parameters.

Agenda

In this tutorial, we will cover:

1. Introduction
2. Add workflow parameters to a workflow
3. Add an integer workflow parameter
4. Create a text parameter from multiple parameters
5. Read a parameter from a dataset
6. Conclusion

Add workflow parameters to a workflow

We will import a simple workflow and then demonstrate how workflow parameters can be added.

Hands-on: Import integer workflow

1. Click on Workflow in the top menu
2. Click on the galaxy-upload import workflow button on the right

3. Enter the following URL into the “Archived Workflow” URL box

   https://raw.githubusercontent.com/galaxyproject/training-material/master/topics/galaxy-interface/tutorials/workflow-parameters/workflows/cut_n_lines.ga
   

4. Click on import workflow

Add an integer workflow parameter

This workflow takes an input dataset and outputs the first 10 lines of that input dataset. The number of lines that should be output can be set using the Number of lines parameter of the select first tool.

Instead of selecting a specific number of lines we can choose to insert a workflow parameter that allows the user to set this parameter when running the workflow.

Hands-on: Updating the Workflow

1. Upload a dataset using “Paste/Fetch data” with the contents:

   E. coli (O19:H34. Extracellular pathogenic)
   E. coli (O7:K1. Extracellular pathogenic)
   E. coli 536
   E. coli APEC O1 (O1:K12:H7. APEC
   E. coli CFT083 (O6:K2:H1. UPEC)
   E. coli E2348/69 (O127:H6. Enteropathogenic)
   E. coli EC4115 (O157:H7 EHEC)
   E. coli ED1a O81 (Commensal)
   E. coli EDL933 (O157:H7 EHEC)
   E. coli F11
   E. coli S88 (O45:K1. Extracellular pathogenic)
   E. coli SE15 (O150:H5. Commensal)
   E. coli Sakai (O157:H7 EHEC)
   E. coli TW14359 (O157:H7 EHEC)
   E. coli UMN026 (O17:K52:H18. Extracellular pathogenic)
   E. coli UTI89 O18:K1:H7. UPEC)
   

2. Open the workflow editor

   Tip: Opening the workflow editor

   1. In the top menu bar, click on Workflows
   2. Click on the name of the workflow you want to edit
   3. Select galaxy-wf-edit Edit from the dropdown menu to open the workflow in the workflow editor

3. Click the Select first tool tool in your workflow
4. Find the Number of lines parameter in the right hand panel.

5. Click on the workflow connection button galaxy-wf-connection to convert the Number of lines parameter into a workflow parameter connection

   You will see that a new input with the text Number of lines has appeared on the Select first tool in the editor.

6. Add a Simple inputs used for workflow logic, found under the Inputs section of your toolbox

7. If you click on the Input parameter box in the editor you will see in the Details panel that the Parameter type is set to Text. If you try to connect this parameter to the “Number of lines” parameter you will see that the noodle turns orange and that you cannot create a connection.

8. Click on Input parameter, and change the Parameter type to Integer

9. Connect “output” from Input parameter tool to the “Number of lines” input of the Select first tool.

10. Save galaxy-save your workflow

11. Run your workflow

    Tip: Running a workflow

    • Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
    • Click on the workflow-run (Run workflow) button next to your workflow
    • Configure the workflow as needed
    • Click the Run Workflow button at the top-right of the screen
    • You may have to refresh your history to see the queued jobs

Notice the new input that can be changed before the workflow is run

Tip: Parameter Reuse

You can connect a parameter to multiple steps or to multiple parameters within a step, everywhere you need to use it.

Create a text parameter from multiple parameters

It is often necessary to create a text parameter that is flanked by additional text. Take for instance a regular expression that finds foo in a string and replaces it with foobar as in s/(foo)/\1bar/. The Galaxy tool Regex Find And Replace can be used to tune such a regular expression. The find part of the regex (s/(foo)/) can be defined in the Find regex parameter and the replacement part can be entered in the Replacement parameter. If we want to make the foo part of the regular expression configurable we can compose this text parameter using the Compose text parameter value tool.

Hands-on: Compose a text parameter

1. Create a new workflow

   Tip: Creating a new workflow

   1. Click Workflow on the top bar
   2. Click the new workflow galaxy-wf-new button
   3. Give it a clear and memorable name
   4. Clicking Save will take you directly into the workflow editor for that workflow
   5. Need more help? Please see the How to make a workflow subsection here

2. Add an Inputs → Input Dataset to the workflow
3. Add an Inputs → Simple inputs used for workflow logic to the workflow
   • param-select “Parameter type”: Text
4. Add Compose text parameter value tool to the workflow
   • Add two more “Components” using the “Insert components” button in the workflow editor tool form on the right of the screen. You will need three “Component” blocks in total.
   • In the first component:
     • “Enter text that should be part of the computed value”: (
   • In the second component:
     • “Enter text that should be part of the computed value”: Leave empty and click “Add connection to module” galaxy-wf-connection
     • Connect the output of the Input parameter tool to this new input
   • In the third component:
     • “Enter text that should be part of the computed value”: )
5. Add the Regex Find And Replace tool to the workflow
   • Click on Insert Check
   • “Find Regex”: Click “Add connection to module” galaxy-wf-connection
   • “Replacement”: \1bar
   • Connect the output of the Compose text parameter value tool to the “Find Regex” parameter of Regex Find And Replace tool.
   • Connect the output of the Input dataset tool to the “Select lines from” input of the Regex Find And Replace tool.
6. Save galaxy-save your workflow

You’ve now built a workflow with a parameterised input! It’s time to test it out.

Hands-on: Running the workflow

1. Upload a dataset using “Paste/Fetch data” with the contents wunder

2. Run your workflow with the following parameters

   • “Input parameter”: wunder

     This is the value that will be looked for in your input dataset.

   Tip: Running a workflow

   • Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
   • Click on the workflow-run (Run workflow) button next to your workflow
   • Configure the workflow as needed
   • Click the Run Workflow button at the top-right of the screen
   • You may have to refresh your history to see the queued jobs

3. Examine the outputs

You should see two new datasets in your history. The first dataset has the data type expression.json and contains the composed parameter value (wunder), the second dataset will contain the output of the Regex Find And Replace tool step. A click on the information details button will show the parameters for the tool. You will see that the “Find Regex” parameter will contain the values that you entered in the workflow run form. If you look at the dataset content you will see it is wunderbar.

Read a parameter from a dataset

Often times it is necessary to calculate a parameter in one step of a workflow and then to use it in another step of the same workflow. This can be accomplished by reading the parameter from a dataset (As long as it is a text, integer, float, boolean or color parameter). In this example we will construct a workflow where we calculate the sum of all values in a dataset and then divide the values in this dataset by the sum calculated in the previous step.

Hands-on: Construct Workflow with Parameters read from a dataset

1. Create a new workflow
2. Add an Input dataset tool
3. Add Datamash (operations on tabular data) tool to the workflow
   • wf-input “Input tabular dataset”: Connect the noodle from the output of the Input dataset tool to this input
   • param-repeat Operation to perform in each group
     • param-select “Type”: sum
     • param-text “On column”: 1
4. Add Parse parameter value tool to the workflow
   • “Select type of parameter to parse”: integer
   • wf-input “Input file containing parameter to parse out of”: Connect the Datamash tool output to this input
5. Add Compose text parameter value tool to the workflow
   • In the first repeat:
     • param-select “Choose the type of parameter for this field”: Text Parameter
     • param-text “Enter text that should be part of the computed value”: c1/
   • Click Insert Repeat
   • In the second repeat:
     • param-select “Choose the type of parameter for this field”: Integer Parameter
     • “Enter integer that should be part of the computed value”: Click on Add connection to module galaxy-wf-connection
     • wf-input “Input”: Connect the output of the Parse parameter value tool
6. Add Compute an expression on every row tool to the workflow
   • “Add expression as a new column to”: click on Add connection to module, then connect the output of Compose text parameter value tool
   • “as a new column to”: Select the output of the Input dataset tool

With this you’re ready to run the workflow!

Hands-on: Running the workflow

1. Upload a dataset using “Paste/Fetch data” with the contents, and set the filetype manually to “tabular” during upload

   16378
   16014
   2831
   5702
   149
   24383
   12220
   4488
   11500
   24724
   

2. Run your workflow with the following parameters

   • “Input dataset”: the table you have just uploaded

   Tip: Running a workflow

   • Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
   • Click on the workflow-run (Run workflow) button next to your workflow
   • Configure the workflow as needed
   • Click the Run Workflow button at the top-right of the screen
   • You may have to refresh your history to see the queued jobs

3. Examine the outputs

This workflow will produce a new dataset, where the last column will be the result of dividing the value in the first column by the sum of all values in the first column.

Tip: You can try many different parameter values at once

• Often times you need to try a couple of different parameter values and pick the best one. You can create or compute a dataset for each parameter you would like to try and run the Parse parameter value tool on it, running all downstream tools once for each parameter.

Conclusion

Galaxy Workflows chain together different steps of an analysis. To make your workflows more useful to your colleagues you can add workflow parameters. Sometimes a parameter is not known in advance, but can be calculated as part of the workflow. Now you know how to read these parameters from datasets!

Key points

• Use Workflow Parameters to make your Workflows more versatile

Frequently Asked Questions

Have questions about this tutorial? Check out the tutorial FAQ page or the FAQ page for the Using Galaxy and Managing your Data topic to see if your question is listed there. If not, please ask your question on the GTN Gitter Channel or the Galaxy Help Forum

Feedback

Did you use this material as an instructor? Feel free to give us feedback on how it went.
Did you use this material as a learner or student? Click the form below to leave feedback.

Citing this Tutorial

1. Marius van den Beek, Helena Rasche, 2022 Using Workflow Parameters (Galaxy Training Materials). https://training.galaxyproject.org/training-material/topics/galaxy-interface/tutorials/workflow-parameters/tutorial.html Online; accessed TODAY
2. Batut et al., 2018 Community-Driven Data Analysis Training for Biology Cell Systems 10.1016/j.cels.2018.05.012

BibTeX

@misc{galaxy-interface-workflow-parameters,
author = "Marius van den Beek and Helena Rasche",
title = "Using Workflow Parameters (Galaxy Training Materials)",
year = "2022",
month = "10",
day = "18"
url = "\url{https://training.galaxyproject.org/training-material/topics/galaxy-interface/tutorials/workflow-parameters/tutorial.html}",
note = "[Online; accessed TODAY]"
}
@article{Batut_2018,
    doi = {10.1016/j.cels.2018.05.012},
    url = {https://doi.org/10.1016%2Fj.cels.2018.05.012},
    year = 2018,
    month = {jun},
    publisher = {Elsevier {BV}},
    volume = {6},
    number = {6},
    pages = {752--758.e1},
    author = {B{\'{e}}r{\'{e}}nice Batut and Saskia Hiltemann and Andrea Bagnacani and Dannon Baker and Vivek Bhardwaj and Clemens Blank and Anthony Bretaudeau and Loraine Brillet-Gu{\'{e}}guen and Martin {\v{C}}ech and John Chilton and Dave Clements and Olivia Doppelt-Azeroual and Anika Erxleben and Mallory Ann Freeberg and Simon Gladman and Youri Hoogstrate and Hans-Rudolf Hotz and Torsten Houwaart and Pratik Jagtap and Delphine Larivi{\`{e}}re and Gildas Le Corguill{\'{e}} and Thomas Manke and Fabien Mareuil and Fidel Ram{\'{\i}}rez and Devon Ryan and Florian Christoph Sigloch and Nicola Soranzo and Joachim Wolff and Pavankumar Videm and Markus Wolfien and Aisanjiang Wubuli and Dilmurat Yusuf and James Taylor and Rolf Backofen and Anton Nekrutenko and Björn Grüning},
    title = {Community-Driven Data Analysis Training for Biology},
    journal = {Cell Systems}
}
                   

Congratulations on successfully completing this tutorial!