feat: allow bwa-mem to output sam format by piplus2 · Pull Request #11457 · nf-core/modules

piplus2 · 2026-05-01T09:18:10Z

PR checklist

The patch allows BWA/MEM to produce SAM format outputs when task.ext.args2 = '--output-fmt sam'. In that case, we can skip samtools view as bwa mem already produces that format.

piplus2 · 2026-05-01T10:23:22Z

There's a possible issue with the subworkflow FASTQ_CREATE_UMI_CONSENSUS_FGBIO.
Also with the untouched code, the CI fails:

🚀 nf-test 0.9.5
https://www.nf-test.com
Please cite: https://doi.org/10.1093/gigascience/giaf130
(c) 2021 - 2026 Lukas Forer and Sebastian Schoenherr

Load .nf-test/plugins/nft-anndata/0.4.1/nft-anndata-0.4.1.jar
Load .nf-test/plugins/nft-bam/0.6.1/nft-bam-0.6.1.jar
Load .nf-test/plugins/nft-csv/0.1.0/nft-csv-0.1.0.jar
Load .nf-test/plugins/nft-compress/0.1.0/nft-compress-0.1.0.jar
Load .nf-test/plugins/nft-fastq/0.1.0/nft-fastq-0.1.0.jar
Load .nf-test/plugins/nft-utils/0.0.9/nft-utils-0.0.9.jar
Load .nf-test/plugins/nft-vcf/1.0.7/nft-vcf-1.0.7.jar

Test Subworkflow FASTQ_CREATE_UMI_CONSENSUS_FGBIO

  Test [517809d8] 'single_umi' FAILED (7.818s)

  Assertion failed:

  assert workflow.success
         |        |
         workflow false

  Nextflow stdout:

  ERROR ~ Error executing process > 'FASTQ_CREATE_UMI_CONSENSUS_FGBIO:BWAMEM1_INDEX'

  Caused by:
    Input tuple does not match tuple declaration in process `FASTQ_CREATE_UMI_CONSENSUS_FGBIO:BWAMEM1_INDEX` -- offending value: [[id:genome], /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta, /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta.fai, /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.dict]



  Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

   -- Check '/mnt/c/Users/pingl/Documents/GitHub/nf-core-modules/.nf-test/tests/517809d8b122e70d816e16bb19a4ce62/meta/nextflow.log' file for details
  Nextflow stderr:



  Test [4c722dd0] 'duplex_umi' FAILED (7.009s)

  Assertion failed:

  assert workflow.success
         |        |
         workflow false

  Nextflow stdout:

  ERROR ~ Error executing process > 'FASTQ_CREATE_UMI_CONSENSUS_FGBIO:BWAMEM1_INDEX'

  Caused by:
    Input tuple does not match tuple declaration in process `FASTQ_CREATE_UMI_CONSENSUS_FGBIO:BWAMEM1_INDEX` -- offending value: [[id:genome], /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta, /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta.fai, /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.dict]



  Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

   -- Check '/mnt/c/Users/pingl/Documents/GitHub/nf-core-modules/.nf-test/tests/4c722dd06f1e8251582b24ecc780003e/meta/nextflow.log' file for details
  Nextflow stderr:



  Test [f9597a38] 'single_umi - stub' Assertion failed:

assert workflow.success
       |        |
       workflow false

java.lang.RuntimeException: Different Snapshot:
[                                                                                                       [
    {                                                                                                       {
        "consensusbam": [                                                                                       "consensusbam": [
            [                                                                                      |
                {                                                                                  <
                    "id": "test_single",                                                           <
                    "single_end": false                                                            <
                },                                                                                 <
                "test_single_consensus_unmapped.bam:md5,d41d8cd98f00b204e9800998ecf8427e"          <
            ]                                                                                      <
        ],                                                                                                      ],
        "groupbam": [                                                                                           "groupbam": [
            [                                                                                      |
                {                                                                                  <
                    "id": "test_single",                                                           <
                    "single_end": false                                                            <
                },                                                                                 <
                "test_single_umi-grouped.bam:md5,d41d8cd98f00b204e9800998ecf8427e"                 <
            ]                                                                                      <
        ],                                                                                                      ],
        "mappedconsensusbam": [                                                                                 "mappedconsensusbam": [
            [                                                                                      |
                {                                                                                  <
                    "id": "test_single",                                                           <
                    "single_end": false                                                            <
                },                                                                                 <
                "test_single.bam:md5,d41d8cd98f00b204e9800998ecf8427e"                             <
            ]                                                                                      <
        ],                                                                                                      ],
        "ubam": [                                                                                               "ubam": [
            [                                                                                      |
                {                                                                                  <
                    "id": "test_single",                                                           <
                    "single_end": false                                                            <
                },                                                                                 <
                "test_single_unaligned.bam:md5,d41d8cd98f00b204e9800998ecf8427e"                   <
            ]                                                                                      <
        ]                                                                                                       ]
    }                                                                                                       }
]                                                                                                       ]

FAILED (6.853s)

  Assertion failed:

  2 of 2 assertions failed

  Nextflow stdout:

  ERROR ~ Error executing process > 'FASTQ_CREATE_UMI_CONSENSUS_FGBIO:BWAMEM1_INDEX'

  Caused by:
    Input tuple does not match tuple declaration in process `FASTQ_CREATE_UMI_CONSENSUS_FGBIO:BWAMEM1_INDEX` -- offending value: [[id:genome], /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta, /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.fasta.fai, /nf-core/test-datasets/modules/data/genomics/homo_sapiens/genome/genome.dict]



  Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

   -- Check '/mnt/c/Users/pingl/Documents/GitHub/nf-core-modules/.nf-test/tests/f9597a38552d027b8d67056807e8428/meta/nextflow.log' file for details
  Nextflow stderr:



  Snapshots:
    Obsolete snapshots can only be checked if all tests of a file are executed successful.


FAILURE: Executed 3 tests in 22.66s (3 failed)

piplus2 · 2026-05-04T15:37:02Z

I think the CI fail because of this bug #11519

famosab · 2026-05-06T11:46:37Z

We usually want to have compressed output so having sam is not the normal output we would expect. Why do you want this to directly output sam?

piplus2 · 2026-05-06T12:40:16Z

The current code shows an inconsistency: when task.ext.args2 = '--output-fmt sam' it never returns the sam file. So this commit makes the code work as expected. If we want to exclude the sam, then it should be forbidden in the args2 or at least there should be a feedback about this behaviour.

famosab · 2026-05-06T13:03:14Z

That is a fair point! I think then we need to solve the other bug and add a test for this new output file and we should be good to go.

piplus2 · 2026-05-06T13:08:14Z

I'm working on the fix of the subworkflow that makes the CI fail.

piplus2 · 2026-05-07T09:00:09Z

the CI fails because of #11548, waiting PR #11549 to be merged.

piplus2 · 2026-05-07T11:17:08Z

I'm updating the snapshot to match the modified test, which expects the sam output too now.

famosab · 2026-05-07T11:17:21Z

Can you add a test where you actually expect the sam file in the output please? :)

I've just added process.out.sam, do you think it may be worth writing a specific test for the sam only? I did not include it to match the cram, csi and crai.

in theory we should test each output properly. So yes please add it :) The way its done for nw we can already see that sam is not created when it should not be created (which is good).

Yes, I'll do it :)

I've added 2 tests (single and paired read) to check that we actually get a sam output.

famosab · 2026-05-07T12:29:28Z

We want the ext.args to be present in the main.nf.test file. That makes everything more readable in one go. See the module specifications for more info.

Other than that good job 🥳

Oh sorry, my bad! Fixing it now!

It looks all fine now.

piplus2 requested a review from maxulysse as a code owner May 1, 2026 09:18

piplus2 force-pushed the feat-bwa-mem-output-sam branch from 7404fd2 to ccf7952 Compare May 7, 2026 08:13

piplus2 force-pushed the feat-bwa-mem-output-sam branch from ccf7952 to e8e3e1b Compare May 7, 2026 10:14

famosab reviewed May 7, 2026

View reviewed changes

piplus2 force-pushed the feat-bwa-mem-output-sam branch 2 times, most recently from 2e4c51d to c8e4f0a Compare May 7, 2026 11:38

famosab reviewed May 7, 2026

View reviewed changes

piplus2 requested a review from famosab May 7, 2026 14:54

piplus2 added 8 commits May 7, 2026 18:49

feat: allow bwa-mem to output sam format

1abf39e

chore: updated snapshots

07595e2

update: meta.yml includes sam output

6d8787f

update: added process.out.sam to match the process

b36cd8c

updated snapshots to match the new test

8a6a6da

added 2 tests for sam output

d1481d2

fix end of file

af078c9

fix: adhere to test specifications

17a2c56

piplus2 force-pushed the feat-bwa-mem-output-sam branch from 0a900f8 to 17a2c56 Compare May 7, 2026 16:49

piplus2 enabled auto-merge May 7, 2026 16:54

piplus2 self-assigned this May 7, 2026

Conversation

piplus2 commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR checklist

Uh oh!

piplus2 commented May 1, 2026

Uh oh!

piplus2 commented May 4, 2026

Uh oh!

famosab commented May 6, 2026

Uh oh!

piplus2 commented May 6, 2026

Uh oh!

famosab commented May 6, 2026

Uh oh!

piplus2 commented May 6, 2026

Uh oh!

piplus2 commented May 7, 2026

Uh oh!

piplus2 commented May 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

piplus2 commented May 1, 2026 •

edited

Loading