Skip to content

Edges with the same label from different source nodes silently lose data #596

Description

@Sahil-u07

If two edges in a workflow have the same file edge label but come from different source nodes, mkconcore only wires up the first source. The second source node gets no out mount for that edge at all, so whatever it writes is lost, and nothing warns about it. concore validate also says the workflow is valid.

Small example, three nodes A, B, C with edges A -> C and B -> C, both labelled shared. Building with --type docker gives this run script:

docker run --name=A -v shared:/out1 ... docker-a&
docker run --name=B docker-b&
docker run --name=C ... -v shared:/in1:ro -v shared:/in2:ro docker-c&

B has no -v shared:/out1, and C mounts the same volume twice as in1 and in2. Posix has the same problem, the build script only creates the out1 link for A.

The cause is in the loop that fills edges_dict in mkconcore.py. It stores [source, [targets]] per label and only sets the source the first time it sees the label, so the source of any later edge with that label is dropped.

None of the graphml files in the repo do this, and since an edge label is one volume with one writer it doesn't look like something that should be supported. I think it should be an error in both mkconcore and concore validate.

What it looks like on dev:

validate passes but B has no mount

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions