Skip to content

Multi-cloud comm optimization #5

@camelop

Description

@camelop

Migrated from: https://github.com/camelop/dds-dev/issues/17

Idea

When considering decentralized data science in a multi-cloud setting, we might be able to do some system level optimization for existing abstractions. For example, consider two parties running a protocol, party A has a core running on cloud AC with storage on cloud AS1, AS2 and protocol operators running on cloud AP1, AP2; similarly party B has a core on BC, storage backends on BS1, BS2 and protocol operators BP1, BP2.

  1. In normal settings, we have AC=AS1=AS2=AP1=AP2, BC=BS1=BS2=BP1=BP2. In that case, there isn't much need for a multi-cloud optimization. Because if AC=BC, everyone is using the same cloud. If AC!=BC, there's no chance for data/execution migration towards optimization.
  2. However, if party A is a large company that owns abundant cloud resource, it might also be the case that AS1!=AS2 and AS2=BP1 (e.g. party A has both storage on AWS and GCP), with a upgraded core that supports operating across clouds, an internal migration before execution might be helpful to improve performance, because BP1 can directly take data from AS2.
  • It might also be the case that a party C want to provide a service to help A, B to accelerate their protocol execution. Serving as something similar to a proxy, for setting 1, there might be CS1=AS1 and CS2=BS1 which provides an opportunity for tunneling.

Note

@suquark might have some better ideas on this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cl-designNew design, research ideas, thoughts, or paper plans.enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions