[Enhancement]: Support different docker context

Module

Core

Proposal

% docker context ls
NAME                  DESCRIPTION                               DOCKER ENDPOINT                                  KUBERNETES ENDPOINT                                           ORCHESTRATOR
default               Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                      https://api.xpto.net:6443 (dlovison)   swarm
desktop-linux                                                   unix:///Users/dlovison/.docker/run/docker.sock                                                                 
my-remote-engine2 *                                             ssh://vagrant@192.168.56.6

When I execute my test

16:25:20,932 ERROR (testng-ExampleJdg4931) [org.testcontainers.dockerclient.DockerClientProviderStrategy] Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
16:25:20,932 ERROR (testng-ExampleJdg4931) [org.testcontainers.dockerclient.DockerClientProviderStrategy]     UnixSocketClientProviderStrategy: failed with exception InvalidConfigurationException (Could not find unix domain socket). Root cause NoSuchFileException (/var/run/docker.sock)
16:25:20,933 ERROR (testng-ExampleJdg4931) [org.testcontainers.dockerclient.DockerClientProviderStrategy] As no valid configuration was found, execution cannot continue

Require docker-java/docker-java#1946

@diegolovison

@diegolovison I think you are using Colima, right? Check the docs on how to set the env vars.

It would be very nice, if Colima would be detected automatically, just like Docker-Desktop is. I assume this is not a political thing 😘

@t1

@t1 I think this assumption is correct. My efforts so far about implementing support for Docker contexts have been well received by the maintainers. I am working on a pull request for docker-java here. Hope to be able to finish it up soon. I am also blogging a bit about my progress, starting here.

Is there any update on this? The relevant PR in docker-java was merged a couple of weeks ago and I think they've done a release 3.3.0 with that change included.

Do we just need a new release of testcontainers-java to pick this change up?

Oh cool, I missed that – indeed 3.3.0 seems to include it. And the dependency in testcontainers-java was also bumped. So yeah, I guess that's what we need!

@eddumelendez

@eddumelendez would you be able to publish a new release? (Alternatively, when does the Testcontainers project have a release scheduled next?)

I build custom release locally, but it does not work out of box, so anyway some adjustment need to be done to make it work before release

@piotrwielgolaski-tomtom

@piotrwielgolaski-tomtom Could you provide some details on what you tried and in what you saw?

@skagedal

@skagedal my scenario was simple, use docker context to connect with remote docker instance.
when I rebuild latest code I wanted to give a try, but it don't work.
it seems that test containers have concept of DockerClientProviderStrategy, but any of existing one will take into account context set by docker, maybe EnvironmentAndSystemPropertyClientProviderStrategy requires same extension or extra strategy need to be added

@skagedal

I take another swing on this and simple change in EnvironmentAndSystemPropertyClientProviderStrategy https://github.com/piotrwielgolaski-tomtom/testcontainers-java/blob/de1324ed2800eff4da326d0c23d281399d006bc0/core/src/main/java/org/testcontainers/dockerclient/EnvironmentAndSystemPropertyClientProviderStrategy.java#L51

            case "autoIgnoringUserProperties":
                dockerClientConfig = configBuilder.build();
                applicable = dockerClientConfig.getDockerHost() != null;
                break;

make it work a bit more, but you need to opt in for dockerconfig.source=autoIgnoringUserProperties

@skagedal I noticed that you focused on docker host, but all TLS is missing, so when I use remote connection over tcp with tls it will not work. It would be lovely if docker-java also support resolution of getSSLConfig based on context

@piotrwielgolaski-tomtom

@piotrwielgolaski-tomtom Right, there is a SkipTLSVerify boolean property in the context that also should have been taken into account. I missed/forgot that. When you say "all TLS", do you mean other things as well?

Oh, there is a TLSMaterial property... https://docs.docker.com/engine/context/working-with-contexts/

I never found any complete documentation on this format, does anyone know what that might be?

@skagedal

@skagedal The best that I can find https://github.com/docker/cli/blob/a0756c3c2cacebf5e5dc6454cc280c3ddf675176/docs/reference/commandline/context_inspect.md?plain=1#L42
I thin that this path if exist should be used for SSL config, of course if skip is not true.

And what would it expect to find in that path?

And what would it expect to find in that path?

I would expect that whatever docker expects to configure SSL https://github.com/skagedal/docker-java/blob/88cea47d18cce4339a202ae7ad66fe68ee00d533/docker-java-core/src/main/java/com/github/dockerjava/core/DefaultDockerClientConfig.java#LL450C58-L450C72
It should be directory with certificate

Now that 1.8.0 is out which uses docker-java 3.3, is this supposed to work out of the box? It's not working for me but maybe I'm doing something wrong.

It does not work out of box, I wrote details above.

to make it work with docker host and TLS it requires Support TLS configuration from docker context docker-java/docker-java#2105
ignoring this it still need extra handling in test containers - snippet above

Ah cool — is there a PR open for that last bit? Is that something I can work around as a user without forking testcontainers?

Support Colima out of the box

While I don't know if this supports all context use cases, #6927 is sufficient to support Colima without any configuration.

FYI in the latest build of Docker Desktop for macOS it no longer automatically sets/updates /var/run/docker.sock

To re-enable use Settings > Advanced > Allow the default Docker socket to be used (requires password).
If /var/run/docker.sock isn't valid then testcontainers fails with:

Could not find a valid Docker environment. Please check configuration. Attempted configurations were: ...
(Could not find unix domain socket). Root cause NoSuchFileException (/var/run/docker.sock)

If testcontainers supported contexts, I could use docker context use desktop-linux which points to a user specific socket: unix:///Users/$USER/.docker/run/docker.sock

what is the current status?

@kiview

In #5039, @kiview and @bsideup both mention it is possible to do it with custom Docker configurations. Would it be possible to please get some guidance on how to achieve this?

This could be related

It would seem the root of the issue is that there is some duplication between testcontainers and docker-java, as confirmed by this TODO in DockerClientProviderStrategy:

    /**
     * TODO we should consider moving this to docker-java at some point
     */
    @UnstableAPI
    protected boolean test() {

If testcontainers were to just delegate the entire process for checking whether Docker is running and whether it can connect to Docker, this should pretty much work out of the box, right?

And that seems like the right thing to do: if we're delegating all the details of connecting to a Docker daemon to Docker-java, then testcontainers shouldn't take it upon itself to try and connect to a TCP or Unix socket just to determine whether a Docker daemon is accessible: that, too, should be handled by Docker-java. And that would then work regardless of how the connection to Docker is actually made, including whether a default context is found and used.

If that's the case, this seems like a fairly simple case of re-implementing the test() method on DockerClientProviderStrategy. A naive approach might be to just call getDockerClient() and if it doesn't throw an exception, return true, otherwise return false.

I'd be happy to have a go at a PR for that. Any pointers or things I should be aware of?

I saw a note on a different issue about following what the NodeJS testcontainers had done. Is there a summary of what that is?

I think it's a reasonable ask that if you can type docker ps on your command line and see a Docker container list, you should be able to run testcontainers from a Java project build on that same command line without having to set additional environment variables.

Sharing a small workaround that makes Testcontainers follow the active docker context (colima in our case) without setting DOCKER_HOST. Tested with Testcontainers 2.0.5.

Why the built-in strategies miss it: docker-java already resolves the current context. DefaultDockerClientConfig.createDefaultConfigBuilder().build() reads currentContext from ~/.docker/config.json (or DOCKER_CONTEXT) and uses its endpoint and TLS material. EnvironmentAndSystemPropertyClientProviderStrategy builds exactly that config, but it only switches itself on when docker.host / DOCKER_HOST is set. dockerconfig.source=autoIgnoringUserProperties doesn't help either: it checks isDockerHostSetExplicitly(), which is still false because the context is only applied later, inside build().

So this strategy wraps the built-in one and drops that applicability check:

package org.testcontainers.dockerclient;

/**
 * Connects Testcontainers to the active {@code docker context} without needing {@code DOCKER_HOST}.
 * Workaround for https://github.com/testcontainers/testcontainers-java/issues/5837
 */
public class DockerContextClientProviderStrategy extends DockerClientProviderStrategy {

    private final EnvironmentAndSystemPropertyClientProviderStrategy delegate =
            new EnvironmentAndSystemPropertyClientProviderStrategy();

    @Override
    public TransportConfig getTransportConfig() {
        return delegate.getTransportConfig();
    }

    @Override
    public String getDescription() {
        return delegate.getDescription();
    }

    @Override
    protected boolean isPersistable() {
        return false;
    }
}

Register it through the SPI, on the test classpath:

src/test/resources/META-INF/services/org.testcontainers.dockerclient.DockerClientProviderStrategy

org.testcontainers.dockerclient.DockerContextClientProviderStrategy

Notes:

isApplicable() keeps the base default (true), and getPriority() keeps the base default (0). That makes it a last-resort fallback: DOCKER_HOST, /var/run/docker.sock and the other built-ins still win when they work. It's only tried when they all fail, which is exactly the "only a docker context is configured" case.
isPersistable() returns false so Testcontainers doesn't write docker.client.strategy=... into ~/.testcontainers.properties. Otherwise other projects, which don't have this class, would try to load it.
Because it reuses the delegate's TransportConfig, TLS settings from the context come along too, not just the host.
The built-in strategy classes are @Deprecated ("used by the SPI and should not be used directly"), so expect a deprecation warning.

With only docker context use colima set (no DOCKER_HOST, no docker.host in ~/.testcontainers.properties), discovery now resolves unix:///Users/<me>/.colima/default/docker.sock. DOCKER_CONTEXT=<name> works too, for a per-run override, since docker-java honours it.

Navigation Menu

Repository navigation

[Enhancement]: Support different docker context #5837

Module

Proposal

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Enhancement]: Support different docker context #5837

Description

Module

Proposal

Activity

diegolovison commented on Sep 13, 2022

eddumelendez commented on Oct 3, 2022

t1 commented on Feb 4, 2023

skagedal commented on Feb 4, 2023

chriskilding commented on Feb 27, 2023

skagedal commented on Feb 27, 2023

chriskilding commented on Feb 27, 2023

piotrwielgolaski-tomtom commented on Mar 23, 2023

skagedal commented on Mar 23, 2023

piotrwielgolaski-tomtom commented on Mar 26, 2023

piotrwielgolaski-tomtom commented on Mar 26, 2023

skagedal commented on Mar 31, 2023

skagedal commented on Mar 31, 2023

piotrwielgolaski-tomtom commented on Apr 2, 2023

skagedal commented on Apr 3, 2023

piotrwielgolaski-tomtom commented on Apr 3, 2023

glasser commented on Apr 11, 2023

piotrwielgolaski-tomtom commented on Apr 13, 2023

glasser commented on Apr 14, 2023

glasser commented on Apr 19, 2023

g0t4 commented on Jun 16, 2023

dominik-kovacs commented on Sep 5, 2023

OscarPalafox commented on Feb 20, 2024

mikiTesf commented on Feb 1, 2025

fransf-anva commented on May 6, 2026

piotrwielgolaski-tomtom commented on Oct 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions