providers: implement SOPS provider

[euphemism]

Hey all,

This implements a SOPS-backed provider, supporting the full range of SOPS capabilities (I think [I am unable to realistically test this]).

This is some LLM output that I am in the process of cleaning up, but I wanted to get the draft in front of you for some initial feedback. Instead of using a re-implementation of SOPS à la rops, this uses Rust's FFI support to interop with the Go library through a C FFI. I feel more comfortable with this from a security perspective, and it allows for utilizing the full feature set/API surface of SOPS.

Closes #5

[domenkozar]

Could we for starters use sops cli?

[euphemism]

@domenkozar to clarify: Are you saying as part of an MVP to utilize the SOPS CLI and later explore this direct integration with the SOPS code, or are you saying this is a direction you don't want to go down at all? Using the CLI means requiring having that separately installed, but I suppose that's the standard approach for the integrations supported by secretspec.

[domenkozar]

I'd also be fine using sops-ffi if it was an externally maintained crate

[euphemism]

I'd also be fine using sops-ffi if it was an externally maintained crate

Ah, but I don't want to maintain it either 😅. When I get around to it I'll refactor this to assume the sops CLI is available in the environment.

[euphemism]

@domenkozar Just listened to the Devenv 2.0 Full Time Nix episode and heard you mentioning the FFI-based approach for the SOPS integration 😅.

[domenkozar]

@domenkozar Just listened to the Devenv 2.0 Full Time Nix episode and heard you mentioning the FFI-based approach for the SOPS integration 😅.

I think it's the right way long term, but I really want to maintain it separately. Maybe we create a repo on cachix and then transfer it over?

[euphemism]

@domenkozar Just listened to the Devenv 2.0 Full Time Nix episode and heard you mentioning the FFI-based approach for the SOPS integration 😅.

I think it's the right way long term, but I really want to maintain it separately. Maybe we create a repo on cachix and then transfer it over?

Okay, I've parked the current implementation at https://github.com/euphemism/secretspec/tree/sops-integration-via-ffi. I will rework this MVP to call out to the SOPS CLI.

[euphemism]

Okay, back to square one (ish). Invokes the SOPS CLI. Still need to do a heavy review and refactor pass on all of this generated code, and flesh out the test cases.

[euphemism]

Initial review and refactor done. Getting closer, not quite yet ready to transition out of draft. Some further refactoring and tests writing remains.

[euphemism]

[euphemism]

This did not appear to be a thing?

[euphemism]

@domenkozar This is generally ready for review.

[euphemism]

I added an entry to the table here, but apart from the table this page seems vestigial? It is duplication of content available on the individual providers' pages.

[euphemism]

Relatedly, should the icons be inverted i.e. No gets a green check and Yes gets a red cross?

[euphemism]

The testing workflows are failing for a few reasons. The non-Windows ones are failing here:

• Running tests
Process exited (Failure), restarting
Restarted (attempt 1)
sh: line 1: astro: command not found

> docs@0.0.1 dev
> astro dev

Implying Node modules are not being installed, so in attempt to address this I set languages.javascript.directory = "./docs" in devenv.nix. since languages.javascript.npm.install is set to true.

The Windows run fails with:

Provider operation failed: The 'sops' CLI is not installed.
Install it from https://github.com/getsops/sops or via your package manager.

This makes sense, devenv provides sops to the environment, but devenv is not used for the Windows tests. I added a step to download the latest SOPS binary into the workspace.

[domenkozar]

uri() leaks credentials into the audit log (secretspec/src/provider/sops/mod.rs)

uri() re-emits every set field into the query string, including secrets like age_key, aws_secret_access_key, hc_vault_token, and google_oauth_access_token. The Provider::uri() contract requires it to be credential free since it goes to the audit log and fallback warnings. Every other provider already does this (onepassword drops the token and emits only account@vault, etc.).

Please emit only the file location plus non-secret attribution and drop the credential params. The values still reach sops via apply_env, so behavior is unchanged. Suggest adding a secret: bool to FieldSpec so uri() can skip secret fields and new fields must declare themselves.

(Not caught today because uri_never_echoes_a_userinfo_password only tests the URL password position, which SOPS never reads.)

[euphemism]

uri() leaks credentials into the audit log (secretspec/src/provider/sops/mod.rs)

uri() re-emits every set field into the query string, including secrets like age_key, aws_secret_access_key, hc_vault_token, and google_oauth_access_token. The Provider::uri() contract requires it to be credential free since it goes to the audit log and fallback warnings. Every other provider already does this (onepassword drops the token and emits only account@vault, etc.).

Please emit only the file location plus non-secret attribution and drop the credential params. The values still reach sops via apply_env, so behavior is unchanged. Suggest adding a secret: bool to FieldSpec so uri() can skip secret fields and new fields must declare themselves.

(Not caught today because uri_never_echoes_a_userinfo_password only tests the URL password position, which SOPS never reads.)

Oops. Added a sensitive field to FieldSpec, and I have added two tests to address this:

• test_sops_provider_sensitive_fields_correctly_marked
• test_sops_provider_uri_drops_sensitive_fields

test_sops_provider_sensitive_fields_correctly_marked validates that these fields are marked as sensitive:

• age_key
• aws_secret_access_key
• azure_client_secret
• hc_vault_token
• huawei_sdk_ak
• huawei_sdk_sk
• google_oauth_access_token

I believe this is the sum total of sensitive fields, but you should probably double check this.

[euphemism]

Error: Unable to resolve action robinraju/release-downloader@28fc21f50d76778e7023361aa1f863e7, the provided ref 28fc21f50d76778e7023361aa1f863e7 is the shortened version of a commit SHA, which is not supported. Please use the full commit SHA 28fc21f50d76778e7023361aa1f863e717d3d56f instead.

Addressed this. I think.

[domenkozar]

Thanks for the contribution! I did a deep review pass on this, including running sops 3.12/3.13 against the new code paths. The yaml/json single-file and directory happy paths work well. There are six blocking issues posted inline: dotenv format is currently broken on both read and write, format inference can panic on common file names, binary mode has no workable semantics, and the Windows CI step cannot resolve its action ref. Happy to discuss any of them.

🤖 Generated with Claude Code

[domenkozar]

sops picks its store from the file extension, while the provider parses according to the (possibly overridden) ?format= parameter. Neither get nor set passes --input-type/--output-type, so when the two disagree the provider breaks in both directions:

• With the dotenv directory layout used by the docs and test fixtures, sops://dir/{project}/.env.{profile}.enc?format=dotenv, sops treats .enc as its binary store: sops set exits 0 but the key is dropped (verified with sops 3.13.1, decrypt only re-emits the data key), so set() reports success while the secret is never stored.
• The registered example sops://secrets.enc.ini?format=json&kms_arn=... makes sops -d emit INI text, which serde_json::from_slice then fails to parse, so every get() returns an error.

sops decrypt and sops set both accept --input-type/--output-type (json, yaml, dotenv, binary). Passing --input-type whenever ?format= overrides the extension, plus --output-type json on decrypt so there is a single parse path, would fix both cases (and would allow dropping the saphyr and rust-ini parsing arms entirely). One caveat: there is no --input-type ini, so an ini format override on a non-ini extension can't be supported and should be rejected at config parse time.

[domenkozar]

Two issues with this step:

1. 28fc21f50d76778e7023361aa1f863e7 is a truncated 32-character SHA (the full commit is 28fc21f50d76778e7023361aa1f863e717d3d56f, tag v1.13). GitHub Actions removed support for abbreviated SHA refs in February 2021 (https://github.blog/changelog/2021-01-21-github-actions-short-sha-deprecation/), so this job fails at workflow resolution and Windows CI goes red for every PR.
2. Even with the ref fixed, sops.exe lands in $GITHUB_WORKSPACE (release-downloader defaults out-file-path to .) and is never added to PATH. cargo runs test binaries with their working directory set to the package root (secretspec/), and since Rust 1.58 Command::new on Windows no longer searches the current directory anyway, so Command::new("sops") hits NotFound and every sops test fails with "The 'sops' CLI is not installed".

Suggest pinning the full SHA, downloading to a fixed directory that gets appended to $GITHUB_PATH, and pinning a specific sops version rather than latest: true (Linux/macOS already get a pinned sops via devenv).

[domenkozar]

This .expect() turns a config mistake into a process abort: sops://secrets.enc without ?format= (a very common sops file naming) panics the CLI with "SOPS provider failed to infer format" inside a TryFrom that is expected to return SecretSpecError. The same pattern exists in infer_format_from_pattern (line 144), which is hit by templated paths like sops://dir/{project}/{profile}.enc, and in detect_format in mod.rs (expect plus unwrap).

These should return an error naming the unrecognized extension and the supported formats instead of panicking.

[domenkozar]

This flattening only runs for SopsFormat::Ini, so in single-file Env mode set() always passes a nested index to sops:

sops set secrets.env '["myapp"]["production"]["API_KEY"]' '"v"'

The dotenv store is flat, so sops exits 4 with "Could not marshal tree: cannot use complex value in dotenv file" (verified on 3.12.1 and 3.13.1). The default profile's two-level path ["default"]["API_KEY"] fails the same way, so single-file env storage is write-broken for every profile.

Env needs the same flattening as Ini here (or single-file env mode should be rejected up front). Note the flat write also has to line up with the Env read path, which currently ignores the key entirely (see the comment on parse_decrypted_content).