[RFC] darktable MCP server exposing the raw engine to AI agents

[andriiryzhkov]

This adds darktable-mcp, a small headless binary (sibling to darktable-cli) that speaks the Model Context Protocol over stdio, so an AI agent can drive darktable's raw pipeline directly.

What it does

13 tools:

• introspect – list modules, dump a module's param schema (ranges + doc URL), decode/encode op_params by field name
• develop – render a raw (plus an optional module stack) to a PNG, or return per-channel stats instead
• library – list images, read an image's history, list/apply/save/import styles, export

Example – measure what an agx param does to the shadows:

{"name":"image_stats","arguments":{"input":{"path":"x.RAF"},
  "disable_tone_mappers":true,
  "stack":[{"operation":"agx","params":{"curve_target_display_black_ratio":0.0008}}]}}

Params are addressed through introspection (never fixed offsets), so it stays correct across module versions. Tool descriptions/schemas live in data/mcp-tools.json (like noiseprofiles.json) – editable without a rebuild.

Notes

• new src/mcp/, one CMake option USE_MCP (ON), depends on json-glib (already a dep)
• renders run on a throwaway duplicate; the source image is never touched
• not included: driving a live darktable GUI – this is a background worker

Feedback I'm after

• Does this belong in-tree, or better as an out-of-tree tool?
• Is the tool surface about right – too much, too little?
• Anything that would block it graduating from experimental?

[ralfbrown]

Given the controversial nature of AI-in-darktable generally, I'll suggest starting out as an out-of-tree tool. It can be migrated in later if there isn't too much pushback.

This is a huge diff.... The introspect portion might be useful to convert the parameter storage in sidecars from hex-encoded binary dumps to key:value pairs, which would allow more general manipulation of edits. That has come up before in the context of tools like a darktable variant of LRTimelapse (where IIRC it would allow smoothing exposure over a long series of timelapse frames covering changing lighting like dawn or dusk). I'll take a look when I get a chance, probably over the weekend.

[wpferguson]

The introspect portion might be useful to convert the parameter storage

It would also be useful for Lua scripts. @houz suggested it long ago, probably in reference to timelapse processing.

[TurboGit]

Correct me if I'm wrong, but here there is no controversial part. The MCP servers will act as an "input" device letting agent control AI using text prompt like: "increase brightness by 2%"...

So all in all, this is not really AI but just a way to control Darktable.

[andriiryzhkov]

Yes, it's not quite an AI tool in an understanding of what we delivered in latest release. It is a way to control Darktable similar to current CLI. It is a separate executable, again very much similar to CLI.

Just to be transparent, I developed it initially for own purposes, but thought it might be useful for others.

My use case was the following: I started using Darktable CLI with Claude for some creative looks reverse engineering experiments. It works, but agents need to do extra steps. MCP shortens that interaction.

Anyway, it is a side tool and just a proposal from my side. I can keep it for myself if there will be a strong pushback or adapt in a way we agree here if useful.

[victoryforce]

I perfectly understand @ralfbrown's concern and caution about the suspicion some users have towards anything AI-related. But in my opinion, if users have already largely accepted the introduction of AI tools in darktable, there will be no pushback against just another binary in the darktable toolkit, even if it is AI-related.

I vote for including this tool in the tree.

[rgo]

To avoid any confusion or controversy, it's important to be clear about what this adds. This is not AI and it does not change how darktable works. It is simply a new interface to darktable, similar to the existing GUI and CLI interface.

MCP is to AI applications/clients what REST APIs were to web applications: a common interface that allows tools and services to communicate in a consistent way.

MCP allows (AI or not)clients to interact with darktable through a standardised protocol.

I hope it helps, regards.

[TurboGit]

• Does this belong in-tree, or better as an out-of-tree tool?

Yes.

• Is the tool surface about right – too much, too little?

Too little :) I'm always asking for more. But I'm not sure from your description that you can fully control all iop. This would be nice to be able to add a new module, change the sliders, the settings... A way to fully control Darktable and so let IA agent later do this to demo things.

Later we could have some interaction like:

• HUMAN: What is the way to add exposure to the center part of the image?
• IA: You can use the exposure module and use a circle drawn mask on the center. Do you want me to do that for you?
• HUMAN: Yes, show me how to do that.
• IA: Give me the image-id of the image to demo this?
• HUMAN: 7854
• IA: Ok, I'll open the image, add a new exposure instance and create a circle drawn mask on the center.

The IA then control Darktable and do the above steps.

• IA: Is tha what you are after?
• HUMAN: Yes, how can I learn more about exposure module?
• IA: Let me open the on-line documentation on the right section.
• ...

That's my dream :) But this PR is a good first step.

• Anything that would block it graduating from experimental?

I don't think there is anything that would block this. That's already a nice first step, the introspection will be quite handy for devs I would say.