Skip to content

heat: generate config for heat.conf - #137

Merged
openshift-merge-bot[bot] merged 1 commit into
openstack-k8s-operators:mainfrom
slagle:heat
Aug 31, 2026
Merged

heat: generate config for heat.conf#137
openshift-merge-bot[bot] merged 1 commit into
openstack-k8s-operators:mainfrom
slagle:heat

Conversation

@slagle

@slagle slagle commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

For the heat-engine image, heat-manage db_sync requires the default
heat.conf, otherwise it fails with the error: ERROR: No
'script_location' key found in configuration during heat db sync which
comes from alembic.

For the heat-api iamge, the keystone certificate can't be verified
without the default heat.conf in place.

Signed-off-by: James Slagle jslagle@redhat.com

For the heat-engine image, heat-manage db_sync requires the default
heat.conf, otherwise it fails with the error: ERROR: No
'script_location' key found in configuration during heat db sync which
comes from alembic.

For the heat-api iamge, the keystone certificate can't be verified
without the default heat.conf in place.

Signed-off-by: James Slagle <jslagle@redhat.com>
@slagle
slagle requested review from elfiesmelfie and rabi August 26, 2026 14:31
@rabi

rabi commented Aug 27, 2026

Copy link
Copy Markdown

Are we sure it's not some packaging issue, as I'm pretty convinced we don't need default heat.conf?

ERROR: No 'script_location' key found in configuration during heat db sync which comes from alembic.

That key is in the package bundled ini https://github.com/openstack/heat/blob/master/heat/db/alembic.ini#L3 and it's looked from there https://github.com/openstack/heat/blob/master/heat/db/migration.py#L59-L62 not from heat.conf AFAIK.

Also db_sync runs with explicit --config-dir https://github.com/openstack-k8s-operators/heat-operator/blob/main/internal/heat/dbsync.go#L32, so there won't be any need for default heat.conf.

For the heat-api iamge, the keystone certificate can't be verified without the default heat.conf in place

heat-api can always read configuration from heat.conf.d https://github.com/openstack/heat/blob/81fc6092fee459fbfe659f2b8f70dcc0c4ba09b9/heat/api/openstack/wsgi.py#L4. Default generated heat.conf would not have any certificates etc.

Do you've the built images somewhere that I can check?

@rabi

rabi commented Aug 27, 2026

Copy link
Copy Markdown

I did build the images from s2i main and the deployed with openstack-k8s-operators/openstack-operator#2042 and did not see any issues. Am I missing something?

Note: I tagged heat-api image as heat-api-cfn after build though it was not required

stack@osp-df-4:~/work/install_yamls$ oc get openstackversion -n openstack -o jsonpath='{.items[0].spec.customContainerImages}' | jq
{
  "heatAPIImage": "quay.io/rabi/openstack-heat-api@sha256:678283d431e092a7b2cc86cd977176e328ab9d9ca3c7e91d84d6f46cb04730f6",
  "heatCfnapiImage": "quay.io/rabi/openstack-heat-api-cfn@sha256:678283d431e092a7b2cc86cd977176e328ab9d9ca3c7e91d84d6f46cb04730f6",
  "heatEngineImage": "quay.io/rabi/openstack-heat-engine@sha256:098295043fad2a0b2bb8642a61f7f9e7fcfff8c2e9dd8d4371ce60d0b1cce729"
}

stack@osp-df-4:~/work/install_yamls$ oc get po | grep heat
heat-api-6c59f956f6-bl4kw                     1/1     Running     0          20m
heat-cfnapi-cc98bf45c-bc6r2                   1/1     Running     0          22m
heat-engine-75c6c7f556-2bzl4                  1/1     Running     0          22m

stack@osp-df-4:~/work/install_yamls$ oc describe po/heat-api-6c59f956f6-bl4kw | grep Image
    Image:           quay.io/rabi/openstack-heat-api@sha256:678283d431e092a7b2cc86cd977176e328ab9d9ca3c7e91d84d6f46cb04730f6
    Image ID:        quay.io/rabi/openstack-heat-api-cfn@sha256:678283d431e092a7b2cc86cd977176e328ab9d9ca3c7e91d84d6f46cb04730f6
  Normal  Pulled          26m   kubelet            spec.containers{heat-api}: Successfully pulled image "quay.io/rabi/openstack-heat-api@sha256:678283d431e092a7b2cc86cd977176e328ab9d9ca3c7e91d84d6f46cb04730f6" in 530ms (530ms including waiting). Image size: 608479537 bytes.

stack@osp-df-4:~/work/install_yamls$ oc exec po/heat-api-6c59f956f6-bl4kw -- ls -alt /etc/heat
total 4
drwxr-xr-x. 1 heat heat   25 Aug 27 04:47 .
drwxr-xr-x. 1 root root   78 Aug 27 04:47 ..
drwxrwsrwt. 3 root heat  180 Aug 27 04:47 heat.conf.d
drwxr-xr-x. 2 heat heat   37 Aug 27 04:26 templates
-rw-r-----. 1 heat heat 3130 Aug 27 04:12 api-paste.ini
stack@osp-df-4:~/work/install_yamls$ oc exec po/heat-engine-75c6c7f556-2bzl4 -- ls -alt /etc/heat
total 0
drwxr-xr-x. 1 heat heat  25 Aug 27 04:45 .
drwxr-xr-x. 1 root root  67 Aug 27 04:45 ..
drwxrwsrwt. 3 root heat 180 Aug 27 04:45 heat.conf.d
drwxr-xr-x. 2 heat heat  26 Aug 27 04:29 environment.d
drwxr-xr-x. 2 heat heat  37 Aug 27 04:29 templates

stack@osp-df-4:~/work/install_yamls$ oc exec openstackclient -- openstack stack list

stack@osp-df-4:~/work/install_yamls$

@fmount

fmount commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@openshift-ci

openshift-ci Bot commented Aug 31, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fmount

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fmount

fmount commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

I had the same issue with glance and I ended up doing #155 to create the required files that were present in the old RPM spec. This allows to not introduce changes in the operator and keep within s2i what was expected in the tcib context.

@openshift-merge-bot
openshift-merge-bot Bot merged commit dd8aa5b into openstack-k8s-operators:main Aug 31, 2026
3 checks passed
@rabi

rabi commented Sep 1, 2026

Copy link
Copy Markdown

I had the same issue with glance and I ended up doing #155 to create the required files that were present in the old RPM spec. This allows to not introduce changes in the operator and keep within s2i what was expected in the tcib context.

I don't think what has been mentioned in the linked PR is correct i.e oslo.config expects /etc/glance/glance-api.conf when prog='glance-api' is used. oslo looks for both [${project}.conf.d/, ${prog}.conf.d/] ex. glance.conf.d and glance-api.conf.d[1][2]. It would be good to know what exact error you guys encountered with glance as I would expect WSGI app too boots fine with only conf.d files present.

Also, would it be fair to ask for leaving for teams that own the components to decide whether they need it or not?

[1] https://github.com/openstack/glance/blob/master/glance/common/wsgi_app.py#L116
[2] https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L374-L375
[3] https://github.com/openstack-k8s-operators/glance-operator/blob/main/templates/common/config/00-config.conf

@slagle

slagle commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Apologies, I'll revert this. I should have closed it once the actual issues were identified, which were specific to the downstream builds.

@fmount

fmount commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

I had the same issue with glance and I ended up doing #155 to create the required files that were present in the old RPM spec. This allows to not introduce changes in the operator and keep within s2i what was expected in the tcib context.

I don't think what has been mentioned in the linked PR is correct i.e oslo.config expects /etc/glance/glance-api.conf when prog='glance-api' is used. oslo looks for both [${project}.conf.d/, ${prog}.conf.d/] ex. glance.conf.d and glance-api.conf.d[1][2]. It would be good to know what exact error you guys encountered with glance as I would expect WSGI app too boots fine with only conf.d files present.

Also, would it be fair to ask for leaving for teams that own the components to decide whether they need it or not?

[1] https://github.com/openstack/glance/blob/master/glance/common/wsgi_app.py#L116 [2] https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L374-L375 [3] https://github.com/openstack-k8s-operators/glance-operator/blob/main/templates/common/config/00-config.conf

@rabi apologies as well. I think the glance error was different from the issue you saw in heat, I might have waited more before landing.
In glance I saw the container going in crashloopbackoff regardless of the backend due to the missing glance-api.conf in /etc/glance. As you have mentioned in your previous reply, we do create all the config files in /etc/glance/glance.conf.d, so I expected everything to work fine w/o placing anything other than that directory under /etc/glance.
However it kept failing with:

[Mon Aug 31 12:44:52.383460 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026] mod_wsgi (pid=6): Failed to exec Python script file '/var/www/cgi-bin/glance/glance-wsgi'.
[Mon Aug 31 12:44:52.383548 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026] mod_wsgi (pid=6): Exception occurred processing WSGI script '/var/www/cgi-bin/glance/glance-wsgi'.
[Mon Aug 31 12:44:52.383996 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026] Traceback (most recent call last):
[Mon Aug 31 12:44:52.384333 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]   File "/var/www/cgi-bin/glance/glance-wsgi", line 4, in <module>
[Mon Aug 31 12:44:52.384358 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]     from glance.wsgi.api import application
[Mon Aug 31 12:44:52.384374 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]   File "/usr/lib/python3.12/site-packages/glance/wsgi/api.py", line 22, in <module>
[Mon Aug 31 12:44:52.384380 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]     application = wsgi_app.init_app()
[Mon Aug 31 12:44:52.384385 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]                   ^^^^^^^^^^^^^^^^^^^
[Mon Aug 31 12:44:52.384396 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]   File "/usr/lib/python3.12/site-packages/glance/common/wsgi_app.py", line 116, in init_app
[Mon Aug 31 12:44:52.384402 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]     CONF([], project='glance', prog='glance-api',
[Mon Aug 31 12:44:52.384412 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]   File "/usr/lib/python3.12/site-packages/oslo_config/cfg.py", line 2847, in __call__
[Mon Aug 31 12:44:52.384418 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026]     raise ConfigFilesNotFoundError(self._namespace._files_not_found)
[Mon Aug 31 12:44:52.384438 2026] [wsgi:error] [pid 6:tid 34] [remote 10.217.0.2:51026] oslo_config.cfg.ConfigFilesNotFoundError: Failed to find some config files: /etc/glance/glance-api.conf

and only after generating that file we passed that point and we were able to see the configured backend defined under /etc/glance/glance.conf.d.
Not sure I misreading how config is actually processed, but by placing the file we were able to move forward.
Is there anything missing in your opinion?

@rabi

rabi commented Sep 2, 2026

Copy link
Copy Markdown

Is there anything missing in your opinion?

The issue is that glance is passing default_config_files=config_files to https://github.com/openstack/glance/blame/master/glance/common/wsgi_app.py#L116 instead of relying on native auto-discovery. Because glance explicitly requests these files, oslo.config enforces that they must exist, throwing ConfigFilesNotFoundError if they don't.

There is a hack in https://github.com/openstack/glance/blame/master/glance/common/wsgi_app.py#L49-L50 that manually searches for ['glance-image-import.conf', 'glance-api.conf'] in /etc/glance. This manual search is only necessary because oslo.config auto-discovery won't automatically pick up glance-image-import.conf if it's placed directly in /etc/glance.

However, if glance-image-import.conf is placed inside glance.conf.d (or if its contents are merged into the main config), oslo.config will automatically pick it up. We actually already merged the import configs
in glance-operator PR #295, so we no longer need the standalone file in /etc/glance/.

Therefore, the upstream code needs to be patched. Instead of dropping the manual search entirely (which would break backward compatibility for non-rhoso deployments still relying on
/etc/glance/glance-image-import.conf), upstream should simply verify the files exist before passing them to default_config_files. This prevents oslo.config from crashing when files are missing, allowing
.conf.d/ setups to work seamlessly.

The upstream fix in glance/common/wsgi_app.py would look like this:

  def _get_config_files(env=None):                                                                                                                                                                                 
      if env is None:                                                                                                                                                                                              
          env = os.environ                                                                                                                                                                                         
      dirname = env.get('OS_GLANCE_CONFIG_DIR', '/etc/glance').strip()                                                                                                                                             
      config_files = []                                                                                                                                                                                            
      for config_file in CONFIG_FILES:                                                                                                                                                                             
          cfg_file = os.path.join(dirname, config_file)                                                                                                                                                            
          if os.path.exists(cfg_file):                                                                                                                                                                             
              config_files.append(cfg_file)                                                                                                                                                                        
                                                                                                                                                                                                                   
      return config_files                                                                                                                                                                                          
  ```"                                         

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants