refactor: denormalize currentMode/discovered into device columns - #1211
refactor: denormalize currentMode/discovered into device columns#1211ShradhaGupta31 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements issue #1210’s first step by denormalizing deviceInfo.currentMode and deviceInfo.discovered into dedicated device-level columns across SQL (Postgres/SQLite) and MongoDB, writing them on every insert/update while keeping the deviceinfo blob as the source of truth.
Changes:
- Add nullable
currentmode(TEXT) anddiscovered(BOOLEAN) columns via DB migration. - Wire SQL insert/update and Mongo update paths to persist the new mirror columns.
- Update the devices use case to sync mirror fields from
DeviceInfoduring DTO→entity transforms (but currently introduces compile errors).
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/usecase/sqldb/device.go | Persist currentmode/discovered on device insert/update. |
| internal/usecase/sqldb/device_test.go | Extend SQLite test schemas with the new nullable columns. |
| internal/usecase/nosqldb/mongo/device.go | Persist currentmode/discovered in Mongo updates. |
| internal/usecase/devices/usecase.go | Attempt to sync mirror columns from DeviceInfo during transforms (contains build-breaking issues). |
| internal/usecase/devices/repo_test.go | Adjust partial update expectations to include the new discovered mirror field. |
| internal/entity/device.go | Add mirror fields to the device entity model. |
| internal/app/migrations/20260820000000_migrate_device_mode_discovered.up.sql | Add currentmode/discovered columns to devices. |
| internal/app/migrations/20260820000000_migrate_device_mode_discovered.down.sql | Drop currentmode/discovered columns from devices. |
Suppressed comments (1)
internal/usecase/devices/usecase.go:350
unmarshalDeviceInfocurrently unmarshals into an undefined identifier (iDeviceInfonfo) and then returns&info, which remains empty. This is a compile-time error and breaks deviceInfo deserialization.
var info dto.DeviceInfo
if err := json.Unmarshal([]byte(raw), &iDeviceInfonfo); err != nil {
return nil, ErrDeviceUseCase.Wrap("unmarshalDeviceInfo", "failed to unmarshal deviceInfo for device "+guid, err)
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
6ae7fa9 to
6518745
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1211 +/- ##
==========================================
+ Coverage 50.82% 50.83% +0.01%
==========================================
Files 149 149
Lines 13873 13877 +4
==========================================
+ Hits 7051 7055 +4
Misses 6218 6218
Partials 604 604 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
6518745 to
25621d1
Compare
25621d1 to
ceb82d9
Compare
| Insert("devices"). | ||
| Columns("guid", "hostname", "tags", "mpsinstance", "connectionstatus", "mpsusername", "tenantid", "friendlyname", "dnssuffix", "deviceinfo", "username", "password", "mpspassword", "mebxpassword", "usetls", "allowselfsigned", "certhash"). | ||
| Values(d.GUID, d.Hostname, d.Tags, d.MPSInstance, d.ConnectionStatus, d.MPSUsername, d.TenantID, d.FriendlyName, d.DNSSuffix, d.DeviceInfo, d.Username, d.Password, d.MPSPassword, d.MEBXPassword, d.UseTLS, d.AllowSelfSigned, d.CertHash) | ||
| Columns("guid", "hostname", "tags", "mpsinstance", "connectionstatus", "mpsusername", "tenantid", "friendlyname", "dnssuffix", "deviceinfo", "username", "password", "mpspassword", "mebxpassword", "usetls", "allowselfsigned", "certhash", "currentmode", "discovered"). |
There was a problem hiding this comment.
currentmode and discovered are not included in the SELECT statement. As a result, DB reads will return empty values even when the data exists in the DB.
There was a problem hiding this comment.
This is kept intentional. currentmode/discovered are write-only mirror columns & we sync them from the deviceinfo JSON on every insert/update purely so we can filter on them in SQL. On read, the API response sources currentMode/discovered from the deviceinfo blob (which is in the SELECT) via entityToDTO, so consumers still get the real values & the empty entity fields never reach the response. Adding them to the SELECT wouldn't change any output, just duplicate data we already read from deviceinfo.
Please let me know if still should be added.
| CertHash *string `bson:"certhash"` | ||
| // Queryable mirrors of the matching deviceinfo JSON fields, synced on every insert/update. | ||
| CurrentMode string `bson:"currentmode"` | ||
| Discovered *bool `bson:"discovered"` |
There was a problem hiding this comment.
@ShradhaGupta31 : do we need to denormalize Discovered at this point. Since with CurrentMode we should be able to get activated and discovered devices.
Ex:
There are 10 devices in the DB
6 are activated in either CCM or ACM mode - for these devices CurrentMode would be either acmProvisioned or ccmProvisioned
remaining 4 devices the CurrentMode would be NotProvisioned
Discovered could help only to know whether the device was auto-discovered ( through rpc amtinfo --discover or some other mechanism )
There was a problem hiding this comment.
Yes, your understanding is right. For the current UI, total = activated + discovered and the two tabs are mutually exclusive, so a device that's discovered and then activated moves to the activated tab.
I kept the separate discovered filter for future use as it would filter was this device ever discovered on the network. If we later add UI tabs or need a backend query for all discovered devices, this filter is what we might use.
That said, if you feel it's not needed right now, I'm happy to remove it and derive discovered purely as the complement of activated.
There was a problem hiding this comment.
you could remove the discovered filter. When required we can implement
There was a problem hiding this comment.
Removed discovered denormalisation
ceb82d9 to
c20cd8b
Compare
Addresses: #1210 - Mirror the deviceinfo currentMode and discovered JSON fields into dedicated queryable device columns, synced on every insert/update from the deviceinfo blob (source of truth). - Adds nullable columns via migration and wires the write path across sqldb (Postgres/SQLite) and mongo backends. No API or behaviour change: the columns are written but not yet read. Signed-off-by: ShradhaGupta31 <shradha.gupta@intel.com>
c20cd8b to
7034e21
Compare
- Removed discovered feilds denormalisation Signed-off-by: ShradhaGupta31 <shradha.gupta@intel.com>
Addresses: #1210
ADR: https://github.com/device-management-toolkit/console/wiki/API-contract-update-&-DB-query-mechanism-to-fetch-Device-Stats
No API or behaviour change: the columns are written but not yet read.