Add retry logic to role synchronization

[jorsol]

Prevents syncRoles() from failing during cluster startup, crash recovery, or primary failover windows.

When a PostgreSQL instance is starting up or processing WAL logs, it will accept connections but reject write operations with a 26006 read-only transaction error. Since role syncing occurs within a reconciliation loop, failing flat during this phase creates unnecessary error noise.

This adds a retry logic around c.userSyncStrategy.ExecuteSyncRequests to not fail immediately.

Closes #3099

[FxKu]

syncing roles is not the only time write operations happen. How about implementing a retry logic around db.Exec(query) parts in users.go file like you've mentioned in your issue. Search for:

retryutil.Retry(
			constants.PostgresConnectTimeout,
			constants.PostgresConnectRetryTimeout,
			func() (bool, error) {
			...

to see how it was done in other places.

[jorsol]

Hi @FxKu, can you review again?

[jorsol]

Hi, this can be merged now?