feat: atomic commit hook changes for relation settings#22090
feat: atomic commit hook changes for relation settings#22090
Conversation
1ad4946 to
0ddfa5d
Compare
gfouillet
left a comment
There was a problem hiding this comment.
There is a lot of duplicated code from relation state.
I wonder if we should consider the option to import relation state in unit state.
Otherwise, QA ok, but few issues spotted.
@manadart's preference is to duplicate code rather than have domains sharing state. There are limited exceptions, though this wouldn't be one. Secrets might be though. In this case, a good bit of the code was moved from the relation domain to unitstate domain. |
0ddfa5d to
69e0cf7
Compare
| ctx context.Context, in []unitstate.RelationSettings, | ||
| ) ([]internal.RelationSettings, error) { | ||
| return transform.SliceOrErr(in, func(in unitstate.RelationSettings) (internal.RelationSettings, error) { | ||
| relationUUID, err := s.getRelationUUIDByKey(ctx, in.RelationKey) |
There was a problem hiding this comment.
If we move it down to the state, then this can be done in the same txn, also avoiding more db hits.
There was a problem hiding this comment.
Reads are less of a hit than an even longer txn.
Simon also wants the relation settings calculations moved to the service.
SimonRichardson
left a comment
There was a problem hiding this comment.
We should try our best to do as much of the reads up front. If for any reason this transaction turns into a write, and then fails after the write, then we've just wasted a lot of time blocking other transactions potentially making progress. Essentially this call can tank the throughput of juju.
| // Determine the keys to set and unset. | ||
| var set []relationUnitSetting | ||
| var unset keys | ||
| for k, v := range settings { | ||
| if v == "" { | ||
| unset = append(unset, k) | ||
| } else { | ||
| set = append(set, relationUnitSetting{ | ||
| UUID: relUnitUUID, | ||
| Key: k, | ||
| Value: v, | ||
| }) | ||
| } | ||
| } |
There was a problem hiding this comment.
This should be in the service.
There was a problem hiding this comment.
I agree this is not performant, however it is how it works today. It will take iterations to fully resolve all of this code. I'll add a todo and try to get some of it updated here an in the relation domain.
| // Determine the keys to set and unset. | ||
| var set []relationApplicationSetting | ||
| var unset keys | ||
| for k, v := range settings { | ||
| if v == "" { | ||
| unset = append(unset, k) | ||
| } else { | ||
| set = append(set, relationApplicationSetting{ | ||
| UUID: endpointUUID, | ||
| Key: k, | ||
| Value: v, | ||
| }) | ||
| } | ||
| } |
There was a problem hiding this comment.
Added a todo for here and the relation domain.
| err = tx.Query(ctx, stmt, getUnitRelAndApp{UnitUUID: unitUUID, RelationUUID: relationUUID}).Get(&row) | ||
| if errors.Is(err, sqlair.ErrNoRows) { | ||
| return "", "", errors.Errorf( | ||
| "relation unit/application not found for unit %q and relation %q", |
There was a problem hiding this comment.
I'm not a fan of not knowing which you can't find.
manadart
left a comment
There was a problem hiding this comment.
Minor things, the most important of which is the error translation for non-leader failures.
I will think about what to do about the duplications we've introduced, but we will live with them for now.
| } | ||
|
|
||
| func (s *LeadershipService) getManagementCaveat(arg unitstate.CommitHookChangesArg) (func(context.Context, func(context.Context) error) error, error) { | ||
| func (s *LeadershipService) getManagementCaveat( |
There was a problem hiding this comment.
Capture/annotate these errors.
SimonRichardson
left a comment
There was a problem hiding this comment.
I'm going to block on the performance issue.
As a follow up, there will be a PR that ensures the integrity of the entities before any insertions and updates.
Will be used for commit hook functionality.
Use the relation keys in the RelationSettings to get relation UUIDs for CommitHookChanges in state. This validates the data and simplies the state methods.
setRelationApplicationAndUnitSettings is copied from the relation domain state and made a private method which can be called within a transaction. It has been adapted to get the relation unit uuid from known data rather than expecting one.
Move updates to relations settings during CommitHookChanges to the atomic version.
SetRelationApplicationAndUnitSettings has moved to the unitstate domain state package for atomic commit hook changes.
69e0cf7 to
2785841
Compare
|
/merge |
1 similar comment
|
/merge |
#22139 ## Merged PR's - #22122 @SimonRichardson - #22123 @jack-w-shaw - #22124 @SimonRichardson - #22125 @SimonRichardson - #22131 @tonyandrewmeyer - #22117 @gfouillet/v4.0/fix - #22090 @hmlanigan - #22087 @tmihoc - #22116 @jack-w-shaw - #22102 @SimonRichardson - #22043 @iyiguncevik - #22086 @SimonRichardson - #22047 @nvinuesa - #21848 @nvinuesa - #22103 @nvinuesa - #21352 @hpidcock - #22109 @kian99 - #22093 @gfouillet/v4.0/ci - #21834 @manadart - #22036 @nvinuesa - #22079 @manadart - #22046 @SimonRichardson - #22029 @SimonRichardson - #22055 @tmihoc - #22085 @SimonRichardson - #21928 @hpidcock - #22054 @manadart - #22082 @jack-w-shaw - #22068 @jack-w-shaw - #22025 @hmlanigan - #22045 @gfouillet/v4.0/fix - #22049 @wallyworld - #21985 @anvial - #22028 @hmlanigan - #22031 @gfouillet/v4.0/fix - #22022 @gfouillet/v4.0/fix - #22008 @nvinuesa - #22007 @jonathan-conder - #22027 @gfouillet/v4.0/fix - #21725 @hpidcock - #22005 @wallyworld - #21981 @tlm - #21968 @anvial - #21987 @manadart - #22014 @gfouillet/v4.0/fix - #22011 @gfouillet/v4.0/fix - #21960 @tlm - #21918 @manadart - #21925 @hmlanigan - #22009 @manadart - #21880 @manadart - #22010 @jack-w-shaw - #22001 @jack-w-shaw - #21982 @wallyworld ## Conflicts - `apiserver/facades/client/applicationoffers/applicationoffers_test.go` - `core/version/version.go` - `domain/schema/controller.go` - `domain/schema/model.go` - `scripts/win-installer/setup.iss` - `snap/snapcraft.yaml` ## Patches - Controller / `0029-tracing.PATCH.sql` - Model / `0052-resource.PATCH.sql`
#22150 Resolve comment from #22090 Just in case UUIDs retrieve in the unitstate service no longer exists, check that all still exist in state. All UUIDs added to CommitHookChangesArg need to be checked as well. Drive-by: use UnitNotFound from the application errors package rather than a local version in unit state. No longer a need for an errors package. ## Checklist - [x] ~Code style: imports ordered, good names, simple structure, etc~ - [x] ~Comments saying why design decisions were made~ - [x] ~Go unit tests, with comments saying what you're testing~ - [ ] [Integration tests](https://github.com/juju/juju/tree/main/tests), with comments saying what you're testing - [ ] [doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages ## QA steps Deploy an application and relate to another. There should be no change in behavior. ## Links **Jira card:** [JUJU-9540](https://warthogs.atlassian.net/browse/JUJU-9540) [JUJU-9540]: https://warthogs.atlassian.net/browse/JUJU-9540?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
#22137 Resolves (ToDo](#22090 (comment)) from #22090 1. refactor: move setting calculations to service Moved relation setting change/removal calculations out of the transaction path and into the service layer. This reduces transaction work and keeps the database interaction focused on persistence. The result is a cleaner separation of responsibilities and less overhead during updates. 1. fix: update method to do what the comment says Renamed the internal relation settings helper to reflect its actual behavior more clearly. Updated relation application settings handling so existing settings are replaced rather than merged. The method now deletes current settings before inserting the new values, matching the documented behavior. This change also applies to the shared SetRelationApplicationSettings path. Migration/import behavior is preserved where settings are inserted rather than updated. The implementation now aligns better with the documented contract for relation settings replacement. This change brought to light that empty values in the relation_*_setting tables where possible while some of the code used an empty value to indicate the setting should be deleted. This is partially based on whether the service code path indicated insert, insert/update, or replace/insert functionality. Added checks to ensure where empty values did not indicate a setting should be removed, the keys were dropped and a warning message logged. Due to the low probability, the table SQL will not be updated in 4.0, but in the main branch. ## Checklist - [x] Code style: imports ordered, good names, simple structure, etc - [x] Comments saying why design decisions were made - [x] Go unit tests, with comments saying what you're testing - [ ] ~[Integration tests](https://github.com/juju/juju/tree/main/tests), with comments saying what you're testing~ - [ ] ~[doc.go](https://discourse.charmhub.io/t/readme-in-packages/451) added or updated in changed packages~ ## QA steps Test Migration with unit settings, ``` $ juju_36 bootstrap localhost src $ juju_36 add-model moveme && juju_36 deploy juju-qa-dummy-source source && juju_36 deploy juju-qa-dummy-sink sink && juju_36 relate sink source && juju_36 config source token=goat $ juju bootstrap localhost dst $ juju migration src:moveme dst $ juju status Model Controller Cloud/Region Version Timestamp moveme dst localhost/localhost 4.0.6.1 20:16:51Z App Version Status Scale Charm Channel Rev Exposed Message sink active 1 juju-qa-dummy-sink latest/stable 7 no Token is goat source active 1 juju-qa-dummy-source latest/stable 6 no Token is goat Unit Workload Agent Machine Public address Ports Message sink/0* active idle 1 10.21.51.18 Token is goat source/0* active idle 0 10.21.51.205 Token is goat Machine State Address Inst id Base AZ Message 0 started 10.21.51.205 juju-ad2d53-0 ubuntu@20.04 ubuntu-nuc-two Running 1 started 10.21.51.18 juju-ad2d53-1 ubuntu@20.04 ubuntu-nuc-two Running $ juju config source token=heyya $ juju status Model Controller Cloud/Region Version Timestamp moveme dst localhost/localhost 4.0.6.1 20:17:09Z App Version Status Scale Charm Channel Rev Exposed Message sink active 1 juju-qa-dummy-sink latest/stable 7 no Token is heyya source active 1 juju-qa-dummy-source latest/stable 6 no Token is heyya Unit Workload Agent Machine Public address Ports Message sink/0* active idle 1 10.21.51.18 Token is heyya source/0* active idle 0 10.21.51.205 Token is heyya Machine State Address Inst id Base AZ Message 0 started 10.21.51.205 juju-ad2d53-0 ubuntu@20.04 ubuntu-nuc-two Running 1 started 10.21.51.18 juju-ad2d53-1 ubuntu@20.04 ubuntu-nuc-two Running ``` Test with CMR as well. ## Links **Jira card:** [JUJU-9539](https://warthogs.atlassian.net/browse/JUJU-9539) [JUJU-9539]: https://warthogs.atlassian.net/browse/JUJU-9539?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
Move updating relation setting changes in
CommitHookChangesfrom individual transactions to the single transaction in the unitstate domain call. Functionality of the relation domain'sSetRelationApplicationAndUnitSettingshas been moved/copied to the unitstate domain.It may be possible in the future to improve the performance of this functionality as it updates each relation's settings individually.
getManagementCaveatensures thatCommitHookChangesis called withEnsureLeadershipprovided that application settings need to be changed. It is verified via unit test.There is no outward change to functionality.
Clean up to rename state structures to be reusable with SQL queries.
Move to using the Unit UUID when calling the state version of CommitHookChanges.
Checklist
Integration tests, with comments saying what you're testingdoc.go added or updated in changed packagesQA steps
Links
Jira card: JUJU-9031