docs: Revamp README to be simpler#761
Conversation
Updated README to improve clarity on TorchJD's features and usage, including new sections on scalarization and Jacobian descent.
KhusPatel4450
added
cc: docs
KhusPatel4450
changed the title
KhusPatel4450
removed
cc: refactor
KhusPatel4450
changed the title
PierreQuinton
left a comment
PierreQuinton
left a comment
There was a problem hiding this comment.
Thanks a lot for the PR, looking forward to improve the README.
I think the state you propose is nice, but maybe it brings something bad to the users: I'm afraid they will essentially just try some of the Scalarizers and then figure that it doesn't work for them, even though it is possible that JD would help them. It would be nice to bring back some importance to JD, saying that even if Scalarization is a good baseline, it is suggested to try using aggregators to try to resolve conflict as it can resolve problems that might be impossible to solve with standard scalarizers.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
PierreQuinton
left a comment
PierreQuinton
left a comment
There was a problem hiding this comment.
I think it would be nice to restore content about autojac and autogram at the end of the file (after the list of aggregs and scalarizers), it is an interesting piece of the lib that some people may want to use even without using aggregation.
This comment was marked as resolved.
This comment was marked as resolved.
I think it's a bit too long. Maybe we could talk about autogram, and hype people about it in like 3 lines, but just use links to the documentation for the actual examples? |
|
Hello this new commit has pretty much all comments addressed, I went in the direction of what Valerian said about the autogram engines. |
Co-authored-by: Pierre Quinton <pierre.quinton@epfl.ch>
|
I think autojac can be exaplained in that potential video we talked about, or sperately somewhere else. I think the main page though should largely stick to the two main contributions which is the scalarizers and aggregators. And maybe that should be the theme around the video too but the video should probably be more general rather than getting too too technical |
|
I really like it a lot, thanks a lot for this PR and for the changes @KhusPatel4450 ! |
| ```diff | ||
| import torch | ||
| from torch.nn import Linear, MSELoss, ReLU, Sequential | ||
| from torch.optim import SGD | ||
|
|
||
| + from torchjd.scalarization import GeometricMean | ||
|
|
||
| model = Sequential(Linear(10, 5), ReLU(), Linear(5, 1)) | ||
| optimizer = SGD(model.parameters(), lr=0.1) | ||
| criterion = MSELoss() | ||
| + scalarizer = GeometricMean() | ||
|
|
||
| inputs = torch.randn(16, 10) | ||
| task1_targets, task2_targets = torch.randn(16, 1), torch.randn(16, 1) | ||
|
|
||
| output = model(inputs) | ||
| - loss = criterion(output, task1_targets) + criterion(output, task2_targets) | ||
| - loss.backward() | ||
| + losses = torch.stack([criterion(output, task1_targets), criterion(output, task2_targets)]) | ||
| + loss = scalarizer(losses) | ||
| + loss.backward() | ||
| optimizer.step() | ||
| optimizer.zero_grad() | ||
| ``` |
There was a problem hiding this comment.
Is it possible to use the same examples as for JD, i.e. have a training loop with 8 batches of data? I would be nice if everything is the same between the two examples except for what matters: scalarization vs JD.
Co-authored-by: Pierre Quinton <pierre.quinton@epfl.ch>
Co-authored-by: Valérian Rey <31951177+ValerianRey@users.noreply.github.com>
I like the length of the new autogram section: very short, but straight to the point and a good way to make people want to try it. I think we could add a paragraph of a similar size, just before it, for autojac. |
KhusPatel4450
left a comment
KhusPatel4450
left a comment
There was a problem hiding this comment.
thank you for link change
3 participants
Updated README to improve clarity on TorchJD's features and usage, including new sections on scalarization and Jacobian descent.
EDIT by Valérian: To quickly take a look at the formatted README, click this: https://github.com/SimplexLab/TorchJD/blob/KhusPatel4450-patch-1/README.md