Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[FEATURE] RAdam optimizer implementation#20762

Open
Hunter-Zolomon wants to merge 3 commits intoapache:masterfrom
Hunter-Zolomon:master
Open

[FEATURE] RAdam optimizer implementation#20762
Hunter-Zolomon wants to merge 3 commits intoapache:masterfrom
Hunter-Zolomon:master

Conversation

@Hunter-Zolomon
Copy link

@Hunter-Zolomon Hunter-Zolomon commented Nov 29, 2021

Description

RAdam optimizer Python implementation as requested in #9182

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • RAdam optimizer with internal python doc.
  • Basic test added to the appropriate python file.

Comments

  • Default argument values beta1, beta2, epsilon are aligned with the current PyTorch implementation and both the Tensorflow and PyTorch implementations were referenced for the implementation.
  • Due to hardware restrictions I have not been able to run a proper test of the optimizer so if I could get some help with that, it would be appreciated.

@mxnet-bot
Copy link

Hey @Hunter-Zolomon , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [clang, windows-cpu, website, edge, miscellaneous, centos-gpu, unix-gpu, sanity, windows-gpu, centos-cpu, unix-cpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 29, 2021
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 14, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jan 16, 2022
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 16, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 14, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 1, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 16, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Apr 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

pr-awaiting-testing PR is reviewed and waiting CI build and test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants