From NVIDIA Megatron-LM for visibility by RaymondLi0 · Pull Request #18 · bigcode-project/Megatron-LM

RaymondLi0 · 2023-01-24T20:01:13Z

No description provided.

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

…deepcopy bug from PyTorch 26.01. (#3510) Signed-off-by: Cory Ye <cye@nvidia.com>

…mba block. (#3250)

Signed-off-by: adithyare <adithyare@nvidia.com> Signed-off-by: Soumye Singhal <soumyes@cw-dfw-cs-001-dc-01.cm.cluster> Co-authored-by: Soumye Singhal <soumyes@cw-dfw-cs-001-dc-01.cm.cluster> Co-authored-by: Rabeeh Karimi Mahabadi <rkarimimahab@nvidia.com> Co-authored-by: Seonjin Na <sna@nvidia.com>

Signed-off-by: dimapihtar <dpihtar@gmail.com>

…maller batch sizes (#3527)

#3537) Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com>

Signed-off-by: dimapihtar <dpihtar@gmail.com>

… load-exchange-algo (#2161)

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Co-authored-by: anthropic-code-agent[bot] <242468646+Claude@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Co-authored-by: Xin Yao <xiny@nvidia.com>

…ate tracking (#3748) Signed-off-by: Robin Zhang <robinz@nvidia.com> Co-authored-by: Eric Harper <eharper@nvidia.com>

…#3951) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

…3947) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

…nged (#3948)

…zed MoE layers (#3941) Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>

Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster> Co-authored-by: root <root@nvl72077-T12.cm.cluster> Co-authored-by: root <root@nvl72098-T17.cm.cluster>

…3917)

#3903) Signed-off-by: Cory Ye <cye@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster> Co-authored-by: root <root@nvl72077-T12.cm.cluster> Co-authored-by: root <root@nvl72098-T17.cm.cluster> Co-authored-by: root <root@nvl72160-T18.cm.cluster> Co-authored-by: root <root@nvl72006-T11.cm.cluster>

Signed-off-by: dimapihtar <dpihtar@gmail.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

…ks. (#3976) Signed-off-by: Cory Ye <cye@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…rs (#3979)" (#3982) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12

RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare February 18, 2026 09:18

faradawn and others added 27 commits February 20, 2026 17:48

Multimodal: fix argument checking (#3449)

01b361c

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

Fix Megatron-FSDP optimizer state DCP checkpointing, and fix DTensor …

773c113

…deepcopy bug from PyTorch 26.01. (#3510) Signed-off-by: Cory Ye <cye@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

b555baf

Renable full_iteration cuda graphs for inference. Add them for the ma…

32efeff

…mba block. (#3250)

chore(beep boop 🤖): Bump (main) (2026-02-23)

7124748

Do not Slack notify for draft PRs (#3536)

3d1a4ba

Update copy-pr-bot.yaml [skip ci]

9c6b69b

remove deprecated SampleListWebdataset (#3407)

6159399

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove deprecated get_te_version (#3413)

4aeeca8

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove deprecated async_grad_allreduce param (#3412)

33df24c

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove deprecated mamba params (#3411)

eb0da52

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove deprecated params from model parallel config (#3408)

82cbd82

Signed-off-by: dimapihtar <dpihtar@gmail.com>

Remove redundant CUDA calls in the LLaVA dataloader (#3476)

a0b9c16

Inference: Create finer grained cuda-graphs with better coverage of s…

fde3b90

…maller batch sizes (#3527)

fix: skip non-tensor optimizer state entries in distrib_optimizer sav… (

23dd639

#3537) Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com>

remove is_unitialized & get_data_modulo_expert_parallel_group (#3414)

2ec295e

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove deprecated TE module (#3409)

abf97eb

Signed-off-by: dimapihtar <dpihtar@gmail.com>

remove encoder_and_decoder from enums (#3406)

7c7c9e1

Signed-off-by: dimapihtar <dpihtar@gmail.com>

chore(beep boop 🤖): Bump (main) (2026-02-24)

dd39eb5

Add knobs to choose process groups for fully-parallel-save / load and…

cb24802

… load-exchange-algo (#2161)

Fix off-by-2 error in RL sequence packing (#3551)

5f5f465

Skip unnecessary flattening for Save / Load Planner (#3263)

a8efd34

Multimodal: fix model provider (#3508)

5dc98a6

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

docs: Enable nightly docs publish (#3546)

76b200c

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Ensure type-checker understands use of Submodules in unit tests (#3425)

f721069

Use copy_signature to preserve typing of pass-through methods (#3419)

782e54b

returnL and others added 30 commits March 19, 2026 01:58

Fix incorrect HAVE_TE detection in multiple modules (#3763)

9ed8b0c

Co-authored-by: anthropic-code-agent[bot] <242468646+Claude@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

ci: Fix sso users check (#3938)

e01d886

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

move router replay doc to advanced feature part (#3929)

2276e3a

Co-authored-by: Xin Yao <xiny@nvidia.com>

Refactor VisionTECudaGraphHelper to minimize overrides and clarify st…

36b84d4

…ate tracking (#3748) Signed-off-by: Robin Zhang <robinz@nvidia.com> Co-authored-by: Eric Harper <eharper@nvidia.com>

Fix external contributor concurrency to be global across all branches (…

c64c64b

…#3951) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix 3-way merge issue that broke main (#3949)

5811854

Fix Nemo_CICD_Test not catching cancelled/skipped functional tests (#…

f9a4196

…3947) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Guard cudagraph input copy on whether data pointers have actually cha…

9ff763a

…nged (#3948)

Enforce that flashinfer cache has been installed for inference-optimi…

7688557

…zed MoE layers (#3941) Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>

chore: remove nv-grouped-gemm dependency (#3770)

40dce4e

Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Prevent failures due to prevent_retokenization (#3958)

2bee700

ultra refit (#3904)

9a60a18

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster> Co-authored-by: root <root@nvl72077-T12.cm.cluster> Co-authored-by: root <root@nvl72098-T17.cm.cluster>

[Fix][Main] Missing Assertion for moe layer recomptue in A2A Overlap (#…

45b8eac

…3917)

Move Megatron-FSDP MixedPrecisionPolicy arguments from FSDP adapter t… (

f456199

#3903) Signed-off-by: Cory Ye <cye@nvidia.com>

chore: bump FW-CI-templates to v0.80.2 (#3961)

9382abc

Signed-off-by: oliver könig <okoenig@nvidia.com>

Rename RL timers to be consistent (#3878)

5c87d9a

Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>

ci: centralize run configuration in a single configure job (#3962)

c5f9dd3

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

ci: Split unit tests into smaller groups (#3966)

70bda97

Signed-off-by: oliver könig <okoenig@nvidia.com>

common strategy simplification (#3229)

2beb593

Signed-off-by: dimapihtar <dpihtar@gmail.com>

Cudagraphs: Remove fwd_graph_input_surface weakref (#3970)

197242e

fix: interpolate version correctly in release Slack notification (#3977)

a9aa2c8

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Make args and kwargs optional positional arguments for the Module hoo…

3f59f71

…ks. (#3976) Signed-off-by: Cory Ye <cye@nvidia.com>

ci: Add core-adlr and core-nemo to megatron/training codeowners (#3979)

ade81bc

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Small quality-of-life improvements in megatron/training (#3957)

b4e4d96

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Update throughput golden values to reflect speedup (#3983)

f5b0ec6

Revert "ci: Add core-adlr and core-nemo to megatron/training codeowne…

a811ac1

…rs (#3979)" (#3982) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Add --repo flag to gh pr view in configure job (#3989)

b1c9403

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Add common pile scripts (#3902)

febd25e

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Introduce GDN to Mamba (#3535)

8f7fbe7

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

From NVIDIA Megatron-LM for visibility#18

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 6581 commits intobigcode-project:multi-query-attentionfrom
NVIDIA:main

RaymondLi0 commented Jan 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Jan 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants