Skip to content
This repository was archived by the owner on Jan 21, 2025. It is now read-only.
This repository was archived by the owner on Jan 21, 2025. It is now read-only.

mask_1_flat and mask_2_flat applied to gates twice? #378

Open
Open
mask_1_flat and mask_2_flat applied to gates twice?#378
@marhlder

Description

@marhlder
marhlder
opened on Mar 31, 2022

mesh/mesh_tensorflow/transformer/moe.py

Line 1694 in 6b31c0f

gate_1 * mask_1_flat

It should not cause any issues I guess. Just unnecessary computation?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions