Skip to content

Commit 75524d4

Browse files
committed
test(huggingface): Added some variety in bias settings of test config for hf conversion.
1 parent cd75707 commit 75524d4

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

tests/conversion/test_configs/gpt2_config_test.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ model:
1313
ffn_hidden: 1024
1414
n_embd: 768
1515
dropout: 0.0
16-
bias: false # true # True: bias in Linears, like GPT-2. False: a bit better and faster
16+
bias: false # True: bias in Linears, like GPT-2. False: a bit better and faster
1717
attention_config:
1818
qkv_transforms:
1919
- type_hint: RotaryTransform
@@ -30,21 +30,21 @@ model:
3030
config:
3131
normalized_shape: ${model.config.n_embd}
3232
eps: 1e-5
33-
# bias: true
33+
bias: true
3434
ffn_norm:
3535
component_key: layer_norm
3636
variant_key: layer_norm
3737
config:
3838
normalized_shape: ${model.config.n_embd}
3939
eps: 1e-5
40-
# bias: true
40+
bias: false
4141
lm_head_norm:
4242
component_key: layer_norm
4343
variant_key: layer_norm
4444
config:
4545
normalized_shape: ${model.config.n_embd}
4646
eps: 1e-5
47-
# bias: true
47+
bias: true
4848

4949
checkpointed_model:
5050
component_key: model

0 commit comments

Comments
 (0)