distilbert tensor size issue .

[sahlebrahim]

for epoch in range(NUM_EPOCHS):
    
    model.train()
    
    for batch_idx, batch in enumerate(train_loader):
        
        ### Prepare data
        input_ids = batch['input_ids'].to(DEVICE)
        attention_mask = batch['attention_mask'].to(DEVICE)
        labels = batch['labels'].to(DEVICE)

        ### Forward
        outputs = model(input_ids, attention_mask=attention_mask, labels=labels) **
        loss, logits = outputs['loss'], outputs['logits']

this is returning error :

RuntimeError: The size of tensor a (3157) must match the size of tensor b (512) at non-singleton dimension 1

i did everything just like the book says

[rasbt]

Hi there,
this is an odd error. I just tried it on my computer and it seems to work for me. Based on your error message, it seems like the input may be too long for the supported context length of the model. Can you check that it has been indeed correctly truncated to 512?

(The code is here for your convenience: https://github.com/rasbt/machine-learning-book/blob/main/ch16/ch16-part3-bert.ipynb)