Error while creating an Endpoint in Sagemaker

[meg261995]

I am trying to use PytorchModel to deploy a trained model outside Sagemaker to an endpoint. This is my code

pytorch_model = PyTorchModel(model_data='s3://my-bucket/model.tar.gz', role=role,source_dir='model/code',entry_point='inference.py',framework_version='1.3.1',py_version='py3')
pytorch_model.deploy(instance_type='ml.t2.medium', initial_instance_count=1,endpoint_name='test')

I had created one successfully but figured I wanted additional packages.. So I added the requirements.txt under code, deleted the endpoint, endpoint config and tried to create the endpoint again with requirements.txt But its throwing me the below error

ERROR - %s already exists.
Please specify --force/-f option to overwrite the model archive output file.

[CaoHoangTung]

Have you managed to find the solution?

[andrew-aladjev]

Okay, so we have here the most famous cloud company in the world: Amazon. AI is trending today and Amazon sagemaker is trending too. But sagemaker is not working, it provides no logs and nobody is going to help? WAT?

[andrew-aladjev]

As from my experience sagemaker has very limited amount of supported main and third party libraries. I've figured it out reading cloud watch logs. So please watch your requirements.txt carefully.

torch==1.10.2
torchaudio==0.10.1
transformers==4.18.0

These versions are the maximum available for today. I have to deal with those ancient package version issues somehow if I want to use amazon sagemaker. This is not something that I've expected from amazon.

My next project will use OVH cloud, it has machines with GPU with 4x times cheaper than amazon. It has no sagemaker, but it also doesn't have sagemaker pains and issues.

[andrew-aladjev]

Another problem is that Amazon is using the most ancient framework and python by default! This politics is absolutely wrong, but nobody cares. For example you can open the documentation and found the following example:

model = PyTorchModel(
  model_data = f's3://{SAGEMAKER_BUCKET_NAME}/{MODEL_ARCHIVE_NAME}',
  entry_point = 'inference.py',
  role = role,
  sagemaker_session = session
)

And this code will deploy some ancient python 3.6 and torch 1.3.1. Why does Amazon use this absurd politics in software versions?

The right code will be as follows:

model = PyTorchModel(
  model_data = f's3://{SAGEMAKER_BUCKET_NAME}/{MODEL_ARCHIVE_NAME}',
  entry_point = 'inference.py',
  role = role,
  sagemaker_session = session,
  framework_version = '2.5',
  py_version = 'py311'
)

The right requirements.txt will be as follows:

torch==2.5.1
torchaudio==2.5.1
transformers==4.49.0