MLflow S3UploadFailedError: Failed to upload

I’ve created with docker a MinioS3 artifact storage and a mysql bakend storage using the next docker-compose:

    version: '3.8'
    services:
        db:
           environment:
              - MYSQL_DATABASE=${MYSQL_DATABASE}
              - MYSQL_USER=${MYSQL_USER}
              - MYSQL_PASSWORD=${MYSQL_PASSWORD}
              - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
           expose:
              - '3306'        
           volumes:
              - '(path)/server_backend:/var/lib/mysql '
           image: 'mysql'
           container_name: db

        storage:
            environment:
                - MINIO_ACCESS_KEY=${MINIO_USR}
                - MINIO_SECRET_KEY=${MINIO_PASS}
            expose:
                - '9000'
            ports:
                - '9000:9000'        
            depends_on:
                - db
            command: server /data
            volumes:
                - '(path)/server_artifact:/data'
            image: minio/minio:RELEASE.2021-02-14T04-01-33Z
            container_name: MinIO

        mlflow:
            build: ./mlflow
            environment:
                - AWS_ACCESS_KEY_ID=${MINIO_USR}
                - AWS_SECRET_ACCESS_KEY=${MINIO_PASS}       
            expose:
                - '5000'
            ports:
                - '5000:5000'
            depends_on:
                - storage                       
            image: 'mlflow:Dockerfile'
            container_name: server

The Mlflow server docker was created using the next Dockerfile:

    FROM python:3.8-slim-buster
    WORKDIR /usr/src/app
    RUN pip install cryptography mlflow psycopg2-binary boto3 pymysql
    ENV MLFLOW_S3_ENDPOINT_URL=http://storage:9000
    CMD mlflow server 
        --backend-store-uri mysql+pymysql://MLFLOW:<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="97e3f2fae7f8e5f6fbd7f3f5">[email protected]</a>:3306/DBMLFLOW 
        --default-artifact-root s3://artifacts 
        --host 0.0.0.0

The credantials are defined in a .env file.

The results of the docker-compose up command :

    [+] Running 21/22
     - mlflow Error                                                                                                                              5.6s
     - storage Pulled                                                                                                                           36.9s
       - a6b97b4963f5 Pull complete                                                                                                             24.6s
       - 13948a011eec Pull complete                                                                                                             24.7s
       - 40cdef9976a6 Pull complete                                                                                                             24.7s
       - f47162848743 Pull complete                                                                                                             24.8s
       - 5f2758d8e94c Pull complete                                                                                                             24.9s
       - c2950439edb8 Pull complete                                                                                                             25.0s
       - 1b08f8a15998 Pull complete                                                                                                             30.7s
     - db Pulled                                                                                                                                45.8s
       - 07aded7c29c6 Already exists                                                                                                             0.0s
       - f68b8cbd22de Pull complete                                                                                                              0.7s
       - 30c1754a28c4 Pull complete                                                                                                              2.1s
       - 1b7cb4d6fe05 Pull complete                                                                                                              2.2s
       - 79a41dc56b9a Pull complete                                                                                                              2.3s
       - 00a75e3842fb Pull complete                                                                                                              6.7s
       - b36a6919c217 Pull complete                                                                                                              6.8s
       - 635b0b84d686 Pull complete                                                                                                              6.8s
       - 6d24c7242d02 Pull complete                                                                                                             39.4s
       - 5be6c5edf16f Pull complete                                                                                                             39.5s
       - cb35eac1242c Pull complete                                                                                                             39.5s
       - a573d4e1c407 Pull complete                                                                                                             39.6s
    [+] Building 1.4s (7/7) FINISHED
     => [internal] load build definition from Dockerfile                                                                                         0.0s
     => => transferring dockerfile: 32B                                                                                                          0.0s
     => [internal] load .dockerignore                                                                                                            0.0s
     => => transferring context: 2B                                                                                                              0.0s
     => [internal] load metadata for docker.io/library/python:3.8-slim-buster                                                                    1.3s
     => [1/3] FROM docker.io/library/python:<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="e0d3ced8cd938c898dcd829593948592a0938881d2d5d6">[email protected]</a>:13a3f2bffb4b18ff7eda2763a3b0ba316dd82e548f52ea8b4fd11c94b97afa7d              0.0s
     => CACHED [2/3] WORKDIR /usr/src/app                                                                                                        0.0s
     => CACHED [3/3] RUN pip install cryptography mlflow psycopg2-binary boto3 pymysql                                                           0.0s
     => exporting to image                                                                                                                       0.0s
     => => exporting layers                                                                                                                      0.0s
     => => writing image sha256:76d4e4462b5c7c1826734e59a54488b56660de0dd5ecc188c308202608a8f20b                                                 0.0s
     => => naming to docker.io/library/mlflow:Dockerfile                                                                                         0.0s
    
    Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
    [+] Running 3/3
     - Container db  Created                                                                                                       0.5s
     - Container MinIO      Created                                                                                                       0.1s
     - Container server     Created                                                                                                       0.1s
    Attaching to server, MinIO, db
    db  | 2021-10-06 12:12:57+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
    db  | 2021-10-06 12:12:57+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
    db  | 2021-10-06 12:12:57+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
    db  | 2021-10-06 12:12:57+00:00 [Note] [Entrypoint]: Initializing database files
    db  | 2021-10-06T12:12:57.679527Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.26) initializing of server in progress as process 44
    db  | 2021-10-06T12:12:57.687748Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
    db  | 2021-10-06T12:12:58.230036Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
    db  | 2021-10-06T12:12:59.888820Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1 is enabled for channel mysql_main
    db  | 2021-10-06T12:12:59.889102Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1.1 is enabled for channel mysql_main
    db  | 2021-10-06T12:12:59.997461Z 6 [Warning] [MY-010453] [Server] <a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="8cfee3e3f8cce0e3efede0e4e3fff8">[email protected]</a> is created with an empty password ! Please consider switching off the --initialize-insecure option.
    MinIO      | Attempting encryption of all config, IAM users and policies on MinIO backend
    MinIO      | Endpoint: http://172.18.0.3:9000  http://127.0.0.1:9000
    MinIO      |
    MinIO      | Browser Access:
    MinIO      |    http://172.18.0.3:9000  http://127.0.0.1:9000
    MinIO      |
    MinIO      | Object API (Amazon S3 compatible):
    MinIO      |    Go:         https://docs.min.io/docs/golang-client-quickstart-guide
    MinIO      |    Java:       https://docs.min.io/docs/java-client-quickstart-guide
    MinIO      |    Python:     https://docs.min.io/docs/python-client-quickstart-guide
    MinIO      |    JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
    MinIO      |    .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
    server     | 2021/10/06 12:13:02 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
    server     | (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
    server     | (Background on this error at: https://sqlalche.me/e/14/e3q8)
    server     | Operation will be retried in 0.1 seconds
    server     | 2021/10/06 12:13:02 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
    server     | (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
    server     | (Background on this error at: https://sqlalche.me/e/14/e3q8)
    server     | Operation will be retried in 0.3 seconds
    server     | 2021/10/06 12:13:02 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
    server     | (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
    server     | (Background on this error at: https://sqlalche.me/e/14/e3q8)
    server     | Operation will be retried in 0.7 seconds
    server     | 2021/10/06 12:13:03 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
    server     | (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
    server     | (Background on this error at: https://sqlalche.me/e/14/e3q8)
    server     | Operation will be retried in 1.5 seconds
    db  | 2021-10-06 12:13:04+00:00 [Note] [Entrypoint]: Database files initialized
    db  | 2021-10-06 12:13:04+00:00 [Note] [Entrypoint]: Starting temporary server
    db  | 2021-10-06T12:13:04.422603Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.26) starting as process 93
    db  | 2021-10-06T12:13:04.439806Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
    db  | 2021-10-06T12:13:04.575773Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
    db  | 2021-10-06T12:13:04.827307Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1 is enabled for channel mysql_main
    db  | 2021-10-06T12:13:04.827865Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1.1 is enabled for channel mysql_main
    db  | 2021-10-06T12:13:04.832827Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
    db  | 2021-10-06T12:13:04.834132Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
    db  | 2021-10-06T12:13:04.841629Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
    db  | 2021-10-06T12:13:04.855748Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
    db  | 2021-10-06T12:13:04.855801Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.26'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
    db  | 2021-10-06 12:13:04+00:00 [Note] [Entrypoint]: Temporary server started.
    server     | 2021/10/06 12:13:05 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
    server     | (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
    server     | (Background on this error at: https://sqlalche.me/e/14/e3q8)
    server     | Operation will be retried in 3.1 seconds
    db  | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
    db  | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
    db  | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
    db  | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
    db  | 2021-10-06 12:13:06+00:00 [Note] [Entrypoint]: Creating database DBMLFLOW
    db  | 2021-10-06 12:13:06+00:00 [Note] [Entrypoint]: Creating user MLFLOW
    db  | 2021-10-06 12:13:06+00:00 [Note] [Entrypoint]: Giving user MLFLOW access to schema DBMLFLOW
    db  |
    db  | 2021-10-06 12:13:06+00:00 [Note] [Entrypoint]: Stopping temporary server
    db  | 2021-10-06T12:13:06.948482Z 13 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.26).
    server     | 2021/10/06 12:13:08 WARNING mlflow.store.db.utils: SQLAlchemy engine could not be created. The following exception is caught.
    server     | (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
    server     | (Background on this error at: https://sqlalche.me/e/14/e3q8)
    server     | Operation will be retried in 6.3 seconds
    db  | 2021-10-06T12:13:08.716131Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.26)  MySQL Community Server - GPL.
    db  | 2021-10-06 12:13:08+00:00 [Note] [Entrypoint]: Temporary server stopped
    db  |
    db  | 2021-10-06 12:13:08+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
    db  |
    db  | 2021-10-06T12:13:09.159115Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.26) starting as process 1
    db  | 2021-10-06T12:13:09.167405Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
    db  | 2021-10-06T12:13:09.298925Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
    db  | 2021-10-06T12:13:09.488958Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1 is enabled for channel mysql_main
    db  | 2021-10-06T12:13:09.489087Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1.1 is enabled for channel mysql_main
    db  | 2021-10-06T12:13:09.489934Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
    db  | 2021-10-06T12:13:09.490169Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
    db  | 2021-10-06T12:13:09.494728Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
    db  | 2021-10-06T12:13:09.509856Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
    db  | 2021-10-06T12:13:09.509982Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.26'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
    db  | mbind: Operation not permitted
    server     | 2021/10/06 12:13:14 INFO mlflow.store.db.utils: Creating initial MLflow database tables...
    server     | 2021/10/06 12:13:14 INFO mlflow.store.db.utils: Updating database tables
    server     | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
    server     | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
    server     | INFO  [alembic.runtime.migration] Running upgrade  -> 451aebb31d03, add metric step
    server     | INFO  [alembic.runtime.migration] Running upgrade 451aebb31d03 -> 90e64c465722, migrate user column to tags
    server     | INFO  [alembic.runtime.migration] Running upgrade 90e64c465722 -> 181f10493468, allow nulls for metric values
    server     | INFO  [alembic.runtime.migration] Running upgrade 181f10493468 -> df50e92ffc5e, Add Experiment Tags Table
    server     | INFO  [alembic.runtime.migration] Running upgrade df50e92ffc5e -> 7ac759974ad8, Update run tags with larger limit
    server     | INFO  [alembic.runtime.migration] Running upgrade 7ac759974ad8 -> 89d4b8295536, create latest metrics table
    server     | INFO  [89d4b8295536_create_latest_metrics_table_py] Migration complete!
    server     | INFO  [alembic.runtime.migration] Running upgrade 89d4b8295536 -> 2b4d017a5e9b, add model registry tables to db
    server     | INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Adding registered_models and model_versions tables to database.
    server     | INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Migration complete!
    server     | INFO  [alembic.runtime.migration] Running upgrade 2b4d017a5e9b -> cfd24bdc0731, Update run status constraint with killed
    server     | INFO  [alembic.runtime.migration] Running upgrade cfd24bdc0731 -> 0a8213491aaa, drop_duplicate_killed_constraint
    server     | INFO  [alembic.runtime.migration] Running upgrade 0a8213491aaa -> 728d730b5ebd, add registered model tags table
    server     | INFO  [alembic.runtime.migration] Running upgrade 728d730b5ebd -> 27a6a02d2cf1, add model version tags table
    server     | INFO  [alembic.runtime.migration] Running upgrade 27a6a02d2cf1 -> 84291f40a231, add run_link to model_version
    server     | INFO  [alembic.runtime.migration] Running upgrade 84291f40a231 -> a8c4a736bde6, allow nulls for run_id
    server     | INFO  [alembic.runtime.migration] Running upgrade a8c4a736bde6 -> 39d1c3be5f05, add_is_nan_constraint_for_metrics_tables_if_necessary
    server     | INFO  [alembic.runtime.migration] Running upgrade 39d1c3be5f05 -> c48cb773bb87, reset_default_value_for_is_nan_in_metrics_table_for_mysql
    server     | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
    server     | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
    db  | mbind: Operation not permitted
    server     | [2021-10-06 12:13:16 +0000] [17] [INFO] Starting gunicorn 20.1.0
    server     | [2021-10-06 12:13:16 +0000] [17] [INFO] Listening at: http://0.0.0.0:5000 (17)
    server     | [2021-10-06 12:13:16 +0000] [17] [INFO] Using worker: sync
    server     | [2021-10-06 12:13:16 +0000] [19] [INFO] Booting worker with pid: 19
    server     | [2021-10-06 12:13:16 +0000] [20] [INFO] Booting worker with pid: 20
    server     | [2021-10-06 12:13:16 +0000] [21] [INFO] Booting worker with pid: 21
    server     | [2021-10-06 12:13:16 +0000] [22] [INFO] Booting worker with pid: 22

It makes me suspect because on the second line appears - mlflow Error but i think that this is why the other builds haven’t finished.

Then I’ve set my environment variables on the client to create the information flow between my script and the storages:

    os.environ['MLFLOW_S3_ENDPOINT_URL'] = 'http://localhost:9000/'
    os.environ['AWS_ACCESS_KEY_ID'] = 'key'
    os.environ['AWS_SECRET_ACCESS_KEY'] = 'pw'
    
    remote_server_uri = "http://localhost:5000/" # server URI
    mlflow.set_tracking_uri(remote_server_uri)
    
    mlflow.set_experiment("mnist_mLflow_demo")

finally i trained a tensorflow network and i didn’t have problems storing parameters and metrics but gave me some warnings (refering to next error). But the model haven’t been auto log, so i tryed to do it manually:

    with mlflow.start_run(run_name = "test0") as run:
    
        mlflow.keras.log_model(model2, 'model2')

    mlflow.end_run()

It dosen’t work and it gives me the next INFO (but essencialy an error):

    INFO:tensorflow:Assets written to: (path)Temptmpgr5eaha2modeldatamodelassets
    INFO:tensorflow:Assets written to: (path)Temptmpgr5eaha2modeldatamodelassets
    2021/10/06 14:16:00 ERROR mlflow.utils.environment: Encountered an unexpected error while inferring pip requirements (model URI: (path)AppDataLocalTemptmpgr5eaha2model, flavor: keras)
    Traceback (most recent call last):
      File "(path)PythonPython39libsite-packagesmlflowutilsenvironment.py", line 212, in infer_pip_requirements
        return _infer_requirements(model_uri, flavor)
      File "(path)PythonPython39libsite-packagesmlflowutilsrequirements_utils.py", line 263, in _infer_requirements
        modules = _capture_imported_modules(model_uri, flavor)
      File "(path)PythonPython39libsite-packagesmlflowutilsrequirements_utils.py", line 221, in _capture_imported_modules
        _run_command(
      File "(path)PythonPython39libsite-packagesmlflowutilsrequirements_utils.py", line 163, in _run_command
        stderr = stderr.decode("utf-8")
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 349: invalid continuation byte

And the next error:

    ClientError                               Traceback (most recent call last)
    ~PythonPython39libsite-packagesboto3s3transfer.py in upload_file(self, filename, bucket, key, callback, extra_args)
        278         try:
    --> 279             future.result()
        280         # If a client error was raised, add the backwards compatibility layer
    
    ~PythonPython39libsite-packagess3transferfutures.py in result(self)
        105             # out of this and propogate the exception.
    --> 106             return self._coordinator.result()
        107         except KeyboardInterrupt as e:
    
    ~PythonPython39libsite-packagess3transferfutures.py in result(self)
        264         if self._exception:
    --> 265             raise self._exception
        266         return self._result
    
    ~PythonPython39libsite-packagess3transfertasks.py in __call__(self)
        125             if not self._transfer_coordinator.done():
    --> 126                 return self._execute_main(kwargs)
        127         except Exception as e:
    
    ~PythonPython39libsite-packagess3transfertasks.py in _execute_main(self, kwargs)
        149 
    --> 150         return_value = self._main(**kwargs)
        151         # If the task is the final task, then set the TransferFuture's
    
    ~PythonPython39libsite-packagess3transferupload.py in _main(self, client, fileobj, bucket, key, extra_args)
        693         with fileobj as body:
    --> 694             client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
        695 
    
    ~PythonPython39libsite-packagesbotocoreclient.py in _api_call(self, *args, **kwargs)
        385             # The "self" in this scope is referring to the BaseClient.
    --> 386             return self._make_api_call(operation_name, kwargs)
        387 
    
    ~PythonPython39libsite-packagesbotocoreclient.py in _make_api_call(self, operation_name, api_params)
        704             error_class = self.exceptions.from_code(error_code)
    --> 705             raise error_class(parsed_response, operation_name)
        706         else:
    
    ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The Access Key Id you provided does not exist in our records.
    
    During handling of the above exception, another exception occurred:
    
    S3UploadFailedError                       Traceback (most recent call last)
    C:UsersFCAIZA~1AppDataLocalTemp/ipykernel_7164/2476247499.py in <module>
          1 with mlflow.start_run(run_name = "test0") as run:
          2 
    ----> 3     mlflow.keras.log_model(model2, 'model2')
          4 
          5 mlflow.end_run()
    
    ~PythonPython39libsite-packagesmlflowkeras.py in log_model(keras_model, artifact_path, conda_env, custom_objects, keras_module, registered_model_name, signature, input_example, await_registration_for, pip_requirements, extra_pip_requirements, **kwargs)
        402             mlflow.keras.log_model(keras_model, "models")
        403     """
    --> 404     Model.log(
        405         artifact_path=artifact_path,
        406         flavor=mlflow.keras,
    
    ~PythonPython39libsite-packagesmlflowmodelsmodel.py in log(cls, artifact_path, flavor, registered_model_name, await_registration_for, **kwargs)
        186             mlflow_model = cls(artifact_path=artifact_path, run_id=run_id)
        187             flavor.save_model(path=local_path, mlflow_model=mlflow_model, **kwargs)
    --> 188             mlflow.tracking.fluent.log_artifacts(local_path, artifact_path)
        189             try:
        190                 mlflow.tracking.fluent._record_logged_model(mlflow_model)
    
    ~PythonPython39libsite-packagesmlflowtrackingfluent.py in log_artifacts(local_dir, artifact_path)
        582     """
        583     run_id = _get_or_start_run().info.run_id
    --> 584     MlflowClient().log_artifacts(run_id, local_dir, artifact_path)
        585 
        586 
    
    ~PythonPython39libsite-packagesmlflowtrackingclient.py in log_artifacts(self, run_id, local_dir, artifact_path)
        975             is_dir: True
        976         """
    --> 977         self._tracking_client.log_artifacts(run_id, local_dir, artifact_path)
        978 
        979     @contextlib.contextmanager
    
    ~PythonPython39libsite-packagesmlflowtracking_tracking_serviceclient.py in log_artifacts(self, run_id, local_dir, artifact_path)
        332         :param artifact_path: If provided, the directory in ``artifact_uri`` to write to.
        333         """
    --> 334         self._get_artifact_repo(run_id).log_artifacts(local_dir, artifact_path)
        335 
        336     def list_artifacts(self, run_id, path=None):
    
    ~PythonPython39libsite-packagesmlflowstoreartifacts3_artifact_repo.py in log_artifacts(self, local_dir, artifact_path)
        102                 upload_path = posixpath.join(dest_path, rel_path)
        103             for f in filenames:
    --> 104                 self._upload_file(
        105                     s3_client=s3_client,
        106                     local_file=os.path.join(root, f),
    
    ~PythonPython39libsite-packagesmlflowstoreartifacts3_artifact_repo.py in _upload_file(self, s3_client, local_file, bucket, key)
         78         if environ_extra_args is not None:
         79             extra_args.update(environ_extra_args)
    ---> 80         s3_client.upload_file(Filename=local_file, Bucket=bucket, Key=key, ExtraArgs=extra_args)
         81 
         82     def log_artifact(self, local_file, artifact_path=None):
    
    ~PythonPython39libsite-packagesboto3s3inject.py in upload_file(self, Filename, Bucket, Key, ExtraArgs, Callback, Config)
        128     """
        129     with S3Transfer(self, Config) as transfer:
    --> 130         return transfer.upload_file(
        131             filename=Filename, bucket=Bucket, key=Key,
        132             extra_args=ExtraArgs, callback=Callback)
    
    ~PythonPython39libsite-packagesboto3s3transfer.py in upload_file(self, filename, bucket, key, callback, extra_args)
        283         # client error.
        284         except ClientError as e:
    --> 285             raise S3UploadFailedError(
        286                 "Failed to upload %s to %s: %s" % (
        287                     filename, '/'.join([bucket, key]), e))
    
    S3UploadFailedError: Failed to upload (path)AppDataLocalTemptmpgr5eaha2modelconda.yaml to artifacts/1/5ae5fcef2d07432d811c3d7eb534382c/artifacts/model2/conda.yaml: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The Access Key Id you provided does not exist in our records.

Do you know how to help me with it? I have been looking all this morning but i did not find a solution. Thank you!!

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

I found the solution of this issue. It is a tricky problem due to spanish characters, my system’s user profile in “C:/” is “fcañizares” (Cañizares is my first last name). I have created another user named “fcanizares” and all is working fine. Hope you find this solution helpfull.

PS: Moral of the issue, get rid of the extrange characters!


All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x