can't find python packages installed into customized docker image

I am creating a Docker container that runs Python 3.6.15 and the pip install function in my Dockerfile runs during the build process but when I try to execute functions within it after the build completes and I run it the ‘installed’ packages do not exist.

For more context, here is my Dockerfile. For clarity, I am building a Docker container that is being uploaded to AWS ECR to be used in a Lambda function but I don’t think that’s entirely relevant to this question (good for context though):

# Define function directory
ARG FUNCTION_DIR="/function"

FROM python:3.6 as build-image

# Install aws-lambda-cpp build dependencies
RUN apt-get clean && apt-get update && 
  apt-get install -y 
  g++ 
  make 
  cmake 
  unzip 
  libcurl4-openssl-dev 
  ffmpeg

# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Create function directory
RUN mkdir -p ${FUNCTION_DIR}

# Copy function code
COPY . ${FUNCTION_DIR}

# Install the runtime interface client
RUN /usr/local/bin/python -m pip install 
        --target ${FUNCTION_DIR} 
        awslambdaric

# Install the runtime interface client
COPY requirements.txt /requirements.txt
RUN /usr/local/bin/python -m pip install -r requirements.txt

# Multi-stage build: grab a fresh copy of the base image
FROM python:3.6

# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}

# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

COPY entry-point.sh /entry_script.sh
ADD aws-lambda-rie /usr/local/bin/aws-lambda-rie
ENTRYPOINT [ "/entry_script.sh" ]

CMD [ "app.handler" ]

When I run my docker run command in Terminal, I can see that it is collecting and installing the packages from the requirements.txt file that is in my project’s root. I then try to run an get an Import Module error. To troubleshoot, I ran some command line exec functions such as:

docker exec <container-id> bash -c "ls"  # This returns the folder structure which looks great

docker exec <container-id> bash -c "pip freeze". # This only returns 'pip', 'wheel' and some other basic Python modules

The only why I could solve it is that after I build and run it, I run this command:

docker exec <container-id> bash -c "/usr/local/bin/python -m pip install -r requirements.txt"

Which manually installs the modules and they then show up in the freeze command and I can execute the code. This is not ideal as I would like to have pip install run correctly during the build process so there are less steps in the future as I make changes to the code.

Any pointers as to where I am going wrong would be great, thank you!

Contents hide

Answers:

Method 1

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

According to Docker Docs, multi-stage builds

With multi-stage builds, you use multiple FROM statements in your
Dockerfile. Each FROM instruction can use a different base, and each
of them begins a new stage of the build. You can selectively copy
artifacts from one stage to another, leaving behind everything you
don’t want in the final image.

So the 2nd from python:3.6 in the Dockerfile resets the image build, deleting the module installations.

The subsequent copy saves what was in /function (the aws module) but not the other modules saved to the system in the other pip install.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

0 0 votes

Article Rating