I have a Python (2.7) app which is started in my dockerfile:
CMD ["python","main.py"]
main.py prints some strings when it is started and goes into a loop afterwards:
print "App started"
while True:
time.sleep(1)
As long as I start the container with the -it flag, everything works as expected:
$ docker run --name=myapp -it myappimage > App started
And I can see the same output via logs later:
$ docker logs myapp > App started
If I try to run the same container with the -d flag, the container seems to start normally, but I can’t see any output:
$ docker run --name=myapp -d myappimage > b82db1120fee5f92c80000f30f6bdc84e068bafa32738ab7adb47e641b19b4d1 $ docker logs myapp $ (empty)
But the container still seems to run;
$ docker ps Container Status ... myapp up 4 minutes ...
Attach does not display anything either:
$ docker attach --sig-proxy=false myapp (working, no output)
Any ideas whats going wrong? Does “print” behave differently when ran in background?
Docker version:
Client version: 1.5.0 Client API version: 1.17 Go version (client): go1.4.2 Git commit (client): a8a31ef OS/Arch (client): linux/arm Server version: 1.5.0 Server API version: 1.17 Go version (server): go1.4.2 Git commit (server): a8a31ef
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Finally I found a solution to see Python output when running daemonized in Docker, thanks to @ahmetalpbalkan over at GitHub. Answering it here myself for further reference :
Using unbuffered output with
CMD ["python","-u","main.py"]
instead of
CMD ["python","main.py"]
solves the problem; you can see the output (both, stderr and stdout) via
docker logs myapp
now!
Method 2
In my case, running Python with -u didn’t change anything. What did the trick, however, was to set PYTHONUNBUFFERED=1 as environment variable:
docker run --name=myapp -e PYTHONUNBUFFERED=1 -d myappimage
[Edit]: Updated PYTHONUNBUFFERED=0 to PYTHONUNBUFFERED=1 after Lars’s comment. This doesn’t change the behavior and adds clarity.
Method 3
See this article which explain detail reason for the behavior:
There are typically three modes for buffering:
- If a file descriptor is unbuffered then no buffering occurs whatsoever, and function calls that read or write data occur immediately (and will block).
- If a file descriptor is fully-buffered then a fixed-size buffer is used, and read or write calls simply read or write from the buffer. The buffer isn’t flushed until it fills up.
- If a file descriptor is line-buffered then the buffering waits until it sees a newline character. So data will buffer and buffer until a n is seen, and then all of the data that buffered is flushed at that point in time. In reality there’s typically a maximum size on the buffer (just as in the fully-buffered case), so the rule is actually more like “buffer until a newline character is seen or 4096 bytes of data are encountered, whichever occurs first”.
And GNU libc (glibc) uses the following rules for buffering:
Stream Type Behavior stdin input line-buffered stdout (TTY) output line-buffered stdout (not a TTY) output fully-buffered stderr output unbuffered
So, if use -t, from docker document, it will allocate a pseudo-tty, then stdout becomes line-buffered, thus docker run --name=myapp -it myappimage could see the one-line output.
And, if just use -d, no tty was allocated, then, stdout is fully-buffered, one line App started surely not able to flush the buffer.
Then, use -dt to make stdout line buffered or add -u in python to flush the buffer is the way to fix it.
Method 4
If you want to add your print output to your Flask output when running docker-compose up, add the following to your docker compose file.
web:
environment:
- PYTHONUNBUFFERED=1
https://docs.docker.com/compose/environment-variables/
Method 5
Since I haven’t seen this answer yet:
You can also flush stdout after you print to it:
import time
if __name__ == '__main__':
while True:
print('cleaner is up', flush=True)
time.sleep(5)
Method 6
Try to add these two environment variables to your solution PYTHONUNBUFFERED=1 and PYTHONIOENCODING=UTF-8
Method 7
You can see logs on detached image if you change print to logging.
main.py:
import time
import logging
print "App started"
logging.warning("Log app started")
while True:
time.sleep(1)
Dockerfile:
FROM python:2.7-stretch ADD . /app WORKDIR /app CMD ["python","main.py"]
Method 8
As a quick fix, try this:
from __future__ import print_function
# some code
print("App started", file=sys.stderr)
This works for me when I encounter the same problems. But, to be honest, I don’t know why does this error happen.
Method 9
I had to use PYTHONUNBUFFERED=1 in my docker-compose.yml file to see the output from django runserver.
Method 10
If you aren’t using docker-compose and just normal docker instead, you can add this to your Dockerfile that is hosting a flask app
ARG FLASK_ENV="production"
ENV FLASK_ENV="${FLASK_ENV}"
PYTHONUNBUFFERED="true"
CMD [ "flask", "run" ]
Method 11
If anybody is running the python application with conda you should add --no-capture-output to the command since conda buffers to stdout by default.
ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "my-app", "python", "main.py"]
Method 12
When using python manage.py runserver for a Django application, adding environment variable PYTHONUNBUFFERED=1 solve my problem. print('helloworld', flush=True) also works for me.
However, python -u doesn’t work for me.
Method 13
Usually, we redirect it to a specific file (by mounting a volume from host and writing it to that file).
Adding a tty using -t is also fine. You need to pick it up in docker logs.
Using large log outputs, I did not have any issue with buffer storing all without putting it in dockers log.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0