I have a NodeJS application running on a k8s pod. The actual size of the pod is 2GB, but in environment variables, we set this value to 4GB --max-old-space-size=4096 (which will not be true in my case – for some tenants we do allocate 4GB but most pods have 2GB).
I am using node js to download large files(300MB) from a server and pipe the response to a file write stream. As far as I understand pipes in nodejs, the data flow is managed by node and I don’t have to consider draining and other events. The issue I face is that the memory usage of the docker where my application is running increases in the same amount as the file being downloaded (i.e It seems the file is being saved in memory). This memory usage persists even when I delete the file in the docker. I am attaching the code used for creating request and piping, below for reference. The code is running fine but causing performance issues like huge memory/CPU usage and crashes with OOM error. I am not able to understand what I am doing wrong.
My NodeJS application currently uses a Sequelize object for working across multiple stages. This object is the result of a Model.findOne operation, which includes several other models, similar to a left join with multiple tables in plain SQL.
Today I ran my script for filesystem indexing to refresh RAID files index and after 4h it crashed with following error:
If I use gcore to make a code dump of a Node.js process, what are the best tools to analyze it? Inspired by: Tool for analyzing java core dump In my specific case, I’m interested in investigating some memory leaks, so I’m really curious to get some heap analysis. General tools and even instrumentation packages … Read more
