Calculating mem requirements for parallel queries and Docker and coredumps!

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Calculating mem requirements for parallel queries and Docker and coredumps!

Jorge Torralba
So this weekend we ran into the the docker limit of 64m for /dev/shm in one of our highest volume db's in production which we just upgraded to 11 from 9.5. The typical ERROR: could not resize shared memory segment .... was flooding our log files and we were core dumping constantly. We had to set max_parallel_workers_per = 0 in order to disable the feature. This resolved the problem but . took away a feature were looking forward to use.

Our system is 44 cores with 300 gigs of ram, 75 gig shared_buffers, 250mb work_mem and 600 max _connections although we use 270 consistent connection remainders are for redeploy that consume additional connections while others expire.

Having said all this, the parallel query options where all default values and we were brought to our knees. 

So, I cannot find any documentation anywhere that would help me calculate the size for shm_size and have docker started with that value. Are there any tios or formula to help with this ?

Lastly, is there a way to turn off postgres core dump on segfaults ? Every time we generated the could not resize error, it generated a core dump which killed IO on the system.

Thanks


Thanks,

Jorge Torralba
----------------------------

Note: This communication may contain privileged or other confidential information. If you are not the intended recipient, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this email in error and delete the copy you received. Thank You.
Reply | Threaded
Open this post in threaded view
|

Re: Calculating mem requirements for parallel queries and Docker and coredumps!

Peter Tormey

The --shm-size flag is available in docker run cli. Using k8s mount an emptyDir  to /dev/shm and set the medium as Memory.



Peter Tormey

[hidden email]






On Mon, Feb 11, 2019 at 10:31 AM Jorge Torralba <[hidden email]> wrote:
So this weekend we ran into the the docker limit of 64m for /dev/shm in one of our highest volume db's in production which we just upgraded to 11 from 9.5. The typical ERROR: could not resize shared memory segment .... was flooding our log files and we were core dumping constantly. We had to set max_parallel_workers_per = 0 in order to disable the feature. This resolved the problem but . took away a feature were looking forward to use.

Our system is 44 cores with 300 gigs of ram, 75 gig shared_buffers, 250mb work_mem and 600 max _connections although we use 270 consistent connection remainders are for redeploy that consume additional connections while others expire.

Having said all this, the parallel query options where all default values and we were brought to our knees. 

So, I cannot find any documentation anywhere that would help me calculate the size for shm_size and have docker started with that value. Are there any tios or formula to help with this ?

Lastly, is there a way to turn off postgres core dump on segfaults ? Every time we generated the could not resize error, it generated a core dump which killed IO on the system.

Thanks


Thanks,

Jorge Torralba
----------------------------

Note: This communication may contain privileged or other confidential information. If you are not the intended recipient, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this email in error and delete the copy you received. Thank You.

The information contained in this email message is PRIVATE and intended only for the personal and confidential use of the recipient named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited.  If you have received this communication in error, please notify us immediately by email, and delete the original message.