BUG #16033: segmentation fault when runing update

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

BUG #16033: segmentation fault when runing update

PG Bug reporting form
The following bug has been logged on the website:

Bug reference:      16033
Logged by:          Miha Vrhovnik
Email address:      [hidden email]
PostgreSQL version: 12rc1
Operating system:   Ubuntu 18.04.3
Description:        

We are launching a ne service and decided to test the new 12rc1
It happens that from time to time the postgresql crashes with segmentation
fault.


When it crashes the query running is always of the following form:
UPDATE calculation SET updated_xy = NOW(), xy = $1, types =
array_distinct(array_cat(types, $2)), failed =
array_distinct(array_cat(failed, $3))

Now the array_distinct is defined as following:

CREATE OR REPLACE FUNCTION array_distinct(anyarray)
RETURNS anyarray AS $$
    SELECT ARRAY(SELECT DISTINCT unnest($1) ORDER BY 1)
$$ LANGUAGE sql;

everything of "importance" around the UPDATE sentence is
2019-10-02 12:03:03.137 UTC [1215] LOCATION:  LogChildExit,
postmaster.c:3680
2019-10-02 12:03:03.137 UTC [1215] LOG:  00000: terminating any other active
server processes
2019-10-02 12:03:03.137 UTC [1215] LOCATION:  HandleChildCrash,
postmaster.c:3400

Reply | Threaded
Open this post in threaded view
|

Re: BUG #16033: segmentation fault when runing update

Tom Lane-2
PG Bug reporting form <[hidden email]> writes:
> We are launching a ne service and decided to test the new 12rc1
> It happens that from time to time the postgresql crashes with segmentation
> fault.

We're unlikely to be able to do anything about this without more
information.  A self-contained test case would be ideal, otherwise
maybe you could get a stack trace from the crash?

https://wiki.postgresql.org/wiki/Generating_a_stack_trace_of_a_PostgreSQL_backend

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: BUG #16033: segmentation fault when runing update

Miha Vrhovnik
On 02/10/2019 15:51, Tom Lane wrote:

> PG Bug reporting form <[hidden email]> writes:
>> We are launching a ne service and decided to test the new 12rc1
>> It happens that from time to time the postgresql crashes with segmentation
>> fault.
> We're unlikely to be able to do anything about this without more
> information.  A self-contained test case would be ideal, otherwise
> maybe you could get a stack trace from the crash?
>
> https://wiki.postgresql.org/wiki/Generating_a_stack_trace_of_a_PostgreSQL_backend
>
> regards, tom lane

FYI: This is PG12 specific I've moved the service to 11.5 and it works
without problems.

Unfortunately I'd need a bit of hand holding for generating a core as
I've added the `ulimit -c unlimited` at the top of the init.d file but
to no avail.


cat /etc/init.d/postgresql
#!/bin/sh
set -e
ulimit -c unlimited

### BEGIN INIT INFO
# Provides:             postgresql
# Required-Start:       $local_fs $remote_fs $network $time
# Required-Stop:        $local_fs $remote_fs $network $time
# Should-Start:         $syslog
# Should-Stop:          $syslog
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    PostgreSQL RDBMS server
### END INIT INFO

# Setting environment variables for the postmaster here does not work;
please
# set them in /etc/postgresql/<version>/<cluster>/environment instead.

[ -r /usr/share/postgresql-common/init.d-functions ] || exit 0

. /usr/share/postgresql-common/init.d-functions

# versions can be specified explicitly
if [ -n "$2" ]; then
     versions="$2 $3 $4 $5 $6 $7 $8 $9"
else
     get_versions
fi

case "$1" in
     start|stop|restart|reload)
         if [ "$1" = "start" ]; then
             create_socket_directory
         fi
         if [ -z "`pg_lsclusters -h`" ]; then
             log_warning_msg 'No PostgreSQL clusters exist; see "man
pg_createcluster"'
             exit 0
         fi
         for v in $versions; do
             $1 $v || EXIT=$?
         done
         exit ${EXIT:-0}
         ;;
     status)
         LS=`pg_lsclusters -h`
         # no clusters -> unknown status
         [ -n "$LS" ] || exit 4
         echo "$LS" | awk 'BEGIN {rc=0} {if (match($4, "down")) rc=3;
printf ("%s/%s (port %s): %s\n", $1, $2, $3, $4)}; END {exit rc}'
         ;;
     force-reload)
         for v in $versions; do
             reload $v
         done
         ;;
     *)
         echo "Usage: $0 {start|stop|restart|reload|force-reload|status}
[version ..]"
         exit 1
         ;;
esac

exit 0


find / -name core
/dev/core
find: ‘/sys/kernel/debug’: Permission denied
find: ‘/sys/fs/pstore’: Permission denied
find: ‘/sys/fs/fuse/connections/65’: Permission denied
find: ‘/sys/fs/fuse/connections/64’: Permission denied
/proc/sys/net/core
find: ‘/proc/tty/driver’: Permission denied


BTW this is running in unprivileged LXC container.

--
Z lepimi pozdravi / with kind regards,

Miha Vrhovnik



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16033: segmentation fault when runing update

Tom Lane-2
Miha Vrhovnik <[hidden email]> writes:
> Unfortunately I'd need a bit of hand holding for generating a core as
> I've added the `ulimit -c unlimited` at the top of the init.d file but
> to no avail.

That seems reasonable.  (Just to check off the obvious, you did
restart the postmaster service afterwards, right?)

> find / -name core

There are few if any modern platforms on which the name of a core
file is just "core" -- most people add a PID, for instance.
Searching the whole filesystem is also unlikely to be productive.
I'd just take a look into the PG data directory and see if you
see anything including "core" in its name.

Also, I don't know much about Ubuntu, but if it uses systemd then
probably systemd has commandeered collection of core files and
you need to negotiate with systemd-coredump.  (Your reference to
an init.d file makes this theory unlikely, but I mention it
for completeness.)

                        regards, tom lane