process crash when a plpython function returns unicode

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

process crash when a plpython function returns unicode

Zac-7
Hi,
I found this bug:
if you define a plpythonu function that returns a unicode python string
the server process crashes calling that function.

The example below refers to a PostgreSQL 8.1devel installation
downloaded today from nightly snapshot but I found the same problem on
PostgreSQL 8.0.1.

Bye
Zac


Example:
OS: SuSE Linux 9.1
Python version: Python 2.3.3

test=# select version();
                                         version
----------------------------------------------------------------------------------------
  PostgreSQL 8.1devel on i686-pc-linux-gnu, compiled by GCC gcc (GCC)
3.3.3 (SuSE Linux)
(1 row)

test=# CREATE FUNCTION test_unicode() RETURNS text AS
test-# $$
test$# return u'\xe0'
test$# $$ LANGUAGE plpythonu;
CREATE FUNCTION
test=# SELECT test_unicode();
server closed the connection unexpectedly
         This probably means the server terminated abnormally
         before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>



Server log (debug level 5):

DEBUG:  invoking IpcMemoryCreate(size=10387456)
DEBUG:  max_safe_fds = 985, usable_fds = 1019, already_open = 5
LOG:  database system was shut down at 2005-06-15 14:05:00 CEST
LOG:  checkpoint record is at 0/359FE4
LOG:  redo record is at 0/359FE4; undo record is at 0/0; shutdown TRUE
LOG:  next transaction ID: 638; next OID: 24578
LOG:  next MultiXactId: 1; next MultiXactOffset: 0
LOG:  database system is ready
LOG:  transaction ID wrap limit is 2147484134, limited by database "test"
DEBUG:  proc_exit(0)
DEBUG:  shmem_exit(0)
DEBUG:  exit(0)
DEBUG:  reaping dead processes
LOG:  connection received: host=[local] port=
DEBUG:  forked new backend, pid=7842 socket=6
LOG:  connection authorized: user=postgres database=test
DEBUG:  postmaster child[7842]: starting with (
DEBUG:          postgres
DEBUG:          -v196608
DEBUG:          -p
DEBUG:          test
DEBUG:  )
DEBUG:  InitPostgres
DEBUG:  StartTransaction
DEBUG:  name: unnamed; blockState:       DEFAULT; state: INPROGR,
xid/subid/cid: 638/1/0, nestlvl: 1, children: <>
DEBUG:  CommitTransaction
DEBUG:  name: unnamed; blockState:       STARTED; state: INPROGR,
xid/subid/cid: 638/1/0, nestlvl: 1, children: <>
DEBUG:  StartTransactionCommand
DEBUG:  StartTransaction
DEBUG:  name: unnamed; blockState:       DEFAULT; state: INPROGR,
xid/subid/cid: 639/1/0, nestlvl: 1, children: <>
LOG:  statement: select version();
DEBUG:  parse tree:
DETAIL:  {QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
         :resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
         :jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
         :targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 89
:funcresulttype 25
         :funcretset false :funcformat 0 :args <>} :resno 1 :resname version
         :ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
         <> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
         :limitCount <> :setOperations <> :resultRelations <>}

DEBUG:  rewritten parse tree:
DETAIL:  ({QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
         :resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
         :jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
         :targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 89
:funcresulttype 25
         :funcretset false :funcformat 0 :args <>} :resno 1 :resname version
         :ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
         <> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
         :limitCount <> :setOperations <> :resultRelations <>})

DEBUG:  plan:
DETAIL:  {RESULT :startup_cost 0.00 :total_cost 0.01 :plan_rows 1
:plan_width 0
         :targetlist ({TARGETENTRY :expr {FUNCEXPR :funcid 89
:funcresulttype 25
         :funcretset false :funcformat 0 :args <>} :resno 1 :resname version
         :ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:qual <>
         :lefttree <> :righttree <> :initPlan <> :extParam (b) :allParam (b)
         :nParamExec 0 :resconstantqual <>}

DEBUG:  CommitTransactionCommand
DEBUG:  CommitTransaction
DEBUG:  name: unnamed; blockState:       STARTED; state: INPROGR,
xid/subid/cid: 639/1/0, nestlvl: 1, children: <>
DEBUG:  checkpoint starting
DEBUG:  checkpoint complete; 0 transaction log file(s) added, 0 removed,
0 recycled
DEBUG:  StartTransactionCommand
DEBUG:  StartTransaction
DEBUG:  name: unnamed; blockState:       DEFAULT; state: INPROGR,
xid/subid/cid: 640/1/0, nestlvl: 1, children: <>
LOG:  statement: CREATE FUNCTION test_unicode() RETURNS text AS
         $$
         return u'\xe0'
         $$ LANGUAGE plpythonu;
DEBUG:  parse tree:
DETAIL:  {QUERY :commandType 5 :querySource 0 :canSetTag true :utilityStmt ?
         :resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
         :jointree <> :rowMarks <> :forUpdate false :targetList <>
:groupClause <>
         :havingQual <> :distinctClause <> :sortClause <> :limitOffset
<> :limitCount
         <> :setOperations <> :resultRelations <>}

DEBUG:  rewritten parse tree:
DETAIL:  ({QUERY :commandType 5 :querySource 0 :canSetTag true
:utilityStmt ?
         :resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
         :jointree <> :rowMarks <> :forUpdate false :targetList <>
:groupClause <>
         :havingQual <> :distinctClause <> :sortClause <> :limitOffset
<> :limitCount
         <> :setOperations <> :resultRelations <>})

DEBUG:  ProcessUtility
DEBUG:  CommitTransactionCommand
DEBUG:  CommitTransaction
DEBUG:  name: unnamed; blockState:       STARTED; state: INPROGR,
xid/subid/cid: 640/1/0, nestlvl: 1, children: <>
DEBUG:  StartTransactionCommand
DEBUG:  StartTransaction
DEBUG:  name: unnamed; blockState:       DEFAULT; state: INPROGR,
xid/subid/cid: 641/1/0, nestlvl: 1, children: <>
LOG:  statement: SELECT test_unicode();
DEBUG:  parse tree:
DETAIL:  {QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
         :resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
         :jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
         :targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 24578
:funcresulttype 25
         :funcretset false :funcformat 0 :args <>} :resno 1 :resname
test_unicode
         :ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
         <> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
         :limitCount <> :setOperations <> :resultRelations <>}

DEBUG:  rewritten parse tree:
DETAIL:  ({QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
         :resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
         :jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
         :targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 24578
:funcresulttype 25
         :funcretset false :funcformat 0 :args <>} :resno 1 :resname
test_unicode
         :ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
         <> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
         :limitCount <> :setOperations <> :resultRelations <>})

DEBUG:  plan:
DETAIL:  {RESULT :startup_cost 0.00 :total_cost 0.01 :plan_rows 1
:plan_width 0
         :targetlist ({TARGETENTRY :expr {FUNCEXPR :funcid 24578
:funcresulttype 25
         :funcretset false :funcformat 0 :args <>} :resno 1 :resname
test_unicode
         :ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:qual <>
         :lefttree <> :righttree <> :initPlan <> :extParam (b) :allParam (b)
         :nParamExec 0 :resconstantqual <>}

DEBUG:  reaping dead processes
DEBUG:  server process (PID 7842) was terminated by signal 11
LOG:  server process (PID 7842) was terminated by signal 11
LOG:  terminating any other active server processes
DEBUG:  sending SIGQUIT to process 7396
DEBUG:  sending SIGQUIT to process 7397
LOG:  connection received: host=[local] port=
FATAL:  the database system is in recovery mode
DEBUG:  proc_exit(0)
DEBUG:  shmem_exit(0)
DEBUG:  exit(0)
DEBUG:  forked new backend, pid=8613 socket=6
DEBUG:  reaping dead processes
DEBUG:  server process (PID 8613) exited with exit code 0
LOG:  all server processes terminated; reinitializing
DEBUG:  shmem_exit(0)
DEBUG:  invoking IpcMemoryCreate(size=10387456)
LOG:  database system was interrupted at 2005-06-15 14:10:18 CEST
LOG:  checkpoint record is at 0/35A03C
LOG:  redo record is at 0/35A03C; undo record is at 0/0; shutdown FALSE
LOG:  next transaction ID: 640; next OID: 24578
LOG:  next MultiXactId: 1; next MultiXactOffset: 0
LOG:  database system was not properly shut down; automatic recovery in
progress
LOG:  redo starts at 0/35A080
LOG:  record with zero length at 0/360524
LOG:  redo done at 0/3604FC
LOG:  database system is ready
LOG:  transaction ID wrap limit is 2147484134, limited by database "test"
DEBUG:  proc_exit(0)
DEBUG:  shmem_exit(0)
DEBUG:  exit(0)
DEBUG:  reaping dead processes

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to [hidden email] so that your
      message can get through to the mailing list cleanly
Reply | Threaded
Open this post in threaded view
|

Re: process crash when a plpython function returns unicode

Michael Fuhr
On Wed, Jun 15, 2005 at 02:29:27PM +0200, Zac wrote:

> if you define a plpythonu function that returns a unicode python string
> the server process crashes calling that function.

The crash happens if the Unicode string has the high bit set.  For
example, u'\x7f' doesn't cause a crash, but u'\x80' does.  Here's
a stack trace from HEAD and Python 2.4.1:

#0  0xfec6967c in PyString_AsString (op=0x0) at Objects/stringobject.c:698
#1  0xfed76a38 in PLy_function_handler (fcinfo=0xffbfde28, proc=0x47c610)
    at plpython.c:777
#2  0xfed77df4 in plpython_call_handler (fcinfo=0xffbfde28) at plpython.c:352

Lines 776-77 in plpython.c are:

    plrv_so = PyObject_Str(plrv);
    plrv_sc = PyString_AsString(plrv_so);

PyObject_Str() is documented to return NULL on failure:

http://www.python.org/doc/2.4.1/api/object.html

Apparently PyString_AsString() isn't expecting a NULL argument, so
the code should probably check the return value of PyObject_Str()
before calling PyString_AsString().

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

               http://www.postgresql.org/docs/faq