BUG #15781: subselect on foreign table (postgres_fdw) can crash (segfault)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

BUG #15781: subselect on foreign table (postgres_fdw) can crash (segfault)

PG Doc comments form
The following bug has been logged on the website:

Bug reference:      15781
Logged by:          Sean Johnston
Email address:      [hidden email]
PostgreSQL version: 11.2
Operating system:   docker image postgres:11.2
Description:        

Example Query:

select exists(select c1 from ft4), avg(c1) from ft4 where c1 = (select
max(c1) from ft4);

Full Steps (modified from the postgres_fdw regression tests):

CREATE EXTENSION postgres_fdw;

CREATE SERVER testserver1 FOREIGN DATA WRAPPER postgres_fdw;
 DO $d$
        BEGIN
        EXECUTE $$CREATE SERVER loopback FOREIGN DATA WRAPPER postgres_fdw
        OPTIONS (dbname '$$||current_database()||$$',
                port '$$||current_setting('port')||$$'
        )$$;
        END;
$d$;
CREATE USER MAPPING FOR CURRENT_USER SERVER loopback;

CREATE SCHEMA "S 1";

CREATE TABLE "S 1"."T 3" (
  c1 int NOT NULL,
  c2 int NOT NULL,
  c3 text,
  CONSTRAINT t3_pkey PRIMARY KEY (c1)
);

CREATE FOREIGN TABLE ft4 (
  c1 int NOT NULL,
  c2 int NOT NULL,
  c3 text
) SERVER loopback OPTIONS (schema_name 'S 1', table_name 'T 3');

select exists(select c1 from ft4), avg(c1) from ft4 where c1 = (select
max(c1) from ft4);

Reply | Threaded
Open this post in threaded view
|

Re: BUG #15781: subselect on foreign table (postgres_fdw) can crash (segfault)

Sergei Kornilov
Hi

I can reproduce this on REL_11_STABLE and HEAD.

Here is backtrace from REL_11_STABLE:

#0  CheckVarSlotCompatibility (slot=slot@entry=0x0, attnum=1, vartype=16) at execExprInterp.c:1867
#1  0x00005611db3cb342 in CheckExprStillValid (state=state@entry=0x5611dd0fa368, econtext=econtext@entry=0x5611dd0f9730) at execExprInterp.c:1831
#2  0x00005611db3cb36e in ExecInterpExprStillValid (state=0x5611dd0fa368, econtext=0x5611dd0f9730, isNull=0x7ffc524ca89f) at execExprInterp.c:1780
#3  0x00007fc3648bac8d in ExecEvalExpr (isNull=0x7ffc524ca89f, econtext=0x5611dd0f9730, state=<optimized out>)
    at ../../src/include/executor/executor.h:294
#4  process_query_params (econtext=0x5611dd0f9730, param_flinfo=0x5611dd0fa2d0, param_exprs=<optimized out>,
    param_values=param_values@entry=0x5611dd0fad50) at postgres_fdw.c:4124
#5  0x00007fc3648baf82 in create_cursor (node=<optimized out>) at postgres_fdw.c:3148
#6  0x00007fc3648bb041 in postgresIterateForeignScan (node=0x5611dd0f9618) at postgres_fdw.c:1451
#7  0x00005611db4026c4 in ForeignNext (node=node@entry=0x5611dd0f9618) at nodeForeignscan.c:54
#8  0x00005611db3db4ff in ExecScanFetch (recheckMtd=0x5611db40256e <ForeignRecheck>, accessMtd=0x5611db402650 <ForeignNext>, node=0x5611dd0f9618)
    at execScan.c:95
#9  ExecScan (node=0x5611dd0f9618, accessMtd=accessMtd@entry=0x5611db402650 <ForeignNext>,
    recheckMtd=recheckMtd@entry=0x5611db40256e <ForeignRecheck>) at execScan.c:145
#10 0x00005611db40254d in ExecForeignScan (pstate=<optimized out>) at nodeForeignscan.c:121
#11 0x00005611db3d9aa2 in ExecProcNodeFirst (node=0x5611dd0f9618) at execProcnode.c:445
#12 0x00005611db3d2039 in ExecProcNode (node=0x5611dd0f9618) at ../../../src/include/executor/executor.h:247
#13 ExecutePlan (estate=estate@entry=0x5611dd0b2718, planstate=0x5611dd0f9618, use_parallel_mode=<optimized out>,
    operation=operation@entry=CMD_SELECT, sendTuples=sendTuples@entry=true, numberTuples=numberTuples@entry=0, direction=ForwardScanDirection,
    dest=0x5611dd0df520, execute_once=true) at execMain.c:1723
#14 0x00005611db3d2c94 in standard_ExecutorRun (queryDesc=0x5611dd0c7be8, direction=ForwardScanDirection, count=0, execute_once=<optimized out>)
    at execMain.c:364
#15 0x00005611db3d2d4f in ExecutorRun (queryDesc=queryDesc@entry=0x5611dd0c7be8, direction=direction@entry=ForwardScanDirection,
    count=count@entry=0, execute_once=<optimized out>) at execMain.c:307
#16 0x00005611db53f0ed in PortalRunSelect (portal=portal@entry=0x5611dd054278, forward=forward@entry=true, count=0,
    count@entry=9223372036854775807, dest=dest@entry=0x5611dd0df520) at pquery.c:932
#17 0x00005611db5407de in PortalRun (portal=portal@entry=0x5611dd054278, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true,
    run_once=run_once@entry=true, dest=dest@entry=0x5611dd0df520, altdest=altdest@entry=0x5611dd0df520, completionTag=0x7ffc524cad10 "")
    at pquery.c:773
#18 0x00005611db53caa9 in exec_simple_query (
    query_string=query_string@entry=0x5611dcfedac8 "select exists(select c1 from ft4), avg(c1) from ft4 where c1 = (select\nmax(c1) from ft4);")
    at postgres.c:1145
#19 0x00005611db53e9ce in PostgresMain (argc=<optimized out>, argv=argv@entry=0x5611dd018910, dbname=<optimized out>, username=<optimized out>)
    at postgres.c:4182
#20 0x00005611db4b8d8b in BackendRun (port=port@entry=0x5611dd0115a0) at postmaster.c:4358
#21 0x00005611db4bbd2f in BackendStartup (port=port@entry=0x5611dd0115a0) at postmaster.c:4030
#22 0x00005611db4bbf52 in ServerLoop () at postmaster.c:1707
#23 0x00005611db4bd459 in PostmasterMain (argc=3, argv=<optimized out>) at postmaster.c:1380
#24 0x00005611db4210c9 in main (argc=3, argv=0x5611dcfe81f0) at main.c:228

Similar from HEAD:

#0  CheckVarSlotCompatibility (slot=slot@entry=0x0, attnum=1, vartype=16) at execExprInterp.c:1850
#1  0x00005581fa6011b7 in CheckExprStillValid (state=state@entry=0x5581fba700c0, econtext=econtext@entry=0x5581fba6f4f0) at execExprInterp.c:1814
#2  0x00005581fa6011e3 in ExecInterpExprStillValid (state=0x5581fba700c0, econtext=0x5581fba6f4f0, isNull=0x7ffcad499ebf) at execExprInterp.c:1763
#3  0x00007f276130d67c in ExecEvalExpr (isNull=0x7ffcad499ebf, econtext=0x5581fba6f4f0, state=<optimized out>)
    at ../../src/include/executor/executor.h:288
#4  process_query_params (econtext=0x5581fba6f4f0, param_flinfo=0x5581fba70028, param_exprs=<optimized out>,
    param_values=param_values@entry=0x5581fba70aa8) at postgres_fdw.c:4307
#5  0x00007f276130d982 in create_cursor (node=<optimized out>) at postgres_fdw.c:3247
#6  0x00007f276130da3c in postgresIterateForeignScan (node=0x5581fba6f3d8) at postgres_fdw.c:1517
#7  0x00005581fa63adad in ForeignNext (node=node@entry=0x5581fba6f3d8) at nodeForeignscan.c:54
#8  0x00005581fa61104b in ExecScanFetch (recheckMtd=0x5581fa63adf1 <ForeignRecheck>, accessMtd=0x5581fa63ad2c <ForeignNext>, node=0x5581fba6f3d8)
    at execScan.c:93
#9  ExecScan (node=0x5581fba6f3d8, accessMtd=accessMtd@entry=0x5581fa63ad2c <ForeignNext>,
    recheckMtd=recheckMtd@entry=0x5581fa63adf1 <ForeignRecheck>) at execScan.c:143
#10 0x00005581fa63add0 in ExecForeignScan (pstate=<optimized out>) at nodeForeignscan.c:115
#11 0x00005581fa60f3e8 in ExecProcNodeFirst (node=0x5581fba6f3d8) at execProcnode.c:445
#12 0x00005581fa607fdd in ExecProcNode (node=0x5581fba6f3d8) at ../../../src/include/executor/executor.h:239
#13 ExecutePlan (estate=estate@entry=0x5581fba2abb8, planstate=0x5581fba6f3d8, use_parallel_mode=<optimized out>,
    operation=operation@entry=CMD_SELECT, sendTuples=sendTuples@entry=true, numberTuples=numberTuples@entry=0, direction=ForwardScanDirection,
    dest=0x5581fba5cac0, execute_once=true) at execMain.c:1648
#14 0x00005581fa608c2a in standard_ExecutorRun (queryDesc=0x5581fba207f8, direction=ForwardScanDirection, count=0, execute_once=<optimized out>)
    at execMain.c:365
#15 0x00005581fa608ce5 in ExecutorRun (queryDesc=queryDesc@entry=0x5581fba207f8, direction=direction@entry=ForwardScanDirection,
    count=count@entry=0, execute_once=<optimized out>) at execMain.c:309
#16 0x00005581fa782d65 in PortalRunSelect (portal=portal@entry=0x5581fb9bb168, forward=forward@entry=true, count=0,
    count@entry=9223372036854775807, dest=dest@entry=0x5581fba5cac0) at pquery.c:929
#17 0x00005581fa78442c in PortalRun (portal=portal@entry=0x5581fb9bb168, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true,
    run_once=run_once@entry=true, dest=dest@entry=0x5581fba5cac0, altdest=altdest@entry=0x5581fba5cac0, completionTag=0x7ffcad49a330 "")
    at pquery.c:770
#18 0x00005581fa780755 in exec_simple_query (
    query_string=query_string@entry=0x5581fb955ac8 "select exists(select c1 from ft4), avg(c1) from ft4 where c1 = (select\nmax(c1) from ft4);")
    at postgres.c:1215
#19 0x00005581fa78263d in PostgresMain (argc=<optimized out>, argv=argv@entry=0x5581fb981310, dbname=<optimized out>, username=<optimized out>)
    at postgres.c:4249
#20 0x00005581fa6f7979 in BackendRun (port=port@entry=0x5581fb978d20) at postmaster.c:4426
#21 0x00005581fa6faa98 in BackendStartup (port=port@entry=0x5581fb978d20) at postmaster.c:4117
#22 0x00005581fa6facbb in ServerLoop () at postmaster.c:1704
#23 0x00005581fa6fc1fc in PostmasterMain (argc=3, argv=<optimized out>) at postmaster.c:1377
#24 0x00005581fa65acf1 in main (argc=3, argv=0x5581fb9501f0) at main.c:228

regards, Sergei


Reply | Threaded
Open this post in threaded view
|

Re: BUG #15781: subselect on foreign table (postgres_fdw) can crash (segfault)

Dmitry Dolgov
In reply to this post by PG Doc comments form
> On Thu, Apr 25, 2019 at 8:24 PM Tom Lane <[hidden email]> wrote:
>
> The proximate cause of the crash is that we have {PARAM 1}
> (representing the output of the InitPlan) in the path's fdw_exprs, and
> also the identical expression in fdw_scan_tlist, and that means that when
> setrefs.c processes the ForeignScan node it thinks it should replace the
> {PARAM 1} in fdw_exprs with a Var representing a reference to the
> fdw_scan_tlist entry.

I've noticed, that it behaves like that since f9f63ed1f2e5 (originally I found
it pretty strange, but after this explanation it does make sense). As an
experiment, I've changed the position of condition of

    if (context->subplan_itlist->has_non_vars)

back - it also made problem to disappear, and what was interesting is that the
test case for update (exactly what this commit was fixing) is not crashing
either. I've checked on the commit right before f9f63ed1f2e5, without mentioned
reordering there is a crash, but I couldn't reproduce it on the master.


Reply | Threaded
Open this post in threaded view
|

Re: BUG #15781: subselect on foreign table (postgres_fdw) can crash (segfault)

Etsuro Fujita-2
In reply to this post by PG Doc comments form
On Sat, Apr 27, 2019 at 2:10 AM Tom Lane <[hidden email]> wrote:

> > (2019/04/26 3:24), Tom Lane wrote:
> >> If we do leave it like this, then the only way for postgres_fdw to
> >> avoid trouble is to not have any entries in fdw_exprs that exactly
> >> match entries in fdw_scan_tlist.  So that pretty much devolves back
> >> to what I said before: don't ship values to the far end that are
> >> just going to be fed back as-is.  But now it's a correctness
> >> requirement not just an optimization.

> Well, the releases are coming up fast, so I spent some time on this.
> If we don't want to change what the core code does with fdw_exprs,
> I think the only way to fix it is to hack postgres_fdw so that it
> won't generate plans involving the problematic case.

Seems reasonable.

> See attached.

I read the patch.  It looks good to me.  I didn't test it, though.

> We end up with slightly weird-looking plans if the troublesome Param
> is actually a GROUP BY expression, but if it's not, I think things
> are fine.  Maybe we could do something smarter about the GROUP BY case,
> but it seems weird enough to maybe not be worth additional trouble.

Agreed.

Thanks for working on this!

Best regards,
Etsuro Fujita