Feedback on table expansion hook (including patch)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Feedback on table expansion hook (including patch)

Erik Nordström-2
Hi,

I am looking for feedback on the possibility of adding a table expansion hook to PostgreSQL (see attached patch). The motivation for this is to allow extensions to optimize table expansion. In particular, TimescaleDB does its own table expansion in order to apply a number of optimizations, including partition pruning (note that TimescaleDB uses inheritance since PostgreSQL 9.6 rather than declarative partitioning ). There's currently no official hook for table expansion, but TimescaleDB has been using the get_relation_info hook for this purpose. Unfortunately, PostgreSQL 12 broke this for us since it moved expansion to a later stage where we can no longer control it without some pretty bad hacks. Given that PostgreSQL 12 changed the expansion state of a table for the get_relation_info hook, we are thinking about this as a regression and are wondering if this could be considered against the head of PG 12 or maybe even PG 13 (although we realize feature freeze has been reached)?

The attached patch is against PostgreSQL master (commit fb544735) and is about ~10 lines of code. It doesn't change any existing behavior; it only allows getting control of expand_inherited_rtentry, which would make a huge difference for TimescaleDB.

Best regards,

Erik
Engineering team lead
Timescale



table-expansion-hook.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Feedback on table expansion hook (including patch)

"Hans-Jürgen Schönig (PostgreSQL)"
that sounds really really useful.
i can see a ton of use cases for that.
we also toyed with the idea recently of having pluggable FSM strategies.
that one could be quite useful as well.

regards,

hans


On 07.05.2020, at 10:11, Erik Nordström <[hidden email]> wrote:

Hi,

I am looking for feedback on the possibility of adding a table expansion hook to PostgreSQL (see attached patch). The motivation for this is to allow extensions to optimize table expansion. In particular, TimescaleDB does its own table expansion in order to apply a number of optimizations, including partition pruning (note that TimescaleDB uses inheritance since PostgreSQL 9.6 rather than declarative partitioning ). There's currently no official hook for table expansion, but TimescaleDB has been using the get_relation_info hook for this purpose. Unfortunately, PostgreSQL 12 broke this for us since it moved expansion to a later stage where we can no longer control it without some pretty bad hacks. Given that PostgreSQL 12 changed the expansion state of a table for the get_relation_info hook, we are thinking about this as a regression and are wondering if this could be considered against the head of PG 12 or maybe even PG 13 (although we realize feature freeze has been reached)?

The attached patch is against PostgreSQL master (commit fb544735) and is about ~10 lines of code. It doesn't change any existing behavior; it only allows getting control of expand_inherited_rtentry, which would make a huge difference for TimescaleDB.

Best regards,

Erik
Engineering team lead
Timescale


<table-expansion-hook.patch>

--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com







Reply | Threaded
Open this post in threaded view
|

Re: Feedback on table expansion hook (including patch)

Euler Taveira-2
In reply to this post by Erik Nordström-2
On Thu, 7 May 2020 at 05:11, Erik Nordström <[hidden email]> wrote:

I am looking for feedback on the possibility of adding a table expansion hook to PostgreSQL (see attached patch). The motivation for this is to allow extensions to optimize table expansion. In particular, TimescaleDB does its own table expansion in order to apply a number of optimizations, including partition pruning (note that TimescaleDB uses inheritance since PostgreSQL 9.6 rather than declarative partitioning ). There's currently no official hook for table expansion, but TimescaleDB has been using the get_relation_info hook for this purpose. Unfortunately, PostgreSQL 12 broke this for us since it moved expansion to a later stage where we can no longer control it without some pretty bad hacks. Given that PostgreSQL 12 changed the expansion state of a table for the get_relation_info hook, we are thinking about this as a regression and are wondering if this could be considered against the head of PG 12 or maybe even PG 13 (although we realize feature freeze has been reached)?


I reviewed your patch and it looks good to me. You mentioned that it would be useful for partitioning using table inheritance but it could also be used for declarative partitioning (at least until someone decides to detach it from inheritance infrastructure).

Unfortunately, you showed up late here. Even though the hook is a straightforward feature, Postgres does not add new features to released versions or after the feature freeze.

The only point that I noticed was that you chose "control over" but similar code uses "control in".


--
Euler Taveira                 http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services