proposal: type info support functions for functions that use "any" type

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

proposal: type info support functions for functions that use "any" type

Pavel Stehule
Hi,

Tom introduced supported functions for calculation function's selectivity. Still I have similar idea to use supported function for calculation function's parameter's types and function return type.

Motivation:

Reduce a necessity of overloading of functions. My motivation is related primary to Orafce, but this feature should be helpful for anybody with similar goals. The function's overloading is great functionality but it is hard for maintenance.

My idea to enhance a CREATE FUNCTION command to be able do

CREATE FUCNTION foo("any")
RETURNS "any" AS ...
TYPEINFO foo_typeinfo

CREATE FUNCTION decode(VARIADIC "any")
RETURNS "any" AS ...
TYPEINFO decode_typeinfo.

The typeinfo functions returns a pointer tu structure with param types and result type. Only function with "any" parameters or "any" result can use TYPEINFO supported function. This functionality should not be allowed for common functions.

This functionality is limited just for C coders. But I expect so typical application coder doesn't need it. It doesn't replace my proposal of introduction other polymorphic type - now named "commontype" (can be named differently). The commontype is good enough solution for application coders, developers.

Comments, notes?

Regards

Pavel


Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Pavel Stehule
Hi

so 9. 3. 2019 v 7:22 odesílatel Pavel Stehule <[hidden email]> napsal:
Hi,

Tom introduced supported functions for calculation function's selectivity. Still I have similar idea to use supported function for calculation function's parameter's types and function return type.

Motivation:

Reduce a necessity of overloading of functions. My motivation is related primary to Orafce, but this feature should be helpful for anybody with similar goals. The function's overloading is great functionality but it is hard for maintenance.

My idea to enhance a CREATE FUNCTION command to be able do

CREATE FUCNTION foo("any")
RETURNS "any" AS ...
TYPEINFO foo_typeinfo

CREATE FUNCTION decode(VARIADIC "any")
RETURNS "any" AS ...
TYPEINFO decode_typeinfo.

The typeinfo functions returns a pointer tu structure with param types and result type. Only function with "any" parameters or "any" result can use TYPEINFO supported function. This functionality should not be allowed for common functions.

This functionality is limited just for C coders. But I expect so typical application coder doesn't need it. It doesn't replace my proposal of introduction other polymorphic type - now named "commontype" (can be named differently). The commontype is good enough solution for application coders, developers.

Comments, notes?

here is a patch

I have not a plan to push decode function to upstream. Patch contains it just as demo.

Regards

Pavel
 

Regards

Pavel



parse_func-support-func.patch (22K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Tom Lane-2
Pavel Stehule <[hidden email]> writes:

> so 9. 3. 2019 v 7:22 odesílatel Pavel Stehule <[hidden email]>
> napsal:
>> Tom introduced supported functions for calculation function's selectivity.
>> Still I have similar idea to use supported function for calculation
>> function's parameter's types and function return type.
>> Motivation:
>> Reduce a necessity of overloading of functions. My motivation is related
>> primary to Orafce, but this feature should be helpful for anybody with
>> similar goals. The function's overloading is great functionality but it is
>> hard for maintenance.

> here is a patch

TBH, I don't like this proposal one bit.  As far as I can see, the idea
is to let a function's support function redefine the function's declared
argument and result types on-the-fly according to no predetermined rules,
and that seems to me like it's a recipe for disaster.  How will anyone
understand which function(s) are candidates to match a query, or why one
particular candidate got selected over others?  It's already hard enough
to understand the behavior of polymorphic functions in complex cases,
and those are much more constrained than this would be.

Moreover, I don't think you've even provided a compelling example
case.  What's this doing that you couldn't do with existing polymorphic
types or the anycompatibletype proposal?

I also strongly suspect that this would break pieces of the system
that expect that the stored pg_proc.prorettype has something to do
with reality.  At minimum, you'd need to fix a number of places you
haven't touched here that have their own knowledge of function type
resolution, such as enforce_generic_type_consistency,
resolve_polymorphic_argtypes, resolve_aggregate_transtype.  Probably
anyplace that treats polymorphics as being any sort of special case
would have to be taught to re-call the support function to find out
what it should think the relevant types are.

(I don't even want to think about what happens if the support function's
behavior changes between original parsing and these re-checking spots.)

Another thing that's very much less than compelling about your example
is that your support function seems to be happy to throw errors
if the argument types don't match what it's expecting.  That seems
quite unacceptable, since it would prevent the parser from moving on
to consider other possibly-matching functions.  Maybe that's just
because it's a quick hack not a polished example, but it doesn't
seem like a good precedent.

In short, I think the added complexity and bug potential outweigh
any possible gain from this.

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Tom Lane-2
I wrote:
> TBH, I don't like this proposal one bit.  As far as I can see, the idea
> is to let a function's support function redefine the function's declared
> argument and result types on-the-fly according to no predetermined rules,
> and that seems to me like it's a recipe for disaster.  How will anyone
> understand which function(s) are candidates to match a query, or why one
> particular candidate got selected over others?  It's already hard enough
> to understand the behavior of polymorphic functions in complex cases,
> and those are much more constrained than this would be.

After thinking about this a bit more, it seems like you could avoid
a lot of problems if you restricted what the support function call
does to be potentially replacing the result type of a function
declared to return ANY with some more-specific type (computed from
examination of the actual arguments).  That would make it act much
more like a traditional polymorphic function.  It'd remove the issues
about interactions among multiple potentially-matching functions,
since we'd only call a single support function for an already-identified
target function.

You'd still need to touch everyplace that knows about polymorphic
type resolution, since this would essentially be another form of
polymorphic function.  And I'm still very dubious that it's worth
the trouble.  But it would be a lot more controllable than the
proposal as it stands.

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Pavel Stehule
In reply to this post by Tom Lane-2


pá 26. 7. 2019 v 22:03 odesílatel Tom Lane <[hidden email]> napsal:
Pavel Stehule <[hidden email]> writes:
> so 9. 3. 2019 v 7:22 odesílatel Pavel Stehule <[hidden email]>
> napsal:
>> Tom introduced supported functions for calculation function's selectivity.
>> Still I have similar idea to use supported function for calculation
>> function's parameter's types and function return type.
>> Motivation:
>> Reduce a necessity of overloading of functions. My motivation is related
>> primary to Orafce, but this feature should be helpful for anybody with
>> similar goals. The function's overloading is great functionality but it is
>> hard for maintenance.

> here is a patch

TBH, I don't like this proposal one bit.  As far as I can see, the idea
is to let a function's support function redefine the function's declared
argument and result types on-the-fly according to no predetermined rules,
and that seems to me like it's a recipe for disaster.  How will anyone
understand which function(s) are candidates to match a query, or why one
particular candidate got selected over others?  It's already hard enough
to understand the behavior of polymorphic functions in complex cases,
and those are much more constrained than this would be.

I quietly expect so this feature will be used without combination with overloading. But the combination of support function and overloading can be explicitly disabled - (in runtime for simple implementation).


Moreover, I don't think you've even provided a compelling example
case.  What's this doing that you couldn't do with existing polymorphic
types or the anycompatibletype proposal?

There are two cases of usage

a) combination of polymorphic types - fx(t1, t1, t2, t1, t2, t1, t2, ...)
b) forcing types fx(t1, t2) t1 force explicit cast for t2 to t1
c) optimization of repeated call of functions like fx("any", "any", "any", ...)

It is pretty hard to create simple non-procedural language to describe syntaxes like @a. But with procedural code it is easy.
 
@c is special case, that we can do already. But we cannot to push casting outside function, and inside function, there is a overhead with casting. With implementing type case inside function, then we increase startup time and it is overhead for function started by plpgsql runtime.


I also strongly suspect that this would break pieces of the system
that expect that the stored pg_proc.prorettype has something to do
with reality.  At minimum, you'd need to fix a number of places you
haven't touched here that have their own knowledge of function type
resolution, such as enforce_generic_type_consistency,
resolve_polymorphic_argtypes, resolve_aggregate_transtype.  Probably
anyplace that treats polymorphics as being any sort of special case
would have to be taught to re-call the support function to find out
what it should think the relevant types are.

(I don't even want to think about what happens if the support function's
behavior changes between original parsing and these re-checking spots.)

The helper function should be immutable - what I know, is not possible to change data types dynamically, so repeated call should not be effective, but should to produce same result, so it should not be a problem. 

Another thing that's very much less than compelling about your example
is that your support function seems to be happy to throw errors
if the argument types don't match what it's expecting.  That seems
quite unacceptable, since it would prevent the parser from moving on
to consider other possibly-matching functions.  Maybe that's just
because it's a quick hack not a polished example, but it doesn't
seem like a good precedent.

In this case it is decision, because I don't expect overloading.

I understand to your objections about mixing parser helper functions and overloading. Currently it is pretty hard to understand what will be expected behave when somebody overload function with polymorphic function.

With parser helper function the overloading is not necessary and can be disabled.


In short, I think the added complexity and bug potential outweigh
any possible gain from this.

                        regards, tom lane
Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Pavel Stehule
In reply to this post by Tom Lane-2


pá 26. 7. 2019 v 22:53 odesílatel Tom Lane <[hidden email]> napsal:
I wrote:
> TBH, I don't like this proposal one bit.  As far as I can see, the idea
> is to let a function's support function redefine the function's declared
> argument and result types on-the-fly according to no predetermined rules,
> and that seems to me like it's a recipe for disaster.  How will anyone
> understand which function(s) are candidates to match a query, or why one
> particular candidate got selected over others?  It's already hard enough
> to understand the behavior of polymorphic functions in complex cases,
> and those are much more constrained than this would be.

After thinking about this a bit more, it seems like you could avoid
a lot of problems if you restricted what the support function call
does to be potentially replacing the result type of a function
declared to return ANY with some more-specific type (computed from
examination of the actual arguments).  That would make it act much
more like a traditional polymorphic function.  It'd remove the issues
about interactions among multiple potentially-matching functions,
since we'd only call a single support function for an already-identified
target function.

I am not sure if I understand well - so I repeat it with my words.

So calculation of result type (replace ANY by some specific) can be ok?

I am able to do it if there will be a agreement.

I wrote a possibility to specify argument types as optimization as protection against repeated type identification and casting (that can be done in planning time, and should not be repeated).

This feature should be used only for functions with types fx("any", "any", ..) returns "any". So it is very probable so in execution type you should to do some work with parameter type identification.

But if we find a agreement just on work with return type, then it is good enough solution. The practical overhead of type cache inside function should not be dramatic.



You'd still need to touch everyplace that knows about polymorphic
type resolution, since this would essentially be another form of
polymorphic function.  And I'm still very dubious that it's worth
the trouble.  But it would be a lot more controllable than the
proposal as it stands.

ok


                        regards, tom lane
Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Thomas Munro-5
On Sat, Jul 27, 2019 at 5:45 PM Pavel Stehule <[hidden email]> wrote:

> pá 26. 7. 2019 v 22:53 odesílatel Tom Lane <[hidden email]> napsal:
>> I wrote:
>> > TBH, I don't like this proposal one bit.  As far as I can see, the idea
>> > is to let a function's support function redefine the function's declared
>> > argument and result types on-the-fly according to no predetermined rules,
>> > and that seems to me like it's a recipe for disaster.  How will anyone
>> > understand which function(s) are candidates to match a query, or why one
>> > particular candidate got selected over others?  It's already hard enough
>> > to understand the behavior of polymorphic functions in complex cases,
>> > and those are much more constrained than this would be.
>>
>> After thinking about this a bit more, it seems like you could avoid
>> a lot of problems if you restricted what the support function call
>> does to be potentially replacing the result type of a function
>> declared to return ANY with some more-specific type (computed from
>> examination of the actual arguments).  That would make it act much
>> more like a traditional polymorphic function.  It'd remove the issues
>> about interactions among multiple potentially-matching functions,
>> since we'd only call a single support function for an already-identified
>> target function.
>
>
> I am not sure if I understand well - so I repeat it with my words.
>
> So calculation of result type (replace ANY by some specific) can be ok?
>
> I am able to do it if there will be a agreement.
...

Hi Pavel,

I see that this is an active project with an ongoing discussion, but
we have run out of July so I have moved this to the September CF and
set it to "Waiting on Author".

--
Thomas Munro
https://enterprisedb.com


Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Pavel Stehule


čt 1. 8. 2019 v 11:01 odesílatel Thomas Munro <[hidden email]> napsal:
On Sat, Jul 27, 2019 at 5:45 PM Pavel Stehule <[hidden email]> wrote:
> pá 26. 7. 2019 v 22:53 odesílatel Tom Lane <[hidden email]> napsal:
>> I wrote:
>> > TBH, I don't like this proposal one bit.  As far as I can see, the idea
>> > is to let a function's support function redefine the function's declared
>> > argument and result types on-the-fly according to no predetermined rules,
>> > and that seems to me like it's a recipe for disaster.  How will anyone
>> > understand which function(s) are candidates to match a query, or why one
>> > particular candidate got selected over others?  It's already hard enough
>> > to understand the behavior of polymorphic functions in complex cases,
>> > and those are much more constrained than this would be.
>>
>> After thinking about this a bit more, it seems like you could avoid
>> a lot of problems if you restricted what the support function call
>> does to be potentially replacing the result type of a function
>> declared to return ANY with some more-specific type (computed from
>> examination of the actual arguments).  That would make it act much
>> more like a traditional polymorphic function.  It'd remove the issues
>> about interactions among multiple potentially-matching functions,
>> since we'd only call a single support function for an already-identified
>> target function.
>
>
> I am not sure if I understand well - so I repeat it with my words.
>
> So calculation of result type (replace ANY by some specific) can be ok?
>
> I am able to do it if there will be a agreement.
...

Hi Pavel,

I see that this is an active project with an ongoing discussion, but
we have run out of July so I have moved this to the September CF and
set it to "Waiting on Author".

sure

Pavel


--
Thomas Munro
https://enterprisedb.com
Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Pavel Stehule
In reply to this post by Tom Lane-2
Hi

pá 26. 7. 2019 v 22:53 odesílatel Tom Lane <[hidden email]> napsal:
I wrote:
> TBH, I don't like this proposal one bit.  As far as I can see, the idea
> is to let a function's support function redefine the function's declared
> argument and result types on-the-fly according to no predetermined rules,
> and that seems to me like it's a recipe for disaster.  How will anyone
> understand which function(s) are candidates to match a query, or why one
> particular candidate got selected over others?  It's already hard enough
> to understand the behavior of polymorphic functions in complex cases,
> and those are much more constrained than this would be.

After thinking about this a bit more, it seems like you could avoid
a lot of problems if you restricted what the support function call
does to be potentially replacing the result type of a function
declared to return ANY with some more-specific type (computed from
examination of the actual arguments).  That would make it act much
more like a traditional polymorphic function.  It'd remove the issues
about interactions among multiple potentially-matching functions,
since we'd only call a single support function for an already-identified
target function.

You'd still need to touch everyplace that knows about polymorphic
type resolution, since this would essentially be another form of
polymorphic function.  And I'm still very dubious that it's worth
the trouble.  But it would be a lot more controllable than the
proposal as it stands.

I am sending reduced version of previous patch. Now, support function is used just for replacement of returned type "any" by some other.

The are two patches - shorter with only support function, larger with demo "decode" function. I don't expect so the "decode" extension should be pushed to master. It is just demo of usage.

Regards

Pavel



                        regards, tom lane

parser-support-function.patch (9K) Download Attachment
parser-support-function-with-demo.patch (28K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: proposal: type info support functions for functions that use "any" type

Pavel Stehule
Hi

rebase

Pavel

parser-support-function-with-demo.patch (28K) Download Attachment