Quorum commit for multiple synchronous replication.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
114 messages Options
1234 ... 6
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Quorum commit for multiple synchronous replication.

Masahiko Sawada
Hi all,

In 9.6 development cycle, we had been discussed about configuration
syntax for a long time while considering expanding.
As a result, we had new dedicated language for multiple synchronous
replication, but it supports only priority method.
We know that quorum commit is very useful for many users and can
expand dedicated language easily for quorum commit.
So I'd like to propose quorum commit for multiple synchronous replication here.

The followings are changes attached patches made.
- Add new syntax 'Any N ( node1, node2, ... )' to
synchornous_standby_names for quorum commit.
- In quorum commit, the master can return commit to client after
received ACK from *at least* any N servers of listed standbys.
- sync_priority of all listed servers are same, 1.
- Add regression test for quorum commit.

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

Regards,

--
Masahiko Sawada


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

000_quorum_commit_v1.patch (18K) Download Attachment
001_add_regression_test_v1.patch (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Michael Paquier
On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]> wrote:
> I was thinking that the syntax for quorum method would use '[ ... ]'
> but it will be confused with '( ... )' priority method used.
> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
> might need to discuss about better syntax, discussion is very welcome.
> Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

-    foreach(cell, sync_standbys)
+    foreach (cell, sync_standbys)
     {
-        WalSnd       *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
+        WalSnd *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
This patch has some noise.
--
Michael


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Masahiko Sawada
On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<[hidden email]> wrote:

> On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]> wrote:
>> I was thinking that the syntax for quorum method would use '[ ... ]'
>> but it will be confused with '( ... )' priority method used.
>> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
>> might need to discuss about better syntax, discussion is very welcome.
>> Attached draft patch, please give me feedback.
>
> I am +1 for using either "{}" or "[]" to define a quorum set, and -1
> for the addition of a keyword in front of the integer defining for how
> many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

> -    foreach(cell, sync_standbys)
> +    foreach (cell, sync_standbys)
>      {
> -        WalSnd       *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
> +        WalSnd *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
> This patch has some noise.

Will fix.

--
Regards,

--
Masahiko Sawada


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Petr Jelinek
On 04/08/16 06:40, Masahiko Sawada wrote:

> On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
> <[hidden email]> wrote:
>> On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]> wrote:
>>> I was thinking that the syntax for quorum method would use '[ ... ]'
>>> but it will be confused with '( ... )' priority method used.
>>> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
>>> might need to discuss about better syntax, discussion is very welcome.
>>> Attached draft patch, please give me feedback.
>>
>> I am +1 for using either "{}" or "[]" to define a quorum set, and -1
>> for the addition of a keyword in front of the integer defining for how
>> many nodes server need to wait for.
>
> Thank you for reply.
> "{}" or "[]" are not bad but because these are not intuitive, I
> thought that it will be hard for uses to use different method for each
> purpose.
>

I think the "any" keyword is more explicit and understandable, also
closer to SQL. So I would be in favor of doing that.

--
   Petr Jelinek                  http://www.2ndQuadrant.com/
   PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Fujii Masao-2
On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <[hidden email]> wrote:

> On 04/08/16 06:40, Masahiko Sawada wrote:
>>
>> On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
>> <[hidden email]> wrote:
>>>
>>> On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]>
>>> wrote:
>>>>
>>>> I was thinking that the syntax for quorum method would use '[ ... ]'
>>>> but it will be confused with '( ... )' priority method used.
>>>> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
>>>> might need to discuss about better syntax, discussion is very welcome.
>>>> Attached draft patch, please give me feedback.
>>>
>>>
>>> I am +1 for using either "{}" or "[]" to define a quorum set, and -1
>>> for the addition of a keyword in front of the integer defining for how
>>> many nodes server need to wait for.
>>
>>
>> Thank you for reply.
>> "{}" or "[]" are not bad but because these are not intuitive, I
>> thought that it will be hard for uses to use different method for each
>> purpose.
>>
>
> I think the "any" keyword is more explicit and understandable, also closer
> to SQL. So I would be in favor of doing that.

+1

Also I like the following Simon's idea.

https://www.postgresql.org/message-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@...
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience
-----------------------

Regards,

--
Fujii Masao


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Simon Riggs
On 29 August 2016 at 14:52, Fujii Masao <[hidden email]> wrote:

> On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <[hidden email]> wrote:
>> On 04/08/16 06:40, Masahiko Sawada wrote:
>>>
>>> On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
>>> <[hidden email]> wrote:
>>>>
>>>> On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]>
>>>> wrote:
>>>>>
>>>>> I was thinking that the syntax for quorum method would use '[ ... ]'
>>>>> but it will be confused with '( ... )' priority method used.
>>>>> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
>>>>> might need to discuss about better syntax, discussion is very welcome.
>>>>> Attached draft patch, please give me feedback.
>>>>
>>>>
>>>> I am +1 for using either "{}" or "[]" to define a quorum set, and -1
>>>> for the addition of a keyword in front of the integer defining for how
>>>> many nodes server need to wait for.
>>>
>>>
>>> Thank you for reply.
>>> "{}" or "[]" are not bad but because these are not intuitive, I
>>> thought that it will be hard for uses to use different method for each
>>> purpose.
>>>
>>
>> I think the "any" keyword is more explicit and understandable, also closer
>> to SQL. So I would be in favor of doing that.
>
> +1
>
> Also I like the following Simon's idea.
>
> https://www.postgresql.org/message-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@...
> -----------------------
> * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
> * any k (n1, n2, n3) – would release waiters as soon as we have the
> responses from k out of N standbys. “any k” would be faster, so is
> desirable for performance and resilience
> -----------------------

+1

"synchronous_method" -> "synchronization_method"

I'm concerned about the performance of this code. Can we work out a
way of measuring it, so we can judge how successful we are at
releasing waiters quickly? Thanks

For 9.6 we implemented something that allows the DBA to define how
slow programs are. Previously, since 9.1 this was something specified
on the application side. I would like to put it back that way, so we
end up with a parameter on client e.g. commit_quorum = k. Forget the
exact parameters/user API for now, but I'd like to allow the code to
work with user defined settings. Thanks.

--
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Masahiko Sawada
On Tue, Sep 6, 2016 at 11:08 PM, Simon Riggs <[hidden email]> wrote:

> On 29 August 2016 at 14:52, Fujii Masao <[hidden email]> wrote:
>> On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <[hidden email]> wrote:
>>> On 04/08/16 06:40, Masahiko Sawada wrote:
>>>>
>>>> On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
>>>> <[hidden email]> wrote:
>>>>>
>>>>> On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> I was thinking that the syntax for quorum method would use '[ ... ]'
>>>>>> but it will be confused with '( ... )' priority method used.
>>>>>> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
>>>>>> might need to discuss about better syntax, discussion is very welcome.
>>>>>> Attached draft patch, please give me feedback.
>>>>>
>>>>>
>>>>> I am +1 for using either "{}" or "[]" to define a quorum set, and -1
>>>>> for the addition of a keyword in front of the integer defining for how
>>>>> many nodes server need to wait for.
>>>>
>>>>
>>>> Thank you for reply.
>>>> "{}" or "[]" are not bad but because these are not intuitive, I
>>>> thought that it will be hard for uses to use different method for each
>>>> purpose.
>>>>
>>>
>>> I think the "any" keyword is more explicit and understandable, also closer
>>> to SQL. So I would be in favor of doing that.
>>
>> +1
>>
>> Also I like the following Simon's idea.
>>
>> https://www.postgresql.org/message-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@...
>> -----------------------
>> * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
>> * any k (n1, n2, n3) – would release waiters as soon as we have the
>> responses from k out of N standbys. “any k” would be faster, so is
>> desirable for performance and resilience
>> -----------------------
>
> +1
>
> "synchronous_method" -> "synchronization_method"

Thanks, will fix.

> I'm concerned about the performance of this code. Can we work out a
> way of measuring it, so we can judge how successful we are at
> releasing waiters quickly? Thanks

I will measure the performance effect of this code.
I'm expecting that performances are,
'first 1 (n1, n2)'  > 'any 1(n1, n2)' > 'first 2(n1, n2)'
'first 1 (n1, n2)' will be highest throughput.

> For 9.6 we implemented something that allows the DBA to define how
> slow programs are. Previously, since 9.1 this was something specified
> on the application side. I would like to put it back that way, so we
> end up with a parameter on client e.g. commit_quorum = k. Forget the
> exact parameters/user API for now, but I'd like to allow the code to
> work with user defined settings. Thanks.

I see. The parameter on client should effect for priority method as well.
And similar to synchronous_commit, the client can specify the how much
standbys the master waits to commit for according to synchronization
method, even if s_s_names is defined.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Josh berkus
In reply to this post by Fujii Masao-2
On 08/29/2016 06:52 AM, Fujii Masao wrote:
> Also I like the following Simon's idea.
>
> https://www.postgresql.org/message-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@...
> -----------------------
> * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
> * any k (n1, n2, n3) – would release waiters as soon as we have the
> responses from k out of N standbys. “any k” would be faster, so is
> desirable for performance and resilience

What are we going to do for backwards compatibility, here?

So, here's the dilemma:

If we want to keep backwards compatibility with 9.6, then:

"k (n1, n2, n3)" == "first k (n1, n2, n3)"

However, "first k" is not what most users will want, most of the time;
users of version 13, years from now, will be getting constantly confused
by "first k" behavior when they wanted quorum.  So the sensible default
would be:

"k (n1, n2, n3)" == "any k (n1, n2, n3)"

... however, that will break backwards compatibility.  Thoughts?

My $0.02 is that we break backwards compat somehow and document the heck
out of it.

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Masahiko Sawada
In reply to this post by Masahiko Sawada
On Wed, Sep 7, 2016 at 12:47 AM, Masahiko Sawada <[hidden email]> wrote:

> On Tue, Sep 6, 2016 at 11:08 PM, Simon Riggs <[hidden email]> wrote:
>> On 29 August 2016 at 14:52, Fujii Masao <[hidden email]> wrote:
>>> On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <[hidden email]> wrote:
>>>> On 04/08/16 06:40, Masahiko Sawada wrote:
>>>>>
>>>>> On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
>>>>> <[hidden email]> wrote:
>>>>>>
>>>>>> On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> I was thinking that the syntax for quorum method would use '[ ... ]'
>>>>>>> but it will be confused with '( ... )' priority method used.
>>>>>>> 001 patch adds 'Any N ( ... )' style syntax but I know that we still
>>>>>>> might need to discuss about better syntax, discussion is very welcome.
>>>>>>> Attached draft patch, please give me feedback.
>>>>>>
>>>>>>
>>>>>> I am +1 for using either "{}" or "[]" to define a quorum set, and -1
>>>>>> for the addition of a keyword in front of the integer defining for how
>>>>>> many nodes server need to wait for.
>>>>>
>>>>>
>>>>> Thank you for reply.
>>>>> "{}" or "[]" are not bad but because these are not intuitive, I
>>>>> thought that it will be hard for uses to use different method for each
>>>>> purpose.
>>>>>
>>>>
>>>> I think the "any" keyword is more explicit and understandable, also closer
>>>> to SQL. So I would be in favor of doing that.
>>>
>>> +1
>>>
>>> Also I like the following Simon's idea.
>>>
>>> https://www.postgresql.org/message-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@...
>>> -----------------------
>>> * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
>>> * any k (n1, n2, n3) – would release waiters as soon as we have the
>>> responses from k out of N standbys. “any k” would be faster, so is
>>> desirable for performance and resilience
>>> -----------------------
>>
>> +1
>>
>> "synchronous_method" -> "synchronization_method"
>
> Thanks, will fix.
>
>> I'm concerned about the performance of this code. Can we work out a
>> way of measuring it, so we can judge how successful we are at
>> releasing waiters quickly? Thanks
>
> I will measure the performance effect of this code.
> I'm expecting that performances are,
> 'first 1 (n1, n2)'  > 'any 1(n1, n2)' > 'first 2(n1, n2)'
> 'first 1 (n1, n2)' will be highest throughput.
>

Sorry, that's wrong.
'any 1(n1, n2)' will be highest throughput or same as 'first 1(n1, n2)'.

>> For 9.6 we implemented something that allows the DBA to define how
>> slow programs are. Previously, since 9.1 this was something specified
>> on the application side. I would like to put it back that way, so we
>> end up with a parameter on client e.g. commit_quorum = k. Forget the
>> exact parameters/user API for now, but I'd like to allow the code to
>> work with user defined settings. Thanks.
>
> I see. The parameter on client should effect for priority method as well.
> And similar to synchronous_commit, the client can specify the how much
> standbys the master waits to commit for according to synchronization
> method, even if s_s_names is defined.
>
> Regards,
>
> --
> Masahiko Sawada
> NIPPON TELEGRAPH AND TELEPHONE CORPORATION
> NTT Open Source Software Center

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Masahiko Sawada
In reply to this post by Josh berkus
On Wed, Sep 7, 2016 at 4:03 AM, Josh Berkus <[hidden email]> wrote:

> On 08/29/2016 06:52 AM, Fujii Masao wrote:
>> Also I like the following Simon's idea.
>>
>> https://www.postgresql.org/message-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@...
>> -----------------------
>> * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
>> * any k (n1, n2, n3) – would release waiters as soon as we have the
>> responses from k out of N standbys. “any k” would be faster, so is
>> desirable for performance and resilience
>
> What are we going to do for backwards compatibility, here?
>
> So, here's the dilemma:
>
> If we want to keep backwards compatibility with 9.6, then:
>
> "k (n1, n2, n3)" == "first k (n1, n2, n3)"
>
> However, "first k" is not what most users will want, most of the time;
> users of version 13, years from now, will be getting constantly confused
> by "first k" behavior when they wanted quorum.  So the sensible default
> would be:
>
> "k (n1, n2, n3)" == "any k (n1, n2, n3)"
>

+1.

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Michael Paquier
On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada <[hidden email]> wrote:
> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
> compatibility but most users would think "k(n1, n2, n3)" as quorum
> after introduced quorum.
> I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
> n3)" style before 9.6 releasing if we got consensus.

Considering breaking backward-compatibility in the next release does
not sound like a good idea to me for a new feature that is going to be
GA soon.
--
Michael


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Vik Fearing-3
On 09/09/2016 03:28 AM, Michael Paquier wrote:

> On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada <[hidden email]> wrote:
>> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
>> compatibility but most users would think "k(n1, n2, n3)" as quorum
>> after introduced quorum.
>> I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
>> n3)" style before 9.6 releasing if we got consensus.
>
> Considering breaking backward-compatibility in the next release does
> not sound like a good idea to me for a new feature that is going to be
> GA soon.

Indeed.  I'll vote for pulling a fast one on 9.6 for this.
--
Vik Fearing                                          +33 6 46 75 15 36
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Petr Jelinek


On 09/09/16 08:23, Vik Fearing wrote:

> On 09/09/2016 03:28 AM, Michael Paquier wrote:
>> On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada <[hidden email]> wrote:
>>> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
>>> compatibility but most users would think "k(n1, n2, n3)" as quorum
>>> after introduced quorum.
>>> I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
>>> n3)" style before 9.6 releasing if we got consensus.
>>
>> Considering breaking backward-compatibility in the next release does
>> not sound like a good idea to me for a new feature that is going to be
>> GA soon.
>
> Indeed.  I'll vote for pulling a fast one on 9.6 for this.
>

+1

--
   Petr Jelinek                  http://www.2ndQuadrant.com/
   PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Simon Riggs
In reply to this post by Masahiko Sawada
On 8 September 2016 at 10:26, Masahiko Sawada <[hidden email]> wrote:

> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
> compatibility but most users would think "k(n1, n2, n3)" as quorum
> after introduced quorum.
> I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
> n3)" style before 9.6 releasing if we got consensus.

Let's see the proposed patch, so we can evaluate the proposal.

--
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Masahiko Sawada
On Fri, Sep 9, 2016 at 6:23 PM, Simon Riggs <[hidden email]> wrote:

> On 8 September 2016 at 10:26, Masahiko Sawada <[hidden email]> wrote:
>
>> "k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
>> compatibility but most users would think "k(n1, n2, n3)" as quorum
>> after introduced quorum.
>> I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
>> n3)" style before 9.6 releasing if we got consensus.
>
> Let's see the proposed patch, so we can evaluate the proposal.
>
Attached 2 patches.
000 patch changes syntax of s_s_names from 'k(n1, n2, n3)' to 'First k
(n1, n2,n3)' for PG9.6.
001 patch adds the quorum commit using syntax 'Any k (n1, n2,n3)' for PG10.

Since we already released 9.6RC1, I understand that it's quite hard to
change syntax of 9.6.
But considering that we support the quorum commit, this could be one
of the solutions in order to avoid breaking backward compatibility and
to provide useful user interface.
So I attached these patches.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

000_change_syntax_96.patch (5K) Download Attachment
001_quorum_commit_v2.patch (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Michael Paquier
On Sat, Sep 17, 2016 at 2:04 AM, Masahiko Sawada <[hidden email]> wrote:
> Since we already released 9.6RC1, I understand that it's quite hard to
> change syntax of 9.6.
> But considering that we support the quorum commit, this could be one
> of the solutions in order to avoid breaking backward compatibility and
> to provide useful user interface.
> So I attached these patches.

 standby_config:
-        standby_list                { $$ = create_syncrep_config("1", $1); }
-        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($1, $4); }
+        standby_list                        { $$ =
create_syncrep_config("1", $1, SYNC_REP_PRIORITY); }
+        | ANY NUM '(' standby_list ')'        { $$ =
create_syncrep_config($2, $4, SYNC_REP_QUORUM); }
+        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($2, $4, SYNC_REP_PRIORITY); }

Reading again the thread, it seems that my previous post [1] was a bit
misunderstood. My position is to not introduce any new behavior
changes in 9.6, so we could just make the FIRST NUM grammar equivalent
to NUM.

[1]: https://www.postgresql.org/message-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@...
--
Michael


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Vik Fearing-3
On 09/21/2016 08:30 AM, Michael Paquier wrote:

> On Sat, Sep 17, 2016 at 2:04 AM, Masahiko Sawada <[hidden email]> wrote:
>> Since we already released 9.6RC1, I understand that it's quite hard to
>> change syntax of 9.6.
>> But considering that we support the quorum commit, this could be one
>> of the solutions in order to avoid breaking backward compatibility and
>> to provide useful user interface.
>> So I attached these patches.
>
>  standby_config:
> -        standby_list                { $$ = create_syncrep_config("1", $1); }
> -        | FIRST NUM '(' standby_list ')'    { $$ =
> create_syncrep_config($1, $4); }
> +        standby_list                        { $$ =
> create_syncrep_config("1", $1, SYNC_REP_PRIORITY); }
> +        | ANY NUM '(' standby_list ')'        { $$ =
> create_syncrep_config($2, $4, SYNC_REP_QUORUM); }
> +        | FIRST NUM '(' standby_list ')'    { $$ =
> create_syncrep_config($2, $4, SYNC_REP_PRIORITY); }
>
> Reading again the thread, it seems that my previous post [1] was a bit
> misunderstood. My position is to not introduce any new behavior
> changes in 9.6, so we could just make the FIRST NUM grammar equivalent
> to NUM.
>
> [1]: https://www.postgresql.org/message-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@...

I misunderstood your intent, then.  But I still stand by what I did
understand, namely that 'k (...)'  should mean 'any k (...)'.  It's much
more natural than having it mean 'first k (...)' and I also think it
will be more frequent in practice.
--
Vik Fearing                                          +33 6 46 75 15 36
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Petr Jelinek
On 21/09/16 09:18, Vik Fearing wrote:

> On 09/21/2016 08:30 AM, Michael Paquier wrote:
>> On Sat, Sep 17, 2016 at 2:04 AM, Masahiko Sawada <[hidden email]> wrote:
>>> Since we already released 9.6RC1, I understand that it's quite hard to
>>> change syntax of 9.6.
>>> But considering that we support the quorum commit, this could be one
>>> of the solutions in order to avoid breaking backward compatibility and
>>> to provide useful user interface.
>>> So I attached these patches.
>>
>>  standby_config:
>> -        standby_list                { $$ = create_syncrep_config("1", $1); }
>> -        | FIRST NUM '(' standby_list ')'    { $$ =
>> create_syncrep_config($1, $4); }
>> +        standby_list                        { $$ =
>> create_syncrep_config("1", $1, SYNC_REP_PRIORITY); }
>> +        | ANY NUM '(' standby_list ')'        { $$ =
>> create_syncrep_config($2, $4, SYNC_REP_QUORUM); }
>> +        | FIRST NUM '(' standby_list ')'    { $$ =
>> create_syncrep_config($2, $4, SYNC_REP_PRIORITY); }
>>
>> Reading again the thread, it seems that my previous post [1] was a bit
>> misunderstood. My position is to not introduce any new behavior
>> changes in 9.6, so we could just make the FIRST NUM grammar equivalent
>> to NUM.
>>
>> [1]: https://www.postgresql.org/message-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@...
>
> I misunderstood your intent, then.  But I still stand by what I did
> understand, namely that 'k (...)'  should mean 'any k (...)'.  It's much
> more natural than having it mean 'first k (...)' and I also think it
> will be more frequent in practice.
>

I think so as well.

--
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Robert Haas
On Wed, Sep 21, 2016 at 5:54 AM, Petr Jelinek <[hidden email]> wrote:

>>> Reading again the thread, it seems that my previous post [1] was a bit
>>> misunderstood. My position is to not introduce any new behavior
>>> changes in 9.6, so we could just make the FIRST NUM grammar equivalent
>>> to NUM.
>>>
>>> [1]: https://www.postgresql.org/message-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@...
>>
>> I misunderstood your intent, then.  But I still stand by what I did
>> understand, namely that 'k (...)'  should mean 'any k (...)'.  It's much
>> more natural than having it mean 'first k (...)' and I also think it
>> will be more frequent in practice.
>>
>
> I think so as well.

Well, I agree, but I think making behavior changes after rc1 is a
non-starter.  It's better to live with the incompatibility than to
change the behavior so close to release.  At least, that's my
position.  Getting the release out on time with a minimal bug count is
more important to me than a minor incompatibility in the meaning of
one GUC.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Quorum commit for multiple synchronous replication.

Masahiko Sawada
On Wed, Sep 21, 2016 at 11:22 PM, Robert Haas <[hidden email]> wrote:

> On Wed, Sep 21, 2016 at 5:54 AM, Petr Jelinek <[hidden email]> wrote:
>>>> Reading again the thread, it seems that my previous post [1] was a bit
>>>> misunderstood. My position is to not introduce any new behavior
>>>> changes in 9.6, so we could just make the FIRST NUM grammar equivalent
>>>> to NUM.
>>>>
>>>> [1]: https://www.postgresql.org/message-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@...
>>>
>>> I misunderstood your intent, then.  But I still stand by what I did
>>> understand, namely that 'k (...)'  should mean 'any k (...)'.  It's much
>>> more natural than having it mean 'first k (...)' and I also think it
>>> will be more frequent in practice.
>>>
>>
>> I think so as well.
>
> Well, I agree, but I think making behavior changes after rc1 is a
> non-starter.  It's better to live with the incompatibility than to
> change the behavior so close to release.  At least, that's my
> position.  Getting the release out on time with a minimal bug count is
> more important to me than a minor incompatibility in the meaning of
> one GUC.
>
As the release team announced, it's better to postpone changing the
syntax of existing s_s_name.
I still vote for changing behaviour of existing syntax 'k (n1, n2)' to
quorum commit.
That is,
1.  'First k (n1, n2, n3)' means that the master server waits for ACKs
from k standby servers whose name appear earlier in the list.
2.  'Any k (n1, n2, n3)' means that the master server waits for ACKs
from any k listed standby servers.
3.  'n1, n2, n3' is the same as #1 with k=1.
4.  '(n1, n2, n3)' is the same as #2 with k=1.

Attached updated patch.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

quorum_commit_v3.patch (33K) Download Attachment
1234 ... 6
Previous Thread Next Thread
Loading...