max_slot_wal_keep_size and wal_keep_segments

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

max_slot_wal_keep_size and wal_keep_segments

Fujii Masao-4
Hi,

When I talked about max_slot_wal_keep_size as new feature in v13
at the conference, I received the question like "Why are the units of
setting values in max_slot_wal_keep_size and wal_keep_segments different?"
from audience. That difference looks confusing for users and
IMO it's better to use the same unit for them. Thought?

There seem to be several options to do this.

(1) Rename max_slot_wal_keep_size to max_slot_wal_keep_segments,
     and make users specify the number of WAL segments in it instead of
     WAL size.

(2) Rename wal_keep_segments to wal_keep_size, and make users specify
      the WAL size in it instead of the number of WAL segments.

(3) Don't rename the parameters, and allow users to specify not only
     the number of WAL segments but also the WAL size in wal_keep_segments.

Since we have been moving away from measuring in segments, e.g.,
max_wal_size, I don't think (1) is good idea.

For backward compatibility, (3) is better. But which needs more
(maybe a bit complicated) code in guc.c. Also the parameter names
are not consistent yet (i.e., _segments and _size).

So for now I like (2).

Thought?

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Kyotaro Horiguchi-4
At Tue, 30 Jun 2020 23:51:40 +0900, Fujii Masao <[hidden email]> wrote in
> Hi,
>
> When I talked about max_slot_wal_keep_size as new feature in v13
> at the conference, I received the question like "Why are the units of
> setting values in max_slot_wal_keep_size and wal_keep_segments
> different?"
> from audience. That difference looks confusing for users and
> IMO it's better to use the same unit for them. Thought?

We are moving the units for amount of WAL from segments to MB.  The
variable is affected by the movement.  I'm not sure wal_keep_segments
is going to die soon but we may change it to wal_keep_size(_mb) sooner
or later if it going to stay alive.

> There seem to be several options to do this.
>
> (1) Rename max_slot_wal_keep_size to max_slot_wal_keep_segments,
>     and make users specify the number of WAL segments in it instead of
>     WAL size.

I don't think this is not the way.

> (2) Rename wal_keep_segments to wal_keep_size, and make users specify
>      the WAL size in it instead of the number of WAL segments.

Yes. I agree to this (as I wrote above before reading this).

> (3) Don't rename the parameters, and allow users to specify not only
>     the number of WAL segments but also the WAL size in wal_keep_segments.

Possible in a short term, but not for a long term.

> Since we have been moving away from measuring in segments, e.g.,
> max_wal_size, I don't think (1) is good idea.
>
> For backward compatibility, (3) is better. But which needs more
> (maybe a bit complicated) code in guc.c. Also the parameter names
> are not consistent yet (i.e., _segments and _size).
>
> So for now I like (2).
>
> Thought?

I agree to you.  If someone found that wal_keep_segment is no longer
usable, the alternative would easily be found by searching config file
for "wal_keep". Or we could have a default config line like this:

wal_keep_size = 0     # in megabytes: 0 disables (formerly wal_keep_segments)

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Alvaro Herrera-9
In reply to this post by Fujii Masao-4
On 2020-Jun-30, Fujii Masao wrote:

> Hi,
>
> When I talked about max_slot_wal_keep_size as new feature in v13
> at the conference, I received the question like "Why are the units of
> setting values in max_slot_wal_keep_size and wal_keep_segments different?"
> from audience. That difference looks confusing for users and
> IMO it's better to use the same unit for them. Thought?

Do we still need wal_keep_segments for anything?  Maybe we should
consider removing that functionality instead ... and even if we don't
remove it in 13, then what is the point of renaming it only to remove it
shortly after?

--
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Fujii Masao-4


On 2020/07/01 12:26, Alvaro Herrera wrote:

> On 2020-Jun-30, Fujii Masao wrote:
>
>> Hi,
>>
>> When I talked about max_slot_wal_keep_size as new feature in v13
>> at the conference, I received the question like "Why are the units of
>> setting values in max_slot_wal_keep_size and wal_keep_segments different?"
>> from audience. That difference looks confusing for users and
>> IMO it's better to use the same unit for them. Thought?
>
> Do we still need wal_keep_segments for anything?

Yeah, personally I like wal_keep_segments because its setting is very
simple and no extra operations on replication slots are necessary.

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Alvaro Herrera-9
On 2020-Jul-01, Fujii Masao wrote:

> On 2020/07/01 12:26, Alvaro Herrera wrote:
> > On 2020-Jun-30, Fujii Masao wrote:
> >
> > > When I talked about max_slot_wal_keep_size as new feature in v13
> > > at the conference, I received the question like "Why are the units of
> > > setting values in max_slot_wal_keep_size and wal_keep_segments different?"
> > > from audience. That difference looks confusing for users and
> > > IMO it's better to use the same unit for them. Thought?
> >
> > Do we still need wal_keep_segments for anything?
>
> Yeah, personally I like wal_keep_segments because its setting is very
> simple and no extra operations on replication slots are necessary.

Okay.  In that case I +1 the idea of renaming to wal_keep_size.

--
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

David Steele
On 7/1/20 10:54 AM, Alvaro Herrera wrote:

> On 2020-Jul-01, Fujii Masao wrote:
>
>> On 2020/07/01 12:26, Alvaro Herrera wrote:
>>> On 2020-Jun-30, Fujii Masao wrote:
>>>
>>>> When I talked about max_slot_wal_keep_size as new feature in v13
>>>> at the conference, I received the question like "Why are the units of
>>>> setting values in max_slot_wal_keep_size and wal_keep_segments different?"
>>>> from audience. That difference looks confusing for users and
>>>> IMO it's better to use the same unit for them. Thought?
>>>
>>> Do we still need wal_keep_segments for anything?
>>
>> Yeah, personally I like wal_keep_segments because its setting is very
>> simple and no extra operations on replication slots are necessary.
>
> Okay.  In that case I +1 the idea of renaming to wal_keep_size.

+1 for renaming to wal_keep_size.

--
-David
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Bruce Momjian
On Wed, Jul  1, 2020 at 01:18:06PM -0400, David Steele wrote:

> On 7/1/20 10:54 AM, Alvaro Herrera wrote:
> > On 2020-Jul-01, Fujii Masao wrote:
> >
> > > On 2020/07/01 12:26, Alvaro Herrera wrote:
> > > > On 2020-Jun-30, Fujii Masao wrote:
> > > >
> > > > > When I talked about max_slot_wal_keep_size as new feature in v13
> > > > > at the conference, I received the question like "Why are the units of
> > > > > setting values in max_slot_wal_keep_size and wal_keep_segments different?"
> > > > > from audience. That difference looks confusing for users and
> > > > > IMO it's better to use the same unit for them. Thought?
> > > >
> > > > Do we still need wal_keep_segments for anything?
> > >
> > > Yeah, personally I like wal_keep_segments because its setting is very
> > > simple and no extra operations on replication slots are necessary.
> >
> > Okay.  In that case I +1 the idea of renaming to wal_keep_size.
>
> +1 for renaming to wal_keep_size.

We have the following wal*size GUC settings:

        SELECT name FROM pg_settings WHERE name LIKE '%wal%size%';
                  name
        ------------------------
         max_slot_wal_keep_size
         max_wal_size
         min_wal_size
         wal_block_size
         wal_segment_size

Does wal_keep_size make sense here?

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Alvaro Herrera-9
On 2020-Jul-01, Bruce Momjian wrote:

> We have the following wal*size GUC settings:
>
> SELECT name FROM pg_settings WHERE name LIKE '%wal%size%';
>          name
> ------------------------
> max_slot_wal_keep_size
> max_wal_size
> min_wal_size
> wal_block_size
> wal_segment_size
>
> Does wal_keep_size make sense here?

I think it does.  What do you think?

Are you suggesting that "keep_wal_size" is better, since it's more in
line with min/max?  I lean towards no.

(I think it's okay to conceptually separate these three options from
wal_block_size, since that's a compile time option and thus it's an
introspective GUC rather than actual configuration, but as I recall that
argument does not hold for wal_segment_size. But at one point, even that
one was an introspective GUC too.)

--
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Bruce Momjian
On Wed, Jul  1, 2020 at 04:25:35PM -0400, Alvaro Herrera wrote:

> On 2020-Jul-01, Bruce Momjian wrote:
>
> > We have the following wal*size GUC settings:
> >
> > SELECT name FROM pg_settings WHERE name LIKE '%wal%size%';
> >          name
> > ------------------------
> > max_slot_wal_keep_size
> > max_wal_size
> > min_wal_size
> > wal_block_size
> > wal_segment_size
> >
> > Does wal_keep_size make sense here?
>
> I think it does.  What do you think?
>
> Are you suggesting that "keep_wal_size" is better, since it's more in
> line with min/max?  I lean towards no.

No, I am more just asking since I saw wal_keep_size as a special version
of wal_size.  I don't have a firm opinion.

>
> (I think it's okay to conceptually separate these three options from
> wal_block_size, since that's a compile time option and thus it's an
> introspective GUC rather than actual configuration, but as I recall that
> argument does not hold for wal_segment_size. But at one point, even that
> one was an introspective GUC too.)

Yep, just asking.

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: max_slot_wal_keep_size and wal_keep_segments

Masahiko Sawada-2
In reply to this post by David Steele
On Thu, 2 Jul 2020 at 02:18, David Steele <[hidden email]> wrote:

>
> On 7/1/20 10:54 AM, Alvaro Herrera wrote:
> > On 2020-Jul-01, Fujii Masao wrote:
> >
> >> On 2020/07/01 12:26, Alvaro Herrera wrote:
> >>> On 2020-Jun-30, Fujii Masao wrote:
> >>>
> >>>> When I talked about max_slot_wal_keep_size as new feature in v13
> >>>> at the conference, I received the question like "Why are the units of
> >>>> setting values in max_slot_wal_keep_size and wal_keep_segments different?"
> >>>> from audience. That difference looks confusing for users and
> >>>> IMO it's better to use the same unit for them. Thought?
> >>>
> >>> Do we still need wal_keep_segments for anything?
> >>
> >> Yeah, personally I like wal_keep_segments because its setting is very
> >> simple and no extra operations on replication slots are necessary.
> >
> > Okay.  In that case I +1 the idea of renaming to wal_keep_size.
>
> +1 for renaming to wal_keep_size.
>

+1 from me, too.

Regards,


--
Masahiko Sawada            http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services