[PATCH] remove pg_archivecleanup and pg_standby

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] remove pg_archivecleanup and pg_standby

Justin Pryzby
Forking this thread:
https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def@...

I think these are old-fashioned since 9.6 (?), so remove them for v14.

I found it confusing when re-familiarizing myself with modern streaming
replication that there are extensions which only help do things the "old way".

v1-0001-Retire-pg_standby.patch (44K) Download Attachment
v1-0002-Retire-pg_archivecleanup.patch (107K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_archivecleanup and pg_standby

Michael Banck-2
Hi,

Am Mittwoch, den 28.10.2020, 21:44 -0500 schrieb Justin Pryzby:
> Forking this thread:
> https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def@...

Glancing over this in the context of pg_standby/pg_archivecleanup, I am
not sure Heikki's "Ditto" is about "remove pg_archivecleanup just like
pg_standby" or rather "keep the note until we get around doing something
with it". Probably the former, but see below.

> I think these are old-fashioned since 9.6 (?), so remove them for v14.

Why 9.6?

> I found it confusing when re-familiarizing myself with modern streaming
> replication that there are extensions which only help do things the "old way".

I guess not many will complain about pg_standby going away, but I am
under the impression that pg_archivecleanup is still used a lot in PITR
backup environments as a handy tool to expire WAL related to expired
base backups. I certainly saw hand-assembled shell code fail with "too
many files" and things when it tried to act on large amount of WAL.

So I think the part about it being used in archive_cleanup_command can
be probably be removed, but the part about it being useful as a stand-
alone tool, in particular this part:

|In this mode, if you specify a .partial or .backup file name, then only

|the file prefix will be used as the oldestkeptwalfile. This treatment
|of .backup file name allows you to remove all WAL files archived prior
|to a specific base backup without error.

At the very least, the commit message should give a rationale on why
pg_archivebackup is retired, and what it should be replaced with, in
case valid use-cases for it are still present.


Michael

--
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax:  +49 2166 9901-100
Email: [hidden email]

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Unser Umgang mit personenbezogenen Daten unterliegt
folgenden Bestimmungen: https://www.credativ.de/datenschutz



Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_archivecleanup and pg_standby

Justin Pryzby
On Thu, Oct 29, 2020 at 08:40:31PM +0100, Michael Banck wrote:
> Am Mittwoch, den 28.10.2020, 21:44 -0500 schrieb Justin Pryzby:
> > Forking this thread:
> > https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def@...

> > I think these are old-fashioned since 9.6 (?), so remove them for v14.
>
> Why 9.6?

My work doesn't currently bring me in contact with replication, so I've had to
dig through release notes.  I think streaming replication was new in 9.0, and
increasingly mature throughout 9.x.  Maybe someone else will say a different
release was when streaming replication became the norm and wal shipping old.

> > I found it confusing when re-familiarizing myself with modern streaming
> > replication that there are extensions which only help do things the "old way".
>
> I guess not many will complain about pg_standby going away, but I am
> under the impression that pg_archivecleanup is still used a lot in PITR
> backup environments as a handy tool to expire WAL related to expired
> base backups. I certainly saw hand-assembled shell code fail with "too
> many files" and things when it tried to act on large amount of WAL.

I anticipate you're right, and I'll withdraw 0002.

--
Justin


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_archivecleanup and pg_standby

Heikki Linnakangas
On 02/11/2020 20:26, Justin Pryzby wrote:

> On Thu, Oct 29, 2020 at 08:40:31PM +0100, Michael Banck wrote:
>> Am Mittwoch, den 28.10.2020, 21:44 -0500 schrieb Justin Pryzby:
>>> Forking this thread:
>>> https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def@...
>
>>> I think these are old-fashioned since 9.6 (?), so remove them for v14.
>>
>> Why 9.6?
>
> My work doesn't currently bring me in contact with replication, so I've had to
> dig through release notes.  I think streaming replication was new in 9.0, and
> increasingly mature throughout 9.x.  Maybe someone else will say a different
> release was when streaming replication became the norm and wal shipping old.

Removing pg_standby has been proposed a couple of times in the past. See
https://www.postgresql.org/message-id/20170913064824.rqflkadxwpboabgw@... 
for the latest attempt.

Masao-san, back in 2014 you mentioned "fast failover" as a feature that
was missing from the built-in standby mode
(https://www.postgresql.org/message-id/CAHGQGwEE_8vvpQk0ex6Qa_aXt-OSJ7OdZjX4uM_FtqKfxq5SbQ%40mail.gmail.com).
I think that's been implemented since, with the recovery_target
settings. Would you agree?

I'm pretty sure we can remove pg_standby by now. But if there's
something crucial missing from the built-in facilities, we need to talk
about implementing them.

- Heikki


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_archivecleanup and pg_standby

Robert Haas
In reply to this post by Michael Banck-2
On Thu, Oct 29, 2020 at 3:40 PM Michael Banck <[hidden email]> wrote:
> I guess not many will complain about pg_standby going away, but I am
> under the impression that pg_archivecleanup is still used a lot in PITR
> backup environments as a handy tool to expire WAL related to expired
> base backups. I certainly saw hand-assembled shell code fail with "too
> many files" and things when it tried to act on large amount of WAL.

Yeah, I see pg_archivecleanup used in customer environments all the
time. Like just this morning, for example.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_archivecleanup and pg_standby

Michael Paquier-2
In reply to this post by Heikki Linnakangas
On Tue, Nov 03, 2020 at 05:28:46PM +0200, Heikki Linnakangas wrote:

> Removing pg_standby has been proposed a couple of times in the past. See https://www.postgresql.org/message-id/20170913064824.rqflkadxwpboabgw@...
> for the latest attempt.
>
> Masao-san, back in 2014 you mentioned "fast failover" as a feature that was
> missing from the built-in standby mode (https://www.postgresql.org/message-id/CAHGQGwEE_8vvpQk0ex6Qa_aXt-OSJ7OdZjX4uM_FtqKfxq5SbQ%40mail.gmail.com).
> I think that's been implemented since, with the recovery_target settings.
> Would you agree?
>
> I'm pretty sure we can remove pg_standby by now. But if there's something
> crucial missing from the built-in facilities, we need to talk about
> implementing them.
Reading the thread you are mentioning, it seems to me that the
statu-quo is the same, but I find rather scary that this tool is used
in exactly zero tests.

Echoing with Robert, I think that pg_archivecleanup is still useful in
many cases, so that's not something we should remove.
--
Michael

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_archivecleanup and pg_standby

Peter Eisentraut-7
In reply to this post by Justin Pryzby
On 2020-10-29 03:44, Justin Pryzby wrote:

> diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
> index 4e833d79ef..be4292ec33 100644
> --- a/doc/src/sgml/contrib.sgml
> +++ b/doc/src/sgml/contrib.sgml
> @@ -199,6 +199,5 @@ pages.
>      part of the core <productname>PostgreSQL</productname> distribution.
>     </para>
>  
> - &pgstandby;
>    </sect1>
>   </appendix>

With this removal, that section becomes empty.  So you probably want to
clean up or reorganize this a bit.

See https://www.postgresql.org/docs/devel/contrib-prog.html for the context.


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] remove pg_standby

Justin Pryzby
On Fri, Nov 20, 2020 at 05:26:54PM +0100, Peter Eisentraut wrote:

> On 2020-10-29 03:44, Justin Pryzby wrote:
> > diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
> > index 4e833d79ef..be4292ec33 100644
> > --- a/doc/src/sgml/contrib.sgml
> > +++ b/doc/src/sgml/contrib.sgml
> > @@ -199,6 +199,5 @@ pages.
> >      part of the core <productname>PostgreSQL</productname> distribution.
> >     </para>
> > - &pgstandby;
> >    </sect1>
> >   </appendix>
>
> With this removal, that section becomes empty.  So you probably want to
> clean up or reorganize this a bit.
>
> See https://www.postgresql.org/docs/devel/contrib-prog.html for the context.
Oops.  I guess I'd write something like this.  If we just remove it, then
there'd no place to add a new server application, and "client applications"
would be the only subsection.

--
Justin

v2-0001-Add-missing-word-are.patch (1K) Download Attachment
v2-0002-Retire-pg_standby.patch (45K) Download Attachment