BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

PG Doc comments form
The following bug has been logged on the website:

Bug reference:      16497
Logged by:          Ram Pratatp maurya
Email address:      [hidden email]
PostgreSQL version: 11.0
Operating system:   RHEL 6
Description:        

Hello team,

I am facing error during upgrade from postgresql -11 to 12.
can you please suggest what is solution.
.................................................................................................................
"bash-4.1$ /usr/pgsql-12/bin/pg_upgrade --old-datadir
/var/lib/pgsql/11/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir
/usr/pgsql-11/bin/ --new-bindir /usr/pgsql-12/bin/ --check
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

old and new pg_controldata WAL segment sizes are invalid or do not match
Failure, exiting
"
regards,
ram ptatap

Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Michael Paquier-2
On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
> I am facing error during upgrade from postgresql -11 to 12.
> can you please suggest what is solution.

As the error message is telling, you need to make the WAL segment
sizes of the old and new clusters match.  This can be defined at
server initialization time using initdb with a default of 16MB, so
that's not a bug from us, but something you need to take care of
within your upgrade process.
--
Michael

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Ram Pratap Maurya
Dear Michael,

Currently I hav installed postgres12 version , need to uninstall and reinstall again.
Or any command that we change current installation wal segment size.

Below location we have installed 12 version.

/var/lib/pgsql/12/data


Regards,
Ram Pratap.

-----Original Message-----
From: Michael Paquier [mailto:[hidden email]]
Sent: 18 June 2020 13:35
To: Ram Pratap Maurya; [hidden email]
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
> I am facing error during upgrade from postgresql -11 to 12.
> can you please suggest what is solution.

As the error message is telling, you need to make the WAL segment sizes of the old and new clusters match.  This can be defined at server initialization time using initdb with a default of 16MB, so that's not a bug from us, but something you need to take care of within your upgrade process.
--
Michael


Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Jeff Janes
In reply to this post by Michael Paquier-2
On Thu, Jun 18, 2020 at 4:05 AM Michael Paquier <[hidden email]> wrote:
On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
> I am facing error during upgrade from postgresql -11 to 12.
> can you please suggest what is solution.

As the error message is telling, you need to make the WAL segment
sizes of the old and new clusters match.  This can be defined at
server initialization time using initdb with a default of 16MB, so
that's not a bug from us, but something you need to take care of
within your upgrade process.

Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with this situation automatically by allowing the upgrade to also change this value, rather than forcing the user to make them match manually?
 
Cheers,

Jeff
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
On Thu, Jun 18, 2020 at 10:18:21AM -0400, Jeff Janes wrote:

> On Thu, Jun 18, 2020 at 4:05 AM Michael Paquier <[hidden email]> wrote:
>
>     On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
>     > I am facing error during upgrade from postgresql -11 to 12.
>     > can you please suggest what is solution.
>
>     As the error message is telling, you need to make the WAL segment
>     sizes of the old and new clusters match.  This can be defined at
>     server initialization time using initdb with a default of 16MB, so
>     that's not a bug from us, but something you need to take care of
>     within your upgrade process.
>
>
> Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade is
> already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with this
> situation automatically by allowing the upgrade to also change this value,
> rather than forcing the user to make them match manually?

Uh, well, pg_upgrade is very conservative and fails if anything looks
odd, like different WAL sizes.  Since pg_upgrade doesn't even copy the
WAL from the old server, I am not sure why we require the WAL segment
size to match, but it might be so we can run pg_upgrade on replicas.

The bottom line is that pg_upgrade is complex enough, and having it try
to do more is not a wise approach.

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Tom Lane-2
In reply to this post by Jeff Janes
Jeff Janes <[hidden email]> writes:
> Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> this situation automatically by allowing the upgrade to also change this
> value, rather than forcing the user to make them match manually?

The issue is that this is an initdb parameter, and pg_upgrade expects you
to have already initdb'd the destination cluster.  We could redefine that,
perhaps, but it'd be a large change in how one uses pg_upgrade and would
certainly break a lot of scripts.

I'm aware that we could use pg_resetwal to deal with this one specific
initdb parameter, but I see no point in hacking around the problem for
just one parameter.  The general principle remains that you need to
initdb the target with the same settings you used for the source.

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
On Thu, Jun 18, 2020 at 11:16:50AM -0400, Tom Lane wrote:

> Jeff Janes <[hidden email]> writes:
> > Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> > is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> > this situation automatically by allowing the upgrade to also change this
> > value, rather than forcing the user to make them match manually?
>
> The issue is that this is an initdb parameter, and pg_upgrade expects you
> to have already initdb'd the destination cluster.  We could redefine that,
> perhaps, but it'd be a large change in how one uses pg_upgrade and would
> certainly break a lot of scripts.
>
> I'm aware that we could use pg_resetwal to deal with this one specific
> initdb parameter, but I see no point in hacking around the problem for
> just one parameter.  The general principle remains that you need to
> initdb the target with the same settings you used for the source.

Yep, that's pretty much the issue.  :-)

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Stephen Frost
Greetings,

* Bruce Momjian ([hidden email]) wrote:

> On Thu, Jun 18, 2020 at 11:16:50AM -0400, Tom Lane wrote:
> > Jeff Janes <[hidden email]> writes:
> > > Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> > > is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> > > this situation automatically by allowing the upgrade to also change this
> > > value, rather than forcing the user to make them match manually?
> >
> > The issue is that this is an initdb parameter, and pg_upgrade expects you
> > to have already initdb'd the destination cluster.  We could redefine that,
> > perhaps, but it'd be a large change in how one uses pg_upgrade and would
> > certainly break a lot of scripts.
I don't think we really need to redefine that, what it seems we'd need
to change is to not have pg_upgrade complain about things being
different between the two which don't actually matter to pg_upgrade.

> > I'm aware that we could use pg_resetwal to deal with this one specific
> > initdb parameter, but I see no point in hacking around the problem for
> > just one parameter.  The general principle remains that you need to
> > initdb the target with the same settings you used for the source.
>
> Yep, that's pretty much the issue.  :-)

Well, except that for pg_upgrade to do its job correctly, the two
clusters *don't* actually need be initdb'd exactly the same way, do they?

Sure, most options to initdb need to be the same between the old cluster
and the new cluster, but this specific option doesn't have to be, since
we require that it's a cleanly shut down cluster, so why are we
complaining about it if it's different..?

Thanks,

Stephen

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> Sure, most options to initdb need to be the same between the old cluster
> and the new cluster, but this specific option doesn't have to be, since
> we require that it's a cleanly shut down cluster, so why are we
> complaining about it if it's different..?

Did you not read my previous email that we might have added this so we
can upgrade replicas?  I am sure I can dig out the commit that added
this and find the original cause, but running pg_upgrade on replicas is
enough of a reason to me.  Yeah, we could add a flag to pg_upgrade to
allow this if you are not upgrading replicas, but why bother?  It might
even work if you create the new replicas with the same WAL segment size,
but why add complexity for pg_upgrade, which is already complex enough.

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Stephen Frost
Greetings,

* Bruce Momjian ([hidden email]) wrote:
> On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> > Sure, most options to initdb need to be the same between the old cluster
> > and the new cluster, but this specific option doesn't have to be, since
> > we require that it's a cleanly shut down cluster, so why are we
> > complaining about it if it's different..?
>
> Did you not read my previous email that we might have added this so we
> can upgrade replicas?

I don't see how this option is related to dealing with replicas..?

> I am sure I can dig out the commit that added
> this and find the original cause, but running pg_upgrade on replicas is
> enough of a reason to me.

pg_upgrade can't be run on replicas, so I don't understand what you're
referring to here..  The 'upgrading replicas' process that involves
rsync also requires everything to have been shut down cleanly.

> Yeah, we could add a flag to pg_upgrade to
> allow this if you are not upgrading replicas, but why bother?  It might
> even work if you create the new replicas with the same WAL segment size,
> but why add complexity for pg_upgrade, which is already complex enough.

Users already have to deal with various options that need to be
configured to match up between the primary and replicas, so this really
seems like it's entirely independent of pg_upgrade and isn't something
pg_upgrade needs to be worrying about..

Thanks,

Stephen

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Jeff Janes
In reply to this post by Tom Lane-2
On Thu, Jun 18, 2020 at 11:16 AM Tom Lane <[hidden email]> wrote:
Jeff Janes <[hidden email]> writes:
> Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> this situation automatically by allowing the upgrade to also change this
> value, rather than forcing the user to make them match manually?

The issue is that this is an initdb parameter, and pg_upgrade expects you
to have already initdb'd the destination cluster.  We could redefine that,
perhaps, but it'd be a large change in how one uses pg_upgrade and would
certainly break a lot of scripts.

I'm aware that we could use pg_resetwal to deal with this one specific
initdb parameter, but I see no point in hacking around the problem for
just one parameter.  The general principle remains that you need to
initdb the target with the same settings you used for the source.

Since you mention it, now that -B is not necessary (inferred from where pg_upgrade itself is found), I have certainly thought it would also be nice if -D could point to a non-existent or empty directory, and have it do the initdb for you.  It would be nice to have it figure out the correct settings of -E, -U, --lc-collate, --lc-ctype and whatever else is needed to make --check pass, rather than doing it all manually (and one at a time, since it stops at the first error).  It doesn't seem like this, or the previously described change, would break any scripts which currently work.  It might cause some currently broken ones to work in ways that are unexpected, though.
 
Cheers,

Jeff
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
In reply to this post by Stephen Frost
On Thu, Jun 18, 2020 at 11:55:37AM -0400, Stephen Frost wrote:

> Greetings,
>
> * Bruce Momjian ([hidden email]) wrote:
> > On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> > > Sure, most options to initdb need to be the same between the old cluster
> > > and the new cluster, but this specific option doesn't have to be, since
> > > we require that it's a cleanly shut down cluster, so why are we
> > > complaining about it if it's different..?
> >
> > Did you not read my previous email that we might have added this so we
> > can upgrade replicas?
>
> I don't see how this option is related to dealing with replicas..?
>
> > I am sure I can dig out the commit that added
> > this and find the original cause, but running pg_upgrade on replicas is
> > enough of a reason to me.
>
> pg_upgrade can't be run on replicas, so I don't understand what you're
> referring to here..  The 'upgrading replicas' process that involves
> rsync also requires everything to have been shut down cleanly.

Oh, I forgot we copy the WAL from the primary and don't run initdb on
the standbys, so it might work, but looking at the pg_upgrade code, I
see:

    /* now reset the wal archives in the new cluster */
    prep_status("Resetting WAL archives");
    exec_prog(UTILITY_LOG_FILE, NULL, true, true,
    /* use timeline 1 to match controldata and no WAL history file */
              "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
              old_cluster.controldata.nextxlogfile + 8,
              new_cluster.pgdata);

So, while we don't copy over the old WAL, we do assume the WAL will be
at the same segment from the old cluster.  I think this is because of
the LSN stored on user data pages.  pg_resetwal doesn't seem to care
about that, so maybe pg_upgrade doesn't need to either.  I don't know.

> > Yeah, we could add a flag to pg_upgrade to
> > allow this if you are not upgrading replicas, but why bother?  It might
> > even work if you create the new replicas with the same WAL segment size,
> > but why add complexity for pg_upgrade, which is already complex enough.
>
> Users already have to deal with various options that need to be
> configured to match up between the primary and replicas, so this really
> seems like it's entirely independent of pg_upgrade and isn't something
> pg_upgrade needs to be worrying about..

Do you know why we require this step?

        https://www.postgresql.org/docs/12/pgupgrade.html
       
        Also, change wal_level to replica in the postgresql.conf file on
        the new primary cluster.

The other modes don't work?  I see this C comment:

     * We unconditionally start/stop the new server because pg_resetwal -o set
     * wal_level to 'minimum'.  If the user is upgrading standby servers using
     * the rsync instructions, they will need pg_upgrade to write its final
     * WAL record showing wal_level as 'replica'.

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
In reply to this post by Jeff Janes
On Thu, Jun 18, 2020 at 12:14:25PM -0400, Jeff Janes wrote:

> On Thu, Jun 18, 2020 at 11:16 AM Tom Lane <[hidden email]> wrote:
>
>     Jeff Janes <[hidden email]> writes:
>     > Since wal-segsize is changeable with pg_resetwal since v11, and
>     pg_upgrade
>     > is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal
>     with
>     > this situation automatically by allowing the upgrade to also change this
>     > value, rather than forcing the user to make them match manually?
>
>     The issue is that this is an initdb parameter, and pg_upgrade expects you
>     to have already initdb'd the destination cluster.  We could redefine that,
>     perhaps, but it'd be a large change in how one uses pg_upgrade and would
>     certainly break a lot of scripts.
>
>     I'm aware that we could use pg_resetwal to deal with this one specific
>     initdb parameter, but I see no point in hacking around the problem for
>     just one parameter.  The general principle remains that you need to
>     initdb the target with the same settings you used for the source.
>
>
> Since you mention it, now that -B is not necessary (inferred from where
> pg_upgrade itself is found), I have certainly thought it would also be nice if
> -D could point to a non-existent or empty directory, and have it do the initdb
> for you.  It would be nice to have it figure out the correct settings of -E,
> -U, --lc-collate, --lc-ctype and whatever else is needed to make --check pass,
> rather than doing it all manually (and one at a time, since it stops at the
> first error).  It doesn't seem like this, or the previously described change,
> would break any scripts which currently work.  It might cause some currently
> broken ones to work in ways that are unexpected, though.

Yes, it would be nice, but there is also the requirement of adjusting
postgresql.conf to match the old cluster.  I am not sure we can get away
without that step, but looking at how people automate pg_upgrade would
help determine that.

Basically, if pg_upgrade ran initdb, we would need to automate steps 5-8
here:

        https://www.postgresql.org/docs/12/pgupgrade.html

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Stephen Frost
In reply to this post by Bruce Momjian
Greetings,

* Bruce Momjian ([hidden email]) wrote:

> On Thu, Jun 18, 2020 at 11:55:37AM -0400, Stephen Frost wrote:
> > * Bruce Momjian ([hidden email]) wrote:
> > > On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> > > > Sure, most options to initdb need to be the same between the old cluster
> > > > and the new cluster, but this specific option doesn't have to be, since
> > > > we require that it's a cleanly shut down cluster, so why are we
> > > > complaining about it if it's different..?
> > >
> > > Did you not read my previous email that we might have added this so we
> > > can upgrade replicas?
> >
> > I don't see how this option is related to dealing with replicas..?
> >
> > > I am sure I can dig out the commit that added
> > > this and find the original cause, but running pg_upgrade on replicas is
> > > enough of a reason to me.
> >
> > pg_upgrade can't be run on replicas, so I don't understand what you're
> > referring to here..  The 'upgrading replicas' process that involves
> > rsync also requires everything to have been shut down cleanly.
>
> Oh, I forgot we copy the WAL from the primary and don't run initdb on
> the standbys, so it might work, but looking at the pg_upgrade code, I
> see:
>
>     /* now reset the wal archives in the new cluster */
>     prep_status("Resetting WAL archives");
>     exec_prog(UTILITY_LOG_FILE, NULL, true, true,
>     /* use timeline 1 to match controldata and no WAL history file */
>               "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
>               old_cluster.controldata.nextxlogfile + 8,
>               new_cluster.pgdata);
>
> So, while we don't copy over the old WAL, we do assume the WAL will be
> at the same segment from the old cluster.  I think this is because of
> the LSN stored on user data pages.  pg_resetwal doesn't seem to care
> about that, so maybe pg_upgrade doesn't need to either.  I don't know.
Yeah, this would need to be adjusted, since we're passing in an actual WAL
filename there rather than an LSN.

> > > Yeah, we could add a flag to pg_upgrade to
> > > allow this if you are not upgrading replicas, but why bother?  It might
> > > even work if you create the new replicas with the same WAL segment size,
> > > but why add complexity for pg_upgrade, which is already complex enough.
> >
> > Users already have to deal with various options that need to be
> > configured to match up between the primary and replicas, so this really
> > seems like it's entirely independent of pg_upgrade and isn't something
> > pg_upgrade needs to be worrying about..
>
> Do you know why we require this step?
>
> https://www.postgresql.org/docs/12/pgupgrade.html
>
> Also, change wal_level to replica in the postgresql.conf file on
> the new primary cluster.
Well, we'll need wal_level to be at least replica if we're going to have
replicas streaming from the primary..

> The other modes don't work?  I see this C comment:
>
>      * We unconditionally start/stop the new server because pg_resetwal -o set
>      * wal_level to 'minimum'.  If the user is upgrading standby servers using
>      * the rsync instructions, they will need pg_upgrade to write its final
>      * WAL record showing wal_level as 'replica'.

Yes, the new system needs to have replica (or higher) as the wal_level,
but that doesn't depend on the WAL segment size.

Thanks,

Stephen

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:

> > > > Yeah, we could add a flag to pg_upgrade to
> > > > allow this if you are not upgrading replicas, but why bother?  It might
> > > > even work if you create the new replicas with the same WAL segment size,
> > > > but why add complexity for pg_upgrade, which is already complex enough.
> > >
> > > Users already have to deal with various options that need to be
> > > configured to match up between the primary and replicas, so this really
> > > seems like it's entirely independent of pg_upgrade and isn't something
> > > pg_upgrade needs to be worrying about..
> >
> > Do you know why we require this step?
> >
> > https://www.postgresql.org/docs/12/pgupgrade.html
> >
> > Also, change wal_level to replica in the postgresql.conf file on
> > the new primary cluster.
>
> Well, we'll need wal_level to be at least replica if we're going to have
> replicas streaming from the primary..

But how do they have replicas if wal_level = minimum?  Also, why not
higher replication levels?  Should we adjust that doc text?

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Stephen Frost
Greetings,

* Bruce Momjian ([hidden email]) wrote:

> On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > > Yeah, we could add a flag to pg_upgrade to
> > > > > allow this if you are not upgrading replicas, but why bother?  It might
> > > > > even work if you create the new replicas with the same WAL segment size,
> > > > > but why add complexity for pg_upgrade, which is already complex enough.
> > > >
> > > > Users already have to deal with various options that need to be
> > > > configured to match up between the primary and replicas, so this really
> > > > seems like it's entirely independent of pg_upgrade and isn't something
> > > > pg_upgrade needs to be worrying about..
> > >
> > > Do you know why we require this step?
> > >
> > > https://www.postgresql.org/docs/12/pgupgrade.html
> > >
> > > Also, change wal_level to replica in the postgresql.conf file on
> > > the new primary cluster.
> >
> > Well, we'll need wal_level to be at least replica if we're going to have
> > replicas streaming from the primary..
>
> But how do they have replicas if wal_level = minimum?  Also, why not
> higher replication levels?  Should we adjust that doc text?
I think the comment is saying that pg_resetwal will rewrite the
pg_control with a WAL level of minimal and that's the issue and why
the server needs to be brought up with a higher WAL level temporarily,
so that pg_control gets updated, for the new cluster.

Of course, before pg_upgrade is run, the old cluster would need to be up
and running with a wal_level higher than minimal in order to have
replicas in the first place, but what we're really talking about here is
the new, upgraded, cluster.

I do think the doc could probably say replica 'or higher'.

Thanks,

Stephen

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Bruce Momjian
On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:

> Greetings,
>
> * Bruce Momjian ([hidden email]) wrote:
> > On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > > > Yeah, we could add a flag to pg_upgrade to
> > > > > > allow this if you are not upgrading replicas, but why bother?  It might
> > > > > > even work if you create the new replicas with the same WAL segment size,
> > > > > > but why add complexity for pg_upgrade, which is already complex enough.
> > > > >
> > > > > Users already have to deal with various options that need to be
> > > > > configured to match up between the primary and replicas, so this really
> > > > > seems like it's entirely independent of pg_upgrade and isn't something
> > > > > pg_upgrade needs to be worrying about..
> > > >
> > > > Do you know why we require this step?
> > > >
> > > > https://www.postgresql.org/docs/12/pgupgrade.html
> > > >
> > > > Also, change wal_level to replica in the postgresql.conf file on
> > > > the new primary cluster.
> > >
> > > Well, we'll need wal_level to be at least replica if we're going to have
> > > replicas streaming from the primary..
> >
> > But how do they have replicas if wal_level = minimum?  Also, why not
> > higher replication levels?  Should we adjust that doc text?
>
> I think the comment is saying that pg_resetwal will rewrite the
> pg_control with a WAL level of minimal and that's the issue and why
> the server needs to be brought up with a higher WAL level temporarily,
> so that pg_control gets updated, for the new cluster.
>
> Of course, before pg_upgrade is run, the old cluster would need to be up
> and running with a wal_level higher than minimal in order to have
> replicas in the first place, but what we're really talking about here is
> the new, upgraded, cluster.
>
> I do think the doc could probably say replica 'or higher'.

OK, I will work on that, thanks.

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Michael Paquier-2
On Thu, Jun 18, 2020 at 02:29:43PM -0400, Bruce Momjian wrote:
> On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:
>> I do think the doc could probably say replica 'or higher'.

+1.  Sounds good to me to just use "or higher" here.  The docs insist
on the concept of hierarchy for values of wal_level.

> OK, I will work on that, thanks.

Thanks.
--
Michael

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Ram Pratap Maurya
In reply to this post by Bruce Momjian
Dear All,

any impact of server. If we Can reset manually postgres old version (11) WAL segment size .


Regards,
Ram Pratap.
Lava International Limited.
Tel+  91-120-4637148



-----Original Message-----
From: Bruce Momjian [mailto:[hidden email]]
Sent: 19 June 2020 00:00
To: Stephen Frost
Cc: Tom Lane; Jeff Janes; Michael Paquier; Ram Pratap Maurya; pgsql-bugs
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:

> Greetings,
>
> * Bruce Momjian ([hidden email]) wrote:
> > On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > > > Yeah, we could add a flag to pg_upgrade to allow this if you
> > > > > > are not upgrading replicas, but why bother?  It might even
> > > > > > work if you create the new replicas with the same WAL
> > > > > > segment size, but why add complexity for pg_upgrade, which is already complex enough.
> > > > >
> > > > > Users already have to deal with various options that need to
> > > > > be configured to match up between the primary and replicas, so
> > > > > this really seems like it's entirely independent of pg_upgrade
> > > > > and isn't something pg_upgrade needs to be worrying about..
> > > >
> > > > Do you know why we require this step?
> > > >
> > > > https://www.postgresql.org/docs/12/pgupgrade.html
> > > >
> > > > Also, change wal_level to replica in the postgresql.conf file on
> > > > the new primary cluster.
> > >
> > > Well, we'll need wal_level to be at least replica if we're going
> > > to have replicas streaming from the primary..
> >
> > But how do they have replicas if wal_level = minimum?  Also, why not
> > higher replication levels?  Should we adjust that doc text?
>
> I think the comment is saying that pg_resetwal will rewrite the
> pg_control with a WAL level of minimal and that's the issue and why
> the server needs to be brought up with a higher WAL level temporarily,
> so that pg_control gets updated, for the new cluster.
>
> Of course, before pg_upgrade is run, the old cluster would need to be
> up and running with a wal_level higher than minimal in order to have
> replicas in the first place, but what we're really talking about here
> is the new, upgraded, cluster.
>
> I do think the doc could probably say replica 'or higher'.

OK, I will work on that, thanks.

--
  Bruce Momjian  <[hidden email]>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee



Reply | Threaded
Open this post in threaded view
|

RE: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Ram Pratap Maurya
In reply to this post by Bruce Momjian

Dear All,

 

We have reinstall postgres-12 version but when I run below command  its showing error again.

Please suggest what I do to resolve this issue.

 

Command:

/usr/pgsql-12/bin/pg_upgrade --old-datadir /var/lib/pgsql/11/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir /usr/pgsql-11/bin/ --new-bindir /usr/pgsql-12/bin/ --check

 

 

Regards,

Ram Pratap.

Lava International Limited.

Tel+  91-120-4637148

 

 

 

-----Original Message-----
From: Bruce Momjian [mailto:[hidden email]]
Sent: 19 June 2020 00:00
To: Stephen Frost
Cc: Tom Lane; Jeff Janes; Michael Paquier; Ram Pratap Maurya; pgsql-bugs
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

 

On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:

> Greetings,

>

> * Bruce Momjian ([hidden email]) wrote:

> > On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:

> > > > > > Yeah, we could add a flag to pg_upgrade to allow this if you

> > > > > > are not upgrading replicas, but why bother?  It might even

> > > > > > work if you create the new replicas with the same WAL

> > > > > > segment size, but why add complexity for pg_upgrade, which is already complex enough.

> > > > >

> > > > > Users already have to deal with various options that need to

> > > > > be configured to match up between the primary and replicas, so

> > > > > this really seems like it's entirely independent of pg_upgrade

> > > > > and isn't something pg_upgrade needs to be worrying about..

> > > >

> > > > Do you know why we require this step?

> > > >

> > > > https://www.postgresql.org/docs/12/pgupgrade.html

> > > >

> > > > Also, change wal_level to replica in the postgresql.conf file on

> > > > the new primary cluster.

> > >

> > > Well, we'll need wal_level to be at least replica if we're going

> > > to have replicas streaming from the primary..

> >

> > But how do they have replicas if wal_level = minimum?  Also, why not

> > higher replication levels?  Should we adjust that doc text?

>

> I think the comment is saying that pg_resetwal will rewrite the

> pg_control with a WAL level of minimal and that's the issue and why

> the server needs to be brought up with a higher WAL level temporarily,

> so that pg_control gets updated, for the new cluster.

>

> Of course, before pg_upgrade is run, the old cluster would need to be

> up and running with a wal_level higher than minimal in order to have

> replicas in the first place, but what we're really talking about here

> is the new, upgraded, cluster.

>

> I do think the doc could probably say replica 'or higher'.

 

OK, I will work on that, thanks.

 

--

  Bruce Momjian  <[hidden email]>        https://momjian.us

  EnterpriseDB                             https://enterprisedb.com

 

  The usefulness of a cup is in its emptiness, Bruce Lee

 

12