pg_rewind copies

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

pg_rewind copies

Heikki Linnakangas
If a file is modified and becomes larger in the source system while
pg_rewind is running, pg_rewind can leave behind a partial copy of file.
That's by design, and it's OK for relation files because they're
replayed from WAL. But it can cause trouble for configuration files.

I ran into this while playing with pg_auto_failover. After failover,
pg_auto_failover would often launch pg_rewind, and run ALTER SYSTEM on
the primary while pg_rewind was running. The resulting rewound system
would fail to start up:

Nov 13 09:24:42 pg-node-a pg_autoctl[2217]: 09:24:42 2220 ERROR
2020-11-13 09:24:32.547 GMT [2246] LOG:  syntax error in file
"/data/pgdata/postgresql.auto.conf" line 4, near token "'"
Nov 13 09:24:42 pg-node-a pg_autoctl[2217]: 09:24:42 2220 ERROR
2020-11-13 09:24:32.547 GMT [2246] FATAL:  configuration file
"postgresql.auto.conf" contains errors

Attached is a patch to mitigate that. It changes pg_rewind so that when
it copies a whole file, it ignores the original file size. It's not a
complete cure: it still believes the original size for files larger than
1 MB. That limit was just expedient given the way the chunking logic in
libpq_source.c works, but should be enough for configuration files.

There's another race condition that this doesn't try to fix: If a file
is modified while it's being copied, you can have a torn file with one
half of the file from the old version, and one half from the new. That's
a much more narrow window, though, and pg_basebackup has the same problem.

- Heikki

0001-pg_rewind-Fetch-small-files-according-to-new-size.patch (10K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pg_rewind copies

Cary Huang-2
The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            not tested

Hello

The patch seems to do as described and the regression and tap tests are passing
+ /*
+ * A local source is not expected to change while we're rewinding, so check
+ * that we size of the file matches our earlier expectation.
+ */
Is this a tyoo?

thanks
Cary
Reply | Threaded
Open this post in threaded view
|

Re: pg_rewind copies

Heikki Linnakangas
On 16/12/2020 00:08, Cary Huang wrote:

> The following review has been posted through the commitfest application:
> make installcheck-world:  tested, passed
> Implements feature:       tested, passed
> Spec compliant:           tested, passed
> Documentation:            not tested
>
> Hello
>
> The patch seems to do as described and the regression and tap tests are passing
> + /*
> + * A local source is not expected to change while we're rewinding, so check
> + * that we size of the file matches our earlier expectation.
> + */
> Is this a tyoo?
Yep, thanks! Attached is a new patch version, with that fixed and
rebased. No other changes.

- Heikki

v2-0001-pg_rewind-Fetch-small-files-according-to-new-size.patch (10K) Download Attachment