PATCH: Exclude temp relations from base backup

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

PATCH: Exclude temp relations from base backup

David Steele
This is a follow-up patch from the exclude unlogged relations discussion
[1].

The patch excludes temporary relations during a base backup using the
existing looks_like_temp_rel_name() function for identification.

It shares code to identify database directories from [1], so for now
that has been duplicated in this patch to make it independent.  I'll
rebase depending on what gets committed first.

Thanks,
--
-David
[hidden email]

[1]
https://www.postgresql.org/message-id/04791bab-cb04-ba43-e9c0-664a4c1ffb2c%40pgmasters.net

backup-exclude-temp-rel-v1.patch (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Exclude temp relations from base backup

David Steele
Hi,

On 2/28/18 10:55 AM, David Steele wrote:
> This is a follow-up patch from the exclude unlogged relations discussion
> [1].
>
> The patch excludes temporary relations during a base backup using the
> existing looks_like_temp_rel_name() function for identification.
>
> It shares code to identify database directories from [1], so for now
> that has been duplicated in this patch to make it independent.  I'll
> rebase depending on what gets committed first.

Updated the patch to change die() to BAIL_OUT() and use append_to_file()
as suggested for another test patch [1].

Regards,
--
-David
[hidden email]

[1]
https://www.postgresql.org/message-id/6bc5d931-5b00-279f-f65a-26e32de400a6%40pgmasters.net

backup-exclude-temp-rel-v2.patch (10K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Exclude temp relations from base backup

David Steele
On 3/13/18 12:34 PM, David Steele wrote:

> Updated the patch to change die() to BAIL_OUT() and use append_to_file()
> as suggested for another test patch.

Updated patch now that the unlogged table exclusions have been committed
[1].

Thanks,
--
-David
[hidden email]

[1]
https://www.postgresql.org/message-id/4d9be1c0-5c58-d9a0-7152-2771224910ae%40sigaev.ru

backup-exclude-temp-rel-v3.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Exclude temp relations from base backup

Teodor Sigaev
Hi!

Will autovacuum (or something else) complain about absense of relfile during
orphan table deleting? I mean, you get a base backup without temp tables, then
you try to run postgres on it and will it complain about existing record in
pg_class and absence of corresponding relfile?


David Steele wrote:

> On 3/13/18 12:34 PM, David Steele wrote:
>
>> Updated the patch to change die() to BAIL_OUT() and use append_to_file()
>> as suggested for another test patch.
>
> Updated patch now that the unlogged table exclusions have been committed
> [1].
>
> Thanks,
>

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/

Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Exclude temp relations from base backup

Stephen Frost
Greetings,

* Teodor Sigaev ([hidden email]) wrote:
> Will autovacuum (or something else) complain about absense of relfile during
> orphan table deleting? I mean, you get a base backup without temp tables,
> then you try to run postgres on it and will it complain about existing
> record in pg_class and absence of corresponding relfile?

I would certainly hope not considering that's what happens during
regular crash recovery also, so if there's an issue with that, we'd have
a problem in released versions.

There's an independent discussion that was being had recently about how
to make sure those records in pg_class get cleaned up in a reasonable
timeframe and don't lead to problems with wrap-arounds, but that's a
different and pre-existing issue.

Thanks!

Stephen

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Exclude temp relations from base backup

David Steele
On 3/26/18 1:06 PM, Stephen Frost wrote:

>
> * Teodor Sigaev ([hidden email]) wrote:
>> Will autovacuum (or something else) complain about absense of relfile during
>> orphan table deleting? I mean, you get a base backup without temp tables,
>> then you try to run postgres on it and will it complain about existing
>> record in pg_class and absence of corresponding relfile?
>
> I would certainly hope not considering that's what happens during
> regular crash recovery also, so if there's an issue with that, we'd have
> a problem in released versions.
Agreed.  The logic for pg_basebackup was modeled off RemovePgTempFiles()
which is called at postmaster start.  We are just doing the cleanup in
advance (in the backup only, of course).

Thanks,
--
-David
[hidden email]


signature.asc (891 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Exclude temp relations from base backup

Teodor Sigaev
Thank you, pushed

David Steele wrote:

> On 3/26/18 1:06 PM, Stephen Frost wrote:
>>
>> * Teodor Sigaev ([hidden email]) wrote:
>>> Will autovacuum (or something else) complain about absense of relfile during
>>> orphan table deleting? I mean, you get a base backup without temp tables,
>>> then you try to run postgres on it and will it complain about existing
>>> record in pg_class and absence of corresponding relfile?
>>
>> I would certainly hope not considering that's what happens during
>> regular crash recovery also, so if there's an issue with that, we'd have
>> a problem in released versions.
>
> Agreed.  The logic for pg_basebackup was modeled off RemovePgTempFiles()
> which is called at postmaster start.  We are just doing the cleanup in
> advance (in the backup only, of course).
>
> Thanks,
>

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/

Previous Thread Next Thread