Extending amcheck to check toast size and compression
During the version 14 development period, a few checks of toasted attributes were written but never committed. For the version 15 development cycle, I'd like to consider extending the checks of toasted attributes. First, no toasted attribute should ever have a rawsize larger than the 1GB varlena limit. Second, no compressed toasted attribute should have an extsize indicating that the toast expanded during toasting. Such a extsize could mean the compression code is malfunctioning, or that the extsize or rawsize fields are corrupt. Third, any compressed attribute should have a valid compression method ID.
These checks are cheap. Actually retrieving the compressed toasted data and checking that it uncompresses correctly would have very different performance implications, but that is not included in this patch.
Re: Extending amcheck to check toast size and compression
> On May 4, 2021, at 9:43 AM, Justin Pryzby <[hidden email]> wrote:
> + /* Oversized toasted attributes should never be stored */
> + if (toast_pointer.va_rawsize > VARLENA_SIZE_LIMIT)
> + report_corruption(ctx,
> + psprintf("toast value %u rawsize %u exceeds limit %u",
> + toast_pointer.va_valueid,
> + toast_pointer.va_rawsize,
> + VARLENA_SIZE_LIMIT));
> I think the comment sounds wrong since toast is precisely for storage of
> "oversized" attributes.
> https://www.postgresql.org/docs/current/storage-toast.html > | This section provides an overview of TOAST (The Oversized-Attribute Storage Technique).
Thanks for reviewing! Changed to:
+ /* Toasted attributes too large to be untoasted should never be stored */