== PostgreSQL Weekly News - March 29, 2020 ==

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

== PostgreSQL Weekly News - March 29, 2020 ==

David Fetter
== PostgreSQL Weekly News - March 29, 2020 ==

Person of the week: https://postgresql.life/post/dave_cramer/

PGConf.be 2020 has been cancelled.

PGDay Ukraine 2020 - POSTPONED due to COVID-19.  Preliminary date: September
5th, 2020

PGCon will be online and not in person. More details to follow.

PGDay.IT has been cancelled.

== PostgreSQL Product News ==

pgBackRest 2.25, a backup and restore system for PostgreSQL, released.

Benetl 5.0, an ETL tool for postgreSQL, released.

There is now an archive of old apt packages at

WAL-G 0.2.15, a backup management system for PostgreSQL written in Go, released.

pg_show_plans, a module to inspect execution plans in real-time, released.

pgFormatter 4.3, a formatter/beautifier for SQL code, released.

== PostgreSQL Jobs for March ==


== PostgreSQL Local ==

PgConf Belgium will take place on May 8, 2020 at the UCLL Campus in Haasrode
(near Leuven). The CfS is open through March 31, 2020 at https://pgconf.be/#cfs

PGCon 2020 will take place online on May 26-29, 2020.

Postgres Ibiza will be held in Ibiza, Spain on June 25-26, 2020.

PostgresLondon 2020 will be July 7-8, 2020 with an optional training day on
July 6.

PG Day Russia will be in Saint Petersburg on July 10, 2020. The CfP is open at
https://pgday.ru/en/2020/for-speakers through April 6, 2020.

FOSS4G 2020, will take place in Calgary, Alberta, Canada August 24-29 2020.
the Call for Papers is currently open at https://2020.foss4g.org/speakers/

PGDay Ukraine will take place September 5th, 2020 in Lviv at the Bank Hotel.

pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv.

PGDay Austria will take place September 18, 2020 at Schloss Schoenbrunn
(Apothekertrakt) in Vienna. The CfP is open until April 19, 2020 at

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to [hidden email].

== Applied Patches ==

Michaël Paquier pushed:

- Add bound checks for ssl_min_protocol_version and ssl_max_protocol_version.
  Mixing incorrect bounds in the SSL context leads to confusing error messages
  generated by OpenSSL which are hard to act on.  New range checks are added
  when both min/max parameters are loaded in the context of a SSL reload to
  improve the error reporting.  Note that this does not make use of the GUC hook
  machinery contrary to 41aadee, as there is no way to ensure a consistent range
  check (except if there is a way one day to define range types for GUC
  parameters?).  Hence, this patch applies only to OpenSSL, and uses a logic
  similar to other parameters to trigger an error when reloading the SSL context
  in a session.  Author: Michael Paquier Reviewed-by: Daniel Gustafsson
  Discussion: https://postgr.es/m/20200114035420.GE1515@...

- Revert "Refactor compile-time assertion checks in c.h". This reverts commit
  b7f64c6, which broke the fallback implementation for C++.  We have discussed a
  couple of alternatives to reduce the number of implementations for those
  asserts, but nothing allowing to reduce the number of implementations down to
  three instead of four, so there is no benefit in keeping this patch.  Thanks
  to Tom Lane for the discussion.  Discussion:

- Doc: Fix type of some storage parameters in CREATE TABLE page.
  autovacuum_vacuum_scale_factor and autovacuum_analyze_scale_factor have been
  documented as "float4", but "floating type" is used in this case for GUCs and
  relation options in the documentation.  Author: Atsushi Torikoshi Discussion:
  Backpatch-through: 9.5

- Move routine building restore_command to src/common/. restore_command has only
  been used until now by the backend, but there is a pending patch for pg_rewind
  to make use of that in the frontend.  Author: Alexey Kondratov Reviewed-by:
  Andrey Borodin, Andres Freund, Alvaro Herrera, Alexander Korotkov, Michael
  Paquier Discussion:

Amit Kapila pushed:

- Add object names to partition integrity violations. All errors of SQLSTATE
  class 23 should include the name of an object associated with the error in
  separate fields of the error report message. We do this so that applications
  need not try to extract them from the possibly-localized human-readable text
  of the message.  Reported-by: Chris Bandy Author: Chris Bandy Reviewed-by:
  Amit Kapila and Amit Langote Discussion:

Tom Lane pushed:

- Fix our getopt_long's behavior for a command line argument of just "-".
  src/port/getopt_long.c failed on such an argument, always seeing it as an
  unrecognized switch.  This is unhelpful; better is to treat such an item as a
  non-switch argument.  That behavior is what we find in GNU's getopt_long();
  it's what src/port/getopt.c does; and it is required by POSIX for getopt(),
  which getopt_long() ought to be generally a superset of.  Moreover, it's
  expected by ecpg, which intends an argument of "-" to mean "read from stdin".
  So fix it.  Also add some documentation about ecpg's behavior in this area,
  since that was miserably underdocumented.  I had to reverse-engineer it from
  the code.  Per bug #16304 from James Gray.  Back-patch to all supported
  branches, since this has been broken forever.  Discussion:

- Doc: explain that LIKE et al can be used in ANY (sub-select) etc. This wasn't
  stated anywhere, and it's perhaps not that obvious, since we get questions
  about it from time to time.  Also undocumented was that the parser actually
  translates these into operators.  Discussion:

- Doc: fix broken markup. Sloppiness in commit cedffbdb8, noted by Erikjan
  Rijkers. (It's fairly unfortunate that xmllint doesn't catch this.)
  Discussion: https://postgr.es/m/2e3dc9e4bfa4802d2c9f5fe15bde44de@...

- Re-implement the ereport() macro using __VA_ARGS__. Now that we require C99,
  we can depend on __VA_ARGS__ to work, and revising ereport() to use it has
  several significant benefits:  * The extra parentheses around the auxiliary
  function calls are now optional.  Aside from being a bit less ugly, this
  removes a common gotcha for new contributors, because in some cases the
  compiler errors you got from forgetting them were unintelligible.  * The
  auxiliary function calls are now evaluated as a comma expression list rather
  than as extra arguments to errfinish().  This means that compilers can be
  expected to warn about no-op expressions in the list, allowing detection of
  several other common mistakes such as forgetting to add errmsg(...) when
  converting an elog() call to ereport().  * Unlike the situation with extra
  function arguments, comma expressions are guaranteed to be evaluated
  left-to-right, so this removes platform dependency in the order of the
  auxiliary function calls.  While that dependency hasn't caused us big problems
  in the past, this change does allow dropping some rather shaky assumptions
  around errcontext() domain handling.  There's no intention to make wholesale
  changes of existing ereport calls, but as proof-of-concept this patch removes
  the extra parens from a couple of calls in postgres.c.  While new code can be
  written either way, code intended to be back-patched will need to use extra
  parens for awhile yet.  It seems worth back-patching this change into v12, so
  as to reduce the window where we have to be careful about that by one year.
  Hence, this patch is careful to preserve ABI compatibility; a followup
  HEAD-only patch will make some additional simplifications.  Andres Freund and
  Tom Lane  Discussion:

- Improve the internal implementation of ereport(). Change all the auxiliary
  error-reporting routines to return void, now that we no longer need to pretend
  they are passing something useful to errfinish().  While this probably doesn't
  save anything significant at the machine-code level, it allows detection of
  some additional types of mistakes.  Pass the error location details (__FILE__,
  __LINE__, PG_FUNCNAME_MACRO) to errfinish not errstart.  This shaves a few
  cycles off the case where errstart decides we're not going to emit anything.
  Re-implement elog() as a trivial wrapper around ereport(), removing the
  separate support infrastructure it used to have.  Aside from getting rid of
  some now-surplus code, this means that elog() now really does have exactly the
  same semantics as ereport(), in particular that it can skip evaluation work if
  the message is not to be emitted.  Andres Freund and Tom Lane  Discussion:

- Go back to returning int from ereport auxiliary functions. This reverts the
  parts of commit 17a28b03645e27d73bf69a95d7569b61e58f06eb that changed
  ereport's auxiliary functions from returning dummy integer values to returning
  void.  It turns out that a minority of compilers complain (not entirely
  unreasonably) about constructs such as   (condition) ? errdetail(...) : 0  if
  errdetail() returns void rather than int.  We could update those call sites to
  say "(void) 0" perhaps, but the expectation for this patch set was that
  ereport callers would not have to change anything. And this aspect of the
  patch set was already the most invasive and least compelling part of it, so
  let's just drop it.  Per buildfarm.  Discussion:

- Fix assorted portability issues in commit 896fcdb23. Some platforms require
  libssl to be linked explicitly in the new SSL test module.  Borrow
  contrib/sslinfo's code for that.  Since src/test/modules/Makefile now has a
  variable SUBDIRS list, it needs to follow the ALWAYS_SUBDIRS protocol for that
  (cf. comments in Makefile.global.in).  Blindly try to fix MSVC build failures
  by adding PGDLLIMPORT.

- Further fixes for ssl_passphrase_callback test module. The Makefile should set
  TAP_TESTS = 1, not implement the infrastructure for itself.  For one thing, it
  missed the appropriate "make clean" steps.  For another, the buildfarm isn't
  running this test because it wasn't hooked into "make installcheck" either.

- Ensure that plpgsql cleans up cleanly during parallel-worker exit.
  plpgsql_xact_cb ought to treat events XACT_EVENT_PARALLEL_COMMIT and
  respectively, since its goal is to do process-local cleanup.  This oversight
  caused plpgsql's end-of-transaction cleanup to not get done in parallel
  workers.  Since a parallel worker will exit just after the transaction
  cleanup, the effects of this are limited.  I couldn't find any case in the
  core code with user-visible effects, but perhaps there are some in extensions.
  In any case it's wrong, so let's fix it before it bites us not after.  In
  passing, add some comments around the handling of expression evaluation
  resources in DO blocks.  There's no live bug there, but it's quite unobvious
  what's happening; at least I thought so.  This isn't related to the other
  issue, except that I found both things while poking at expression-evaluation
  performance.  Back-patch the plpgsql_xact_cb fix to 9.5 where those event
  types were introduced, and the DO-block commentary to v11 where DO blocks
  gained the ability to issue COMMIT/ROLLBACK.  Discussion:

- Improve performance of "simple expressions" in PL/pgSQL. For relatively simple
  expressions (say, "x + 1" or "x > 0"), plpgsql's management overhead exceeds
  the cost of evaluating the expression. This patch substantially improves that
  situation, providing roughly 2X speedup for such trivial expressions.  First,
  add infrastructure in the plancache to allow fast re-validation of cached
  plans that contain no table access, and hence need no locks. Teach plpgsql to
  use this infrastructure for expressions that it's already deemed "simple"
  (which in particular will never contain table references).  The fast path
  still requires checking that search_path hasn't changed, so provide a fast
  path for OverrideSearchPathMatchesCurrent by counting changes that have
  occurred to the active search path in the current session.  This is simplistic
  but seems enough for now, seeing that PushOverrideSearchPath is not used in
  any performance-critical cases.  Second, manage the refcounts on simple
  expressions' cached plans using a transaction-lifespan resource owner, so that
  we only need to take and release an expression's refcount once per transaction
  not once per expression evaluation.  The management of this resource owner
  exactly parallels the existing management of plpgsql's simple-expression
  EState.  Add some regression tests covering this area, in particular verifying
  that expression caching doesn't break semantics for search_path changes.
  Patch by me, but it owes something to previous work by Amit Langote, who
  recognized that getting rid of plancache-related overhead would be a useful
  thing to do here.  Also thanks to Andres Freund for review.  Discussion:

- Rearrange validity checks for plpgsql "simple" expressions. Buildfarm
  experience shows what probably should've occurred to me before: if a cache
  flush occurs partway through building a generic plan, then the plansource may
  have is_valid = false even though the plan is valid. We need to accept this
  case, use the generated plan, and then try to replan the next time.  We can't
  try to replan immediately, because that would produce an infinite loop in
  CLOBBER_CACHE_ALWAYS builds; moreover it's really overkill.  (We can assume
  that the plan is valid, it's just possibly a bit stale.  Note that the
  pre-existing code behaved this way, and the non-simple-expression code paths
  do too.)  Conversely, not using the generated plan would drop us into the
  not-a-simple-expression code path, which is bad for performance and would also
  cause regression-test failures due to visibly different error-reporting
  behavior.  Hence, refactor the validity-check functions so that the initial
  check and recheck cases can react differently to plansource->is_valid. This
  makes their usage a bit simpler, too.  Discussion:

- Protect against overflow of ltree.numlevel and lquery.numlevel. These uint16
  fields could be overflowed by excessively long input, producing strange
  results.  Complain for invalid input.  Likewise check for out-of-range values
  of the repeat counts in lquery. (We don't try too hard on that one, notably
  not bothering to detect if atoi's result has overflowed.)  Also detect length
  overflow in ltree_concat.  In passing, be more consistent about whether
  "syntax error" messages include the type name.  Also, clarify the
  documentation about what the size limit is.  This has been broken for a long
  time, so back-patch to all supported branches.  Nikita Glukhov, reviewed by
  Benjie Gillam and Tomas Vondra  Discussion:

- Fix lquery's behavior for consecutive '*' items. Something like "*{2}.*{3}"
  should presumably mean the same as "*{5}", but it didn't.  Improve that.  Get
  rid of an undocumented and remarkably ugly (though not, as far as I can tell,
  actually unsafe) static variable in favor of passing more arguments to
  checkCond().  Reverse-engineer some commentary.  This function, like all of
  ltree, is still far short of what I would consider the minimum acceptable
  level of internal documentation, but at least now it has more than zero
  comments.  Although this certainly seems like a bug fix, people might not
  thank us for changing query behavior in stable branches, so no back-patch.
  Nikita Glukhov, with cosmetic improvements by me  Discussion:

Alexander Korotkov pushed:

- Fix ordering in typedefs.list.

Andres Freund pushed:

- Add regression tests for constraint errors in partitioned tables. While #16293
  only applied to 11 (and 10 to some degree), it seems best to add tests to all
  branches with partitioning support.  Reported-By: Daniel WM Author: Andres
  Freund Bug: #16293 Discussion:
  https://postgr.es/m/16293-26f5777d10143a66@... Backpatch: 10-

- Report wait event for cost-based vacuum delay. Author: Justin Pryzby
  Discussion: https://postgr.es/m/20200321040750.GD13662@...

- logical decoding: Remove TODO about unnecessary optimization. Measurements
  show, and intuition agrees, that there's currently no known cases where adding
  a fastpath to avoid allocating / ordering a heap for a single transaction is
  worthwhile.  Author: Dilip Kumar Discussion:

- Ensure snapshot is registered within ScanPgRelation(). In 9.4 I added support
  to use a historical snapshot in ScanPgRelation(), while adding logical
  decoding. Unfortunately a conflict with the concurrent removal of SnapshotNow
  was incorrectly resolved, leading to an unregistered snapshot being used.  It
  is not correct to use an unregistered (or non-active) snapshot for anything
  non-trivial, because catalog invalidations can cause the snapshot to be
  invalidated.  Luckily it seems unlikely to actively cause problems in
  practice, as ScanPgRelation() requires that we already have a lock on the
  relation, we only look for a single row, and we don't appear to rely on the
  result's tid to be correct. It however is clearly wrong and potential negative
  consequences would likely be hard to find. So it seems worth backpatching the
  fix, even without a concrete hazard.  Discussion:
  Backpatch: 9.5-

Bruce Momjian pushed:

- docs:  add backend_type to file-fdw CSV log example. backend_type was added to
  the CVS log output in commit 70a7b4776b.  Reported-by: Fabrízio de Royes Mello
  Backpatch-through: master

Jeff Davis pushed:

- Fixes for Disk-based Hash Aggregation. Justin Pryzby raised a couple issues
  with commit 1f39bce0. Fixed.  Also, tweak the way the size of a hash entry is
  estimated and the number of buckets is estimated when calling
  BuildTupleHashTableExt().  Discussion:

- Reduce test time for disk-based Hash Aggregation. Discussion:

- Consider disk-based hash aggregation to implement DISTINCT. Correct oversight
  in 1f39bce0. If enable_hashagg_disk=true, we should consider hash aggregation
  for DISTINCT when applicable.

- Avoid allocating unnecessary zero-sized array. If there are no aggregates,
  there is no need to allocate an array of zero AggStatePerGroupData elements.

- Fix costing for disk-based hash aggregation. Report and suggestions from
  Richard Guo and Tomas Vondra.  Discussion:

Fujii Masao pushed:

- Report NULL as total backup size if it's not estimated. Previously 0 was
  reported in pg_stat_progress_basebackup.total_backup if the total backup size
  was not estimated. Per discussion, our consensus is that NULL is better choise
  as the value in total_backup in that case. So this commit makes
  pg_stat_progress_basebackup view report NULL in total_backup column if the
  estimation is disabled.  Bump catversion.  Author: Fujii Masao Reviewed-by:
  Amit Langote, Magnus Hagander, Alvaro Herrera Discussion:

- Add wait events for WAL archive and recovery pause. This commit introduces new
  wait events BackupWaitWalArchive and RecoveryPause. The former is reported
  while waiting for the WAL files required for the backup to be successfully
  archived. The latter is reported while waiting for recovery in pause state to
  be resumed.  Author: Fujii Masao Reviewed-by: Michael Paquier, Atsushi
  Torikoshi, Robert Haas Discussion:

- Prefer standby promotion over recovery pause. Previously if a promotion was
  triggered while recovery was paused, the paused state continued. Also recovery
  could be paused by executing pg_wal_replay_pause() even while a promotion was
  ongoing. That is, recovery pause had higher priority over a standby promotion.
  But this behavior was not desirable because most users basically wanted the
  recovery to complete as soon as possible and the server to become the master
  when they requested a promotion.  This commit changes recovery so that it
  prefers a promotion over recovery pause. That is, if a promotion is triggered
  while recovery is paused, the paused state ends and a promotion continues.
  Also this commit makes recovery pause functions like pg_wal_replay_pause()
  throw an error if they are executed while a promotion is ongoing.  Internally,
  this commit adds new internal function PromoteIsTriggered() that returns true
  if a promotion is triggered. Since the name of this function and the existing
  function IsPromoteTriggered() are confusingly similar, the commit changes the
  name of IsPromoteTriggered() to IsPromoteSignaled, as more appropriate name.
  Author: Fujii Masao Reviewed-by: Atsushi Torikoshi, Sergei Kornilov

Peter Eisentraut pushed:

- Add support for other normal forms to Unicode normalization API. It previously
  only supported NFKC, for use by SASLprep.  This expands the API to offer the
  choice of all four normalization forms.  Right now, there are no internal
  users of the forms other than NFKC.  Reviewed-by: Daniel Verite
  <[hidden email]> Reviewed-by: Andreas Karlsson <[hidden email]>

- Some refactoring of logical/worker.c. This moves the main operations of
  apply_handle_{insert|update|delete}, that of inserting, updating, deleting a
  tuple into/from a given relation, into corresponding
  apply_handle_{insert|update|delete}_internal functions.  This allows
  performing those operations on relations that are not directly the targets of
  replication, which is something a later patch will use for targeting
  partitioned tables.  Author: Amit Langote <[hidden email]>
  Reviewed-by: Rafia Sabih <[hidden email]> Reviewed-by: Peter
  Eisentraut <[hidden email]> Discussion:

- Fix compiler warning. A variable was unused in non-assert builds.  Simplify
  the code to avoid the issue.  Reported-by: Erik Rijkers <[hidden email]>

- Fix compiler warning on Cygwin. bf68b79e50e3359accc85c94fa23cc03abb9350a
  introduced an unused variable compiler warning on Cygwin.

- Update SQL features. The name of E182 was changed in SQL:2011.  Also, we can
  change it to supported because all it requires is one embedded language to be
  supported, which we do.

- Define EXEC_BACKEND in pg_config_manual.h. It was for unclear reasons defined
  in a separate location, which makes it more cumbersome to override for
  testing, and it also did not have any prominent documentation.  Move to
  pg_config_manual.h, where similar things are already collected.  The previous
  definition on the command-line had the effect of defining it to the value 1,
  but now that we don't need that anymore we just define it to empty, to
  simplify manual editing a bit.  Reviewed-by: Tom Lane <[hidden email]>

- Drop slot's LWLock before returning from SaveSlotToPath(). When
  SaveSlotToPath() is called with elevel=LOG, the early exits didn't release the
  slot's io_in_progress_lock.  This could result in a walsender being stuck on
  the lock forever.  A possible way to get into this situation is if the
  offending code paths are triggered in a low disk space situation.  Author:
  Pavan Deolasee <[hidden email]> Reported-by: Craig Ringer
  <[hidden email]> Discussion:

- Update SQL features. Change F311 to supported.  This was already accomplished
  when subfeature F311-04 (WITH CHECK OPTION) was added, but the top-level
  feature wasn't updated at the time.

- Cleanup in SQL features files. Feature C011 was still listed in
  sql_feature_packages.txt but had been removed from sql_features.txt, so also
  remove from the former.

- Enable Unix-domain sockets support on Windows. As of Windows 10 version 1803,
  Unix-domain sockets are supported on Windows.  But it's not automatically
  detected by configure because it looks for struct sockaddr_un and Windows
  doesn't define that.  So we just make our own definition on Windows and
  override the configure result.  Set DEFAULT_PGSOCKET_DIR to empty on Windows
  so by default no Unix-domain socket is used, because there is no good standard
  location.  In pg_upgrade, we have to do some extra tweaking to preserve the
  existing behavior of not using Unix-domain sockets on Windows.  Adding support
  would be desirable, but it needs further work, in particular a way to select
  whether to use Unix-domain sockets from the command-line or with a run-time
  test.  The pg_upgrade test script needs a fix.  The previous code passed
  "localhost" to postgres -k, which only happened to work because Windows used
  to ignore the -k argument value altogether.  We instead need to pass an empty
  string to get the desired effect.  The test suites will continue to not use
  Unix-domain sockets on Windows.  This requires a small tweak in pg_regress.c.
  The TAP tests don't need to be changed because they decide by the operating
  system rather than HAVE_UNIX_SOCKETS.  Reviewed-by: Andrew Dunstan
  <[hidden email]> Discussion:

- Update SQL features. Change F181 to supported.  It requires that an embedded C
  program can be split across multiple files, which ECPG easily supports.

- pg_regress: Observe TMPDIR. Put the temporary socket directory under TMPDIR,
  if that environment variable is set, instead of the hardcoded /tmp.  This
  allows running the tests if there is no /tmp at all (for example on Windows,
  although running the tests with Unix-domain sockets is not enabled on Windows
  yet).  We also use TMPDIR everywhere else /tmp is hardcoded, so this makes the
  behavior consistent.  Reviewed-by: Andrew Dunstan
  <[hidden email]> Discussion:

- Document color support. Add a documentation appendix that explains the
  PG_COLOR and PG_COLORS environment variables.  Discussion:

Peter Geoghegan pushed:

- Fix nbtree deduplication README commentary. Descriptions of some aspects of
  how deduplication works were unclear in a couple of places.

- Justify nbtree page split locking in code comment. Delaying unlocking the
  right child page until after the point that the left child's parent page has
  been refound is no longer truly necessary. Commit 40dae7ec made nbtree
  tolerant of interrupted page splits.  VACUUM was taught to avoid deleting a
  page that happens to be the right half of an incomplete split.  As long as
  page splits don't unlock the left child page until the end of the second/final
  phase, it should be safe to unlock the right child page earlier (at the end of
  the first phase).  It probably isn't actually useful to release the right
  child's lock earlier like this (it probably won't improve performance).  Even
  still, pointing out that it ought to be safe to do so should make it easier to
  understand the overall design.

- Make deduplication use number of key attributes. Use
  IndexRelationGetNumberOfKeyAttributes() rather than
  IndexRelationGetNumberOfAttributes() when determining whether or not two index
  tuples are suitable for merging together into a single posting list tuple.
  This is a little bit tidier.  It brings affected code in nbtdedup.c a little
  closer to similar, related code in nbtsplitloc.c.

Thomas Munro pushed:

- Allow NULL version for individual collations. Remove the documented
  restriction that collation providers must either return NULL for all
  collations or non-NULL for all collations.  Use NULL for glibc collations like
  "C.UTF-8", which might otherwise lead future proposed commits to force
  unnecessary index rebuilds.  Reviewed-by: Peter Eisentraut
  <[hidden email]> Discussion:

- Add collation versions for Windows. On Vista and later, use GetNLSVersionEx()
  to request collation version information.  Reviewed-by: Juan José Santamaría
  Flecha <[hidden email]> Discussion:

Álvaro Herrera pushed:

- pg_dump: Allow dumping data of specific foreign servers. The new command-line
  switch --include-foreign-data=PATTERN lets the user specify foreign servers
  from which to dump foreign table data.  This can be refined by further
  inclusion/exclusion switches, so that the user has full control over which
  tables to dump.  A limitation is that this doesn't work in combination with
  parallel dumps, for implementation reasons.  This might be lifted in the
  future, but requires shuffling some code around.  Author: Luis Carril
  <[hidden email]> Reviewed-by: Daniel Gustafsson <[hidden email]>
  Reviewed-by: Surafel Temesgen <[hidden email]> Reviewed-by: vignesh C
  <[hidden email]> Reviewed-by: Álvaro Herrera <[hidden email]>

- pg_dump new test: Change order of arguments. Some getopt_long implementations
  don't like to have a non-option argument before option arguments, so put the
  database name as the last switch.  Per buildfarm member hoverfly.

- Set wal_receiver_create_temp_slot PGC_POSTMASTER. Commit 329730827848 gave
  walreceiver the ability to create and use a temporary replication slot, and
  made it controllable by a GUC (enabled by default) that can be changed with
  SIGHUP.  That's useful but has two problems: one, it's possible to cause the
  origin server to fill its disk if the slot doesn't advance in time; and also
  there's a disconnect between state passed down via the startup process and
  GUCs that walreceiver reads directly.  We handle the first problem by setting
  the option to disabled by default.  If the user enables it, its on their head
  to make sure that disk doesn't fill up.  We handle the second problem by
  passing the flag via startup rather than having walreceiver acquire it
  directly, and making it PGC_POSTMASTER (which ensures a walreceiver always has
  the fresh value).  A future commit can relax this (to PGC_SIGHUP again) by
  having the startup process signal walreceiver to shutdown whenever the value
  changes.  Author: Sergei Kornilov <[hidden email]> Reviewed-by: Michael Paquier
  <[hidden email]> Reviewed-by: Álvaro Herrera <[hidden email]>
  Discussion: https://postgr.es/m/20200122055510.GH174860@...

- Allow walreceiver configuration to change on reload. The parameters
  primary_conninfo, primary_slot_name and wal_receiver_create_temp_slot can now
  be changed with a simple "reload" signal, no longer requiring a server
  restart.  This is achieved by signalling the walreceiver process to terminate
  and having it start again with the new values.  Thanks to Andres Freund,
  Kyotaro Horiguchi, Fujii Masao for discussion.  Author: Sergei Kornilov
  <[hidden email]> Reviewed-by: Michael Paquier <[hidden email]> Reviewed-by:
  Álvaro Herrera <[hidden email]> Discussion:

Andrew Dunstan pushed:

- Provide a TLS init hook. The default hook function sets the default password
  callback function. In order to allow preloaded libraries to have an
  opportunity to override the default, TLS initialization if now delayed
  slightly until after shared preloaded libraries have been loaded.  A test
  module is provided which contains a trivial example that decodes an obfuscated
  password for an SSL certificate.  Author: Andrew Dunstan Reviewed By: Andreas
  Karlsson, Asaba Takanori Discussion:

- Don't listen to localhost in ssl_passphrase_callback test. Commit 896fcdb230
  contained an unnecessary setting that listened to localhost. Since the test
  doesn't actually try to make an SSL connection to the database this isn't
  required. Moreover, it's a security hole.  Per gripe from Tom Lane.

Magnus Hagander pushed:

- Document that pg_checksums exists in checksums README. Author: Daniel
  Gustafsson <[hidden email]>

David Rowley pushed:

- Trigger autovacuum based on number of INSERTs. Traditionally autovacuum has
  only ever invoked a worker based on the estimated number of dead tuples in a
  table and for anti-wraparound purposes. For the latter, with certain classes
  of tables such as insert-only tables, anti-wraparound vacuums could be the
  first vacuum that the table ever receives. This could often lead to autovacuum
  workers being busy for extended periods of time due to having to potentially
  freeze every page in the table. This could be particularly bad for very large
  tables. New clusters, or recently pg_restored clusters could suffer even more
  as many large tables may have the same relfrozenxid, which could result in
  large numbers of tables requiring an anti-wraparound vacuum all at once.  Here
  we aim to reduce the work required by anti-wraparound and aggressive vacuums
  in general, by triggering autovacuum when the table has received enough
  INSERTs. This is controlled by adding two new GUCs and reloptions;
  autovacuum_vacuum_insert_threshold and autovacuum_vacuum_insert_scale_factor.
  These work exactly the same as the existing scale factor and threshold
  controls, only base themselves off the number of inserts since the last
  vacuum, rather than the number of dead tuples. New controls were added rather
  than reusing the existing controls, to allow these new vacuums to be tuned
  independently and perhaps even completely disabled altogether, which can be
  done by setting autovacuum_vacuum_insert_threshold to -1.  We make no attempt
  to skip index cleanup operations on these vacuums as they may trigger for an
  insert-mostly table which continually doesn't have enough dead tuples to
  trigger an autovacuum for the purpose of removing those dead tuples. If we
  were to skip cleaning the indexes in this case, then it is possible for the
  index(es) to become bloated over time.  There are additional benefits to
  triggering autovacuums based on inserts, as tables which never contain enough
  dead tuples to trigger an autovacuum are now more likely to receive a vacuum,
  which can mark more of the table as "allvisible" and encourage the query
  planner to make use of Index Only Scans.  Currently, we still obey
  vacuum_freeze_min_age when triggering these new autovacuums based on INSERTs.
  For large insert-only tables, it may be beneficial to lower the table's
  autovacuum_freeze_min_age so that tuples are eligible to be frozen sooner.
  Here we've opted not to zero that for these types of vacuums, since the table
  may just be insert-mostly and we may otherwise freeze tuples that are still
  destined to be updated or removed in the near future.  There was some debate
  to what exactly the new scale factor and threshold should default to. For now,
  these are set to 0.2 and 1000, respectively. There may be some motivation to
  adjust these before the release.  Author: Laurenz Albe, Darafei Praliaskouski
  Reviewed-by: Alvaro Herrera, Masahiko Sawada, Chris Travers, Andres Freund,
  Justin Pryzby Discussion:

- Attempt to fix unstable regression tests. b07642dbc added code to trigger
  autovacuums based on the number of inserts into a table. This seems to have
  caused some regression test results to destabilize. I suspect this is due to
  autovacuum triggering a vacuum sometime after the test's ANALYZE run and
  perhaps reltuples is ending up being set to a slightly different value as a
  result.  Attempt to resolve this by running a VACUUM ANALYZE on the affected
  table instead of just ANALYZE. pg_class.reltuples will still get set to
  whatever ANALYZE chooses but we should no longer get the proceeding autovacuum
  overriding that.  The overhead this adds to each test's runtime seems small
  enough not to worry about. I measure 3-4% on stats_ext and can't measure any
  change in partition_aggregate.  I'm unable to recreate the issue locally, so
  this is a bit of a blind fix.  Discussion:

Dean Rasheed pushed:

- Prevent functional dependency estimates from exceeding column estimates.
  Formerly we applied a functional dependency "a => b with dependency degree f"
  using the formula    P(a,b) = P(a) * [f + (1-f)*P(b)]  This leads to the
  possibility that the combined selectivity P(a,b) could exceed P(b), which is
  not ideal. The addition of support for IN and OR clauses (commits 8f321bd16c
  and ccaa3569f5) would seem to make this more likely, since the user-supplied
  values in such clauses are not necessarily compatible with the functional
  dependency.  Mitigate this by using the formula    P(a,b) = f * Min(P(a),
  P(b)) + (1-f) * P(a) * P(b)  instead, which guarantees that the combined
  selectivity is less than each column's individual selectivity. Logically, this
  is modifies the part of the formula that accounts for dependent rows to handle
  cases where P(a) > P(b), whilst not changing the second term which accounts
  for independent rows.  Additionally, this refactors the way that functional
  dependencies are applied, so now dependencies_clauselist_selectivity()
  estimates both the implying clauses and the implied clauses for each
  functional dependency (formerly only the implied clauses were estimated), and
  now all clauses for each attribute are taken into account (formerly only one
  clause for each implied attribute was estimated). This removes the previously
  built-in assumption that only equality clauses will be seen, which is no
  longer true, and opens up the possibility of applying functional dependencies
  to more general clauses.  Patch by me, reviewed by Tomas Vondra.  Discussion:
  Discussion: https://postgr.es/m/20200318002946.6dvblukm3cfmgir2%40development

- Improve the performance and accuracy of numeric sqrt() and ln(). Instead of
  using Newton's method to compute numeric square roots, use the Karatsuba
  square root algorithm, which performs better for numbers of all sizes. In
  practice, this is 3-5 times faster for inputs with just a few digits and up to
  around 10 times faster for larger inputs.  Also, the new algorithm guarantees
  that the final digit of the result is correctly rounded, since it computes an
  integer square root with truncation, containing at least 1 extra decimal digit
  before rounding. The former algorithm would occasionally round the wrong way
  because it rounded both the intermediate and final results.  In addition,
  arrange for sqrt_var() to explicitly support negative rscale values (rounding
  before the decimal point). This allows the argument reduction phase of
  ln_var() to be optimised for large inputs, since it only needs to compute
  square roots with a few more digits than the final ln() result, rather than
  computing all the digits before the decimal point. For very large inputs, this
  can be many thousands of times faster.  In passing, optimise div_var_fast() in
  a couple of places where it was doing unnecessary work.  Patch be me, reviewed
  by Tom Lane and Tels.  Discussion:

== Pending Patches ==

Pavel Stěhule sent in another revision of a patch to implement generic type

Kyotaro HORIGUCHI sent in another revision of a patch to fix the WAL-skipping

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON

Nikita Glukhov sent in another revision of a patch to implement JSON TABLE.

Daniel Gustafsson sent in a patch to qualify some casts in pg_dump.

Teja Mupparti sent in a patch to fix a bug that caused corruption during WAL

Thomas Munro sent in another revision of a patch to allow NULL version for
individual collations, and add collation versions for Windows.

Amit Langote sent in two more revisions of a patch to enable adding partitioned
tables to publications.

Peter Eisentraut sent in a patch to add some AppVeyor files.

Kyotaro HORIGUCHI sent in another revision of a patch to make the message at
end-of-recovery less scary.

Robert Haas sent in two more revisions of a patch to implement backup manifests.

Justin Pryzby and Amit Kapila traded patches to add block number to the error
context for vacuum.

Andy Fan sent in four more revisions of a patch to maintain UniqueKey at each

Álvaro Herrera sent in another revision of a patch to implement multiranges.

Fujii Masao sent in another revision of a patch to fix a confusing hint in

Anastasia Lubennikova sent in a patch to add a test for a corner case in b-tree

Kyotaro HORIGUCHI sent in another revision of a patch to remove page-read
callback from XLogReaderState.

Konstantin Knizhnik sent in another revision of a patch to implement a built-in
connection pooler.

Dilip Kumar sent in a patch to remove the TODO comments for fast-path.

Dmitry Dolgov sent in two more revisions of a patch to implement index skip

John Naylor sent in another revision of a patch to truncate timestamps on
arbitrary intervals.

Anastasia Lubennikova sent in a patch to add a test to ensure that pg_upgrade
doesn't fail with non-standard ACL.

Ibrar Ahmed sent in another revision of a patch to ensure that COPY FREEZE
actually freezes right.

Etsuro Fujita and Tomáš Vondra traded patches to improve the partition matching
algorithm for partition-wise join.

Tom Lane sent in another revision of a patch to rewrite NOT IN to anti-JOIN in
cases where that's correct.

Pavel Stěhule sent in two more revisions of a patch to implement \gfmt in psql.

James Coleman sent in six more revisions of a patch to implement incremental

Kirk Jamison sent in another revision of a patch to optimize the dropping of
relation buffers using dlist.

Konstantin Knizhnik sent in a patch to make a small computeRegionDelta

John Naylor sent in another revision of a patch to de-TOAST using an iterator.

Konstantin Knizhnik sent in a patch to add some hash join instrumentation.

Pengzhou Tang sent in another revision of a patch to implement parallel grouping

Dean Rasheed sent in another revision of a patch to improve the computation of
sqrt for NUMERIC.

Fabien COELHO sent in two revisions of a patch to allow line continuations in

Jeff Davis sent in two revisions of a patch to implement
AllocSetEstimateChunkSpace() and use same to improve the estimate for the size
of a hash entry for hash aggregation.

Konstantin Knizhnik sent in three more revisions of a patch to add GUCs
auto_explain.add_statistics_threshold and auto_explain.suggest_only.

Cary Huang sent in a patch to include sequence relation support in logical

Zeng Wenjing sent in two more revisions of a patch to implement global temporary

Masahiko Sawada sent in another revision of a patch to implement an internal key
management system.

Guancheng Luo sent in two revisions of a patch to check the compatibility of
equality operator between partition key's opclass and index column's opclass
when creating unique indexes on partitioned tables.

Dave Sharpe sent in a patch to fix a bug in the FDW infrastructure where
XactCallback is not always called for a successful client ROLLBACK statement.

Gareth Palmer sent in another revision of a patch to implement INSERT ... SET.

Vik Fearing sent in a patch to add tab completion for \gx to psql.

Nathan Bossart sent in another revision of a patch to avoid marking WAL segments
as ready-for-archive too early.

Nikita Glukhov sent in another revision of a patch to improve the ltree syntax.

Michaël Paquier and Fabien COELHO traded patches to pgbench to add \aset to
store results of a combined query.

Peter Eisentraut and John Naylor traded patches to add SQL functions for Unicode

Surafel Temesgen sent in another revision of a patch to add conflict handling to

Justin Pryzby and Aleksey Kondratov traded patches to allow CLUSTER, VACUUM FULL
and REINDEX to change tablespace on the fly.

Álvaro Herrera and Sergei Kornilov traded patches to enable online changes to

Mike Palmiotto sent in another revision of a patch to add and use an auxiliary
process infrastructure.

Ivan Kartyshov sent in another revision of a patch to make async replicas wait
for lsn to be replayed.

Haozhou Wang sent in another revision of a patch to add smgr hooks to extend the
logic of storage management, one use for which would be disk quotas.

Muhammad Usama sent in another revision of a patch to fix transactions involving
multiple postgres foreign servers.

Kyotaro HORIGUCHI sent in another revision of a patch to implement a
shared-memory based stats collector.

Peter Eisentraut sent in another revision of a patch to fix and document INSERT

Asif Rehman sent in another revision of a patch to implement parallel backup.

Fabien COELHO sent in another revision of a patch to pgbench to rework variable

Fabien COELHO sent in a patch to document the fact that psql's FETCH_COUNT does
not work with multi-command queries.

Tomáš Vondra sent in two more revisions of a patch to collect SLRU statistics.

Nikita Glukhov and Tom Lane traded patches to fix some wrong results from

Fabien COELHO sent in another revision of a patch to pgbench to refactor init
functions with buffers.

Julien Rouhaud and Fujii Masao traded patches to pass the query string to the
planner, expose BufferUsageAccumDiff, and add planning counters to

Jeff Davis sent in another revision of a patch to add a MemoryContextCount()
function, add a new statistic for "new space", and eliminate the specialization
of the memory context printing.

Movead Li sent in another revision of a patch to fix a bug that produced wrong
results when get_bit() function was called on a long bytea string.

Sergei Kornilov sent in another revision of a patch to make it possible to
change restore_command with a HUP signal.

Julien Rouhaud sent in another revision of a patch to add a pg_check_relation()

Ranier Vilela sent in a patch to fix some places where variables are assigned
values that are never used.

Alexander Korotkov sent in another revision of a patch to implement operator
class parameters.

Nikolay Shaplov sent in another revision of a patch to split StdRdOptions into
HeapOptions and ToastOptions.

Andres Freund sent in a patch to fix some infelicities among Catalog
invalidations, catalog scans, and ScanPgRelation().

Ranier Vilela sent in a patch to remove some redundant initializations of

Peter Geoghegan sent in a patch to consistently truncate non-key suffix columns.

David Rowley sent in a patch to implement telemetry checks for unstable tests.

Julien Rouhaud sent in another revision of a patch to calculate WAL usage.

Matheus de Oliveira sent in another revision of a patch to add support for ON

Tom Lane sent in a patch to ignore concurrent removals in pg_ls_dir_files.

Juan José Santamaría Flecha and Tom Lane traded patches to fix win32 langinfo
uft8 detection.