Announcing Release 6 of PostgreSQL Buildfarm client

PostgreSQL Buildfarm client Release 6 is now available and can be
downloaded from
<https://buildfarm.postgresql.org/downloads/releases/build-farm-6.tgz>

Features

  * use a persistent cache for configure, leading to a substantial
    performance improvement for that step, based on an idea from Tom Lane
  * improve installcheck security, mirroring earlier changes on this in
    Postgres itself
  * allow skipping individual TAP tests, e,g, "--skip_steps=pg_dump-check"
  * a use_installcheck_parallel setting that uses the parallel schedule
    rather than the serial schedule for the installcheck step (not for MSVC)
  * use_valgrind and valgrind_options settings that will cause the
    installed postmaster to be run under valgrind and report any output.
    This doesn't apply to the check step, TAP tests, or any other steps
    not using the postmaster started by the start_db step.
  * change the default git target for new repos to use https rather than
    git protocol
  * get PROVE_FLAGS from the environment (can be set in config file)ok,
    allowing the overriding of the buildfarm default
  * include Redis_FDW and Blackhole_FDW example modules, not enabled by
    default.
  * new --schedule switch allowing the specification of an alternative
    schedule file for the installcheck stage
  * new --tests switch allowing for running a specified set of tests
    rather than a schedule file at the installcheck stage
  * adjust for recent changes in locations of various files such as logs
    and regression outputs
  * include application name in default log_line_prefix

There have also been a number of fairly minor bug fixes and changes, and
some fairly significant code reorganization and refactoring.

Some manual changes are advised.

First, buildfarm owners are advised to change their config files to add
%q%a to the log_line_prefix setting, if they haven't done so. There is
an example in build-farm.conf.sample:

    extra_config =>{
            DEFAULT => [
                q(log_line_prefix = '%m [%p:%l] %q%a '),
                "log_connections = 'true'",
                "log_disconnections = 'true'",
                "log_statement = 'all'",
                "fsync = off"
            ],
    },

Also, if you have existing repos using the git protocol, you can change
them to use https by changing the appropriate line in the repo's config
file, so

    url = git://git.postgresql.org/git/postgresql.git

would become

    url = https://git.postgresql.org/git/postgresql.git


Enjoy


cheers


andrew


--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <[hidden email]> writes:
> PostgreSQL Buildfarm client Release 6 is now available and can be
> downloaded from
> <https://buildfarm.postgresql.org/downloads/releases/build-farm-6.tgz>

This seems to have gotten much chattier than previous versions;
it's emitting routine messages on stdout (or possibly stderr)
like this:

Thu Jan  4 12:29:02 2018: buildfarm run for longfin:REL9_3_STABLE starting
[12:29:02] checking out source ...
[12:29:07] checking if build run needed ...
[12:29:07] No build required: last status = Wed Jan  3 20:39:01 2018 GMT, current snapshot = Wed Jan  3 20:34:21 2018 GMT, changed files = 0
Thu Jan  4 12:29:07 2018: buildfarm run for longfin:REL9_4_STABLE starting
[12:29:08] checking out source ...
[12:29:09] checking if build run needed ...
[12:29:10] No build required: last status = Wed Jan  3 20:49:41 2018 GMT, current snapshot = Wed Jan  3 20:34:02 2018 GMT, changed files = 0
Thu Jan  4 12:29:10 2018: buildfarm run for longfin:REL9_5_STABLE starting
[12:29:10] checking out source ...
[12:29:12] checking if build run needed ...
[12:29:12] No build required: last status = Wed Jan  3 21:06:41 2018 GMT, current snapshot = Wed Jan  3 20:39:59 2018 GMT, changed files = 0
Thu Jan  4 12:29:12 2018: buildfarm run for longfin:REL9_6_STABLE starting
[12:29:12] checking out source ...
[12:29:15] checking if build run needed ...
[12:29:15] No build required: last status = Wed Jan  3 22:39:18 2018 GMT, current snapshot = Wed Jan  3 21:16:43 2018 GMT, changed files = 0
Thu Jan  4 12:29:15 2018: buildfarm run for longfin:REL_10_STABLE starting
[12:29:15] checking out source ...
[12:29:18] checking if build run needed ...
[12:29:18] No build required: last status = Wed Jan  3 21:47:10 2018 GMT, current snapshot = Wed Jan  3 21:22:35 2018 GMT, changed files = 0
Thu Jan  4 12:29:18 2018: buildfarm run for longfin:HEAD starting
[12:29:18] checking out source ...
[12:29:21] checking if build run needed ...
[12:29:21] No build required: last status = Thu Jan  4 16:59:05 2018 GMT, current snapshot = Thu Jan  4 15:34:41 2018 GMT, changed files = 0

which means I get mail every time the cron script runs.  Needless
to say, that will not do.  Previously, mail only showed up when
there was a git checkout failure.  How do I get back to the
old behavior?

                        regards, tom lane

I wrote:
> which means I get mail every time the cron script runs.  Needless
> to say, that will not do.  Previously, mail only showed up when
> there was a git checkout failure.  How do I get back to the
> old behavior?

Diff'ing the v5 and v6 releases soon provided the answer: for some
reason v6 is forcing the verbose switch on.  I hope that was
unintentional?  Anyway, I'm happy again after applying this patch:

--- PGBuild/Options.pm~ 2018-01-04 08:38:29.000000000 -0500
+++ PGBuild/Options.pm  2018-01-04 12:40:14.000000000 -0500
@@ -78,8 +78,6 @@
     GetOptions(%standard_options, @_)
       || die "bad command line";
 
-    # override GetOptions default for :i
-    $verbose = 1 if (defined($verbose) && $verbose==0);
     $verbose ||= 0; # stop complaints about undefined var in numeric comparison
 }
 

                        regards, tom lane

On 01/04/2018 12:51 PM, Tom Lane wrote:


OK, I'll sort that out. Thanks for the fast report.

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 01/04/2018 12:51 PM, Tom Lane wrote:



What is weird is that the only change has been to move this from
run_build.pl into PGBuild::Options::fetch_options. Those two lines have
been in run_build.pl for many years:

2cdd46ae    (Andrew Dunstan 2004-09-24 20:22:32 +0000   111)$verbose=1
if (defined($verbose) && $verbose==0);
4307d647    (Andrew Dunstan 2005-12-23 16:55:53 +0000   112)$verbose ||=
0; # stop complaints about undefined var in numeric comparison

and the lines you showed come from run_build.pl.

The rationale for the move was to make the flag sanely available to
other programs, specifically run_branches.pl.

So it's very odd. I'll do some experimenting.

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 01/04/2018 01:27 PM, Andrew Dunstan wrote:
Hah! I have it. How annoying. It's run_branches passing the adjusted
value through to run_build. Will fix.

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 01/04/2018 12:51 PM, Tom Lane wrote:


This should fix it:
<https://raw.githubusercontent.com/PGBuildFarm/client-code/5c71fb6ba1be83db6aca45e718b4a541b0b89f4c/PGBuild/Options.pm>

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services