Announcing Release 6 of PostgreSQL Buildfarm client

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Announcing Release 6 of PostgreSQL Buildfarm client

Andrew Dunstan-8

PostgreSQL Buildfarm client Release 6 is now available and can be
downloaded from
<https://buildfarm.postgresql.org/downloads/releases/build-farm-6.tgz>

Features

  * use a persistent cache for configure, leading to a substantial
    performance improvement for that step, based on an idea from Tom Lane
  * improve installcheck security, mirroring earlier changes on this in
    Postgres itself
  * allow skipping individual TAP tests, e,g, "--skip_steps=pg_dump-check"
  * a use_installcheck_parallel setting that uses the parallel schedule
    rather than the serial schedule for the installcheck step (not for MSVC)
  * use_valgrind and valgrind_options settings that will cause the
    installed postmaster to be run under valgrind and report any output.
    This doesn't apply to the check step, TAP tests, or any other steps
    not using the postmaster started by the start_db step.
  * change the default git target for new repos to use https rather than
    git protocol
  * get PROVE_FLAGS from the environment (can be set in config file)ok,
    allowing the overriding of the buildfarm default
  * include Redis_FDW and Blackhole_FDW example modules, not enabled by
    default.
  * new --schedule switch allowing the specification of an alternative
    schedule file for the installcheck stage
  * new --tests switch allowing for running a specified set of tests
    rather than a schedule file at the installcheck stage
  * adjust for recent changes in locations of various files such as logs
    and regression outputs
  * include application name in default log_line_prefix

There have also been a number of fairly minor bug fixes and changes, and
some fairly significant code reorganization and refactoring.

Some manual changes are advised.

First, buildfarm owners are advised to change their config files to add
%q%a to the log_line_prefix setting, if they haven't done so. There is
an example in build-farm.conf.sample:

    extra_config =>{
            DEFAULT => [
                q(log_line_prefix = '%m [%p:%l] %q%a '),
                "log_connections = 'true'",
                "log_disconnections = 'true'",
                "log_statement = 'all'",
                "fsync = off"
            ],
    },

Also, if you have existing repos using the git protocol, you can change
them to use https by changing the appropriate line in the repo's config
file, so

    url = git://git.postgresql.org/git/postgresql.git

would become

    url = https://git.postgresql.org/git/postgresql.git


Enjoy


cheers


andrew


--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Reply | Threaded
Open this post in threaded view
|

Re: Announcing Release 6 of PostgreSQL Buildfarm client

Tom Lane-2
Andrew Dunstan <[hidden email]> writes:
> PostgreSQL Buildfarm client Release 6 is now available and can be
> downloaded from
> <https://buildfarm.postgresql.org/downloads/releases/build-farm-6.tgz>

This seems to have gotten much chattier than previous versions;
it's emitting routine messages on stdout (or possibly stderr)
like this:

Thu Jan  4 12:29:02 2018: buildfarm run for longfin:REL9_3_STABLE starting
[12:29:02] checking out source ...
[12:29:07] checking if build run needed ...
[12:29:07] No build required: last status = Wed Jan  3 20:39:01 2018 GMT, current snapshot = Wed Jan  3 20:34:21 2018 GMT, changed files = 0
Thu Jan  4 12:29:07 2018: buildfarm run for longfin:REL9_4_STABLE starting
[12:29:08] checking out source ...
[12:29:09] checking if build run needed ...
[12:29:10] No build required: last status = Wed Jan  3 20:49:41 2018 GMT, current snapshot = Wed Jan  3 20:34:02 2018 GMT, changed files = 0
Thu Jan  4 12:29:10 2018: buildfarm run for longfin:REL9_5_STABLE starting
[12:29:10] checking out source ...
[12:29:12] checking if build run needed ...
[12:29:12] No build required: last status = Wed Jan  3 21:06:41 2018 GMT, current snapshot = Wed Jan  3 20:39:59 2018 GMT, changed files = 0
Thu Jan  4 12:29:12 2018: buildfarm run for longfin:REL9_6_STABLE starting
[12:29:12] checking out source ...
[12:29:15] checking if build run needed ...
[12:29:15] No build required: last status = Wed Jan  3 22:39:18 2018 GMT, current snapshot = Wed Jan  3 21:16:43 2018 GMT, changed files = 0
Thu Jan  4 12:29:15 2018: buildfarm run for longfin:REL_10_STABLE starting
[12:29:15] checking out source ...
[12:29:18] checking if build run needed ...
[12:29:18] No build required: last status = Wed Jan  3 21:47:10 2018 GMT, current snapshot = Wed Jan  3 21:22:35 2018 GMT, changed files = 0
Thu Jan  4 12:29:18 2018: buildfarm run for longfin:HEAD starting
[12:29:18] checking out source ...
[12:29:21] checking if build run needed ...
[12:29:21] No build required: last status = Thu Jan  4 16:59:05 2018 GMT, current snapshot = Thu Jan  4 15:34:41 2018 GMT, changed files = 0

which means I get mail every time the cron script runs.  Needless
to say, that will not do.  Previously, mail only showed up when
there was a git checkout failure.  How do I get back to the
old behavior?

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: Announcing Release 6 of PostgreSQL Buildfarm client

Tom Lane-2
I wrote:
> which means I get mail every time the cron script runs.  Needless
> to say, that will not do.  Previously, mail only showed up when
> there was a git checkout failure.  How do I get back to the
> old behavior?

Diff'ing the v5 and v6 releases soon provided the answer: for some
reason v6 is forcing the verbose switch on.  I hope that was
unintentional?  Anyway, I'm happy again after applying this patch:

--- PGBuild/Options.pm~ 2018-01-04 08:38:29.000000000 -0500
+++ PGBuild/Options.pm  2018-01-04 12:40:14.000000000 -0500
@@ -78,8 +78,6 @@
     GetOptions(%standard_options, @_)
       || die "bad command line";
 
-    # override GetOptions default for :i
-    $verbose = 1 if (defined($verbose) && $verbose==0);
     $verbose ||= 0; # stop complaints about undefined var in numeric comparison
 }
 

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: Announcing Release 6 of PostgreSQL Buildfarm client

Andrew Dunstan-8


On 01/04/2018 12:51 PM, Tom Lane wrote:

> I wrote:
>> which means I get mail every time the cron script runs.  Needless
>> to say, that will not do.  Previously, mail only showed up when
>> there was a git checkout failure.  How do I get back to the
>> old behavior?
> Diff'ing the v5 and v6 releases soon provided the answer: for some
> reason v6 is forcing the verbose switch on.  I hope that was
> unintentional?  Anyway, I'm happy again after applying this patch:
>
> --- PGBuild/Options.pm~ 2018-01-04 08:38:29.000000000 -0500
> +++ PGBuild/Options.pm  2018-01-04 12:40:14.000000000 -0500
> @@ -78,8 +78,6 @@
>      GetOptions(%standard_options, @_)
>        || die "bad command line";
>  
> -    # override GetOptions default for :i
> -    $verbose = 1 if (defined($verbose) && $verbose==0);
>      $verbose ||= 0; # stop complaints about undefined var in numeric comparison
>  }
>  
>
>



OK, I'll sort that out. Thanks for the fast report.

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Reply | Threaded
Open this post in threaded view
|

Re: Announcing Release 6 of PostgreSQL Buildfarm client

Andrew Dunstan-8
In reply to this post by Tom Lane-2


On 01/04/2018 12:51 PM, Tom Lane wrote:

> I wrote:
>> which means I get mail every time the cron script runs.  Needless
>> to say, that will not do.  Previously, mail only showed up when
>> there was a git checkout failure.  How do I get back to the
>> old behavior?
> Diff'ing the v5 and v6 releases soon provided the answer: for some
> reason v6 is forcing the verbose switch on.  I hope that was
> unintentional?  Anyway, I'm happy again after applying this patch:
>
> --- PGBuild/Options.pm~ 2018-01-04 08:38:29.000000000 -0500
> +++ PGBuild/Options.pm  2018-01-04 12:40:14.000000000 -0500
> @@ -78,8 +78,6 @@
>      GetOptions(%standard_options, @_)
>        || die "bad command line";
>  
> -    # override GetOptions default for :i
> -    $verbose = 1 if (defined($verbose) && $verbose==0);
>      $verbose ||= 0; # stop complaints about undefined var in numeric comparison
>  }
>  
>
>




What is weird is that the only change has been to move this from
run_build.pl into PGBuild::Options::fetch_options. Those two lines have
been in run_build.pl for many years:

2cdd46ae    (Andrew Dunstan 2004-09-24 20:22:32 +0000   111)$verbose=1
if (defined($verbose) && $verbose==0);
4307d647    (Andrew Dunstan 2005-12-23 16:55:53 +0000   112)$verbose ||=
0; # stop complaints about undefined var in numeric comparison

and the lines you showed come from run_build.pl.

The rationale for the move was to make the flag sanely available to
other programs, specifically run_branches.pl.

So it's very odd. I'll do some experimenting.

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Reply | Threaded
Open this post in threaded view
|

Re: Announcing Release 6 of PostgreSQL Buildfarm client

Andrew Dunstan-8


On 01/04/2018 01:27 PM, Andrew Dunstan wrote:

>
> On 01/04/2018 12:51 PM, Tom Lane wrote:
>> I wrote:
>>> which means I get mail every time the cron script runs.  Needless
>>> to say, that will not do.  Previously, mail only showed up when
>>> there was a git checkout failure.  How do I get back to the
>>> old behavior?
>> Diff'ing the v5 and v6 releases soon provided the answer: for some
>> reason v6 is forcing the verbose switch on.  I hope that was
>> unintentional?  Anyway, I'm happy again after applying this patch:
>>
>> --- PGBuild/Options.pm~ 2018-01-04 08:38:29.000000000 -0500
>> +++ PGBuild/Options.pm  2018-01-04 12:40:14.000000000 -0500
>> @@ -78,8 +78,6 @@
>>      GetOptions(%standard_options, @_)
>>        || die "bad command line";
>>  
>> -    # override GetOptions default for :i
>> -    $verbose = 1 if (defined($verbose) && $verbose==0);
>>      $verbose ||= 0; # stop complaints about undefined var in numeric comparison
>>  }
>>  
>>
>>
>
>
>
> What is weird is that the only change has been to move this from
> run_build.pl into PGBuild::Options::fetch_options. Those two lines have
> been in run_build.pl for many years:
>
> 2cdd46ae    (Andrew Dunstan 2004-09-24 20:22:32 +0000   111)$verbose=1
> if (defined($verbose) && $verbose==0);
> 4307d647    (Andrew Dunstan 2005-12-23 16:55:53 +0000   112)$verbose ||=
> 0; # stop complaints about undefined var in numeric comparison
>
> and the lines you showed come from run_build.pl.
>
> The rationale for the move was to make the flag sanely available to
> other programs, specifically run_branches.pl.
>
> So it's very odd. I'll do some experimenting.
>

Hah! I have it. How annoying. It's run_branches passing the adjusted
value through to run_build. Will fix.

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Reply | Threaded
Open this post in threaded view
|

Re: Announcing Release 6 of PostgreSQL Buildfarm client

Andrew Dunstan-8
In reply to this post by Tom Lane-2


On 01/04/2018 12:51 PM, Tom Lane wrote:

> I wrote:
>> which means I get mail every time the cron script runs.  Needless
>> to say, that will not do.  Previously, mail only showed up when
>> there was a git checkout failure.  How do I get back to the
>> old behavior?
> Diff'ing the v5 and v6 releases soon provided the answer: for some
> reason v6 is forcing the verbose switch on.  I hope that was
> unintentional?  Anyway, I'm happy again after applying this patch:
>
> --- PGBuild/Options.pm~ 2018-01-04 08:38:29.000000000 -0500
> +++ PGBuild/Options.pm  2018-01-04 12:40:14.000000000 -0500
> @@ -78,8 +78,6 @@
>      GetOptions(%standard_options, @_)
>        || die "bad command line";
>  
> -    # override GetOptions default for :i
> -    $verbose = 1 if (defined($verbose) && $verbose==0);
>      $verbose ||= 0; # stop complaints about undefined var in numeric comparison
>  }
>  



This should fix it:
<https://raw.githubusercontent.com/PGBuildFarm/client-code/5c71fb6ba1be83db6aca45e718b4a541b0b89f4c/PGBuild/Options.pm>

cheers

andrew

--
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services