On Sat, Jan 23, 2021 at 6:51 PM Peter Geoghegan <[hidden email]> wrote:
> coverage.postgresql.org reports that its "current" coverage report is
> from 2021-01-11 19:06:43. I imagine that the backend infrastructure
> that runs lcov is having issues.
It's indeed failing, and it seems we don't have any monitoring on it :/
All runs since the 11th have failed with a dead regression test:
test connect/test1 ... stderr FAILED 202 ms
I notice that the coverage scripts explicitly run:
make -C src/interfaces/ecpg/test checktcp
And that's what's been failing. And it is also failing for me locally.
So it seems we have a test that's failing on debian at least. But we
never run this on the buildfarm anywhere?
Looking at the git log of that directory, I'm guessing it's related to
some of the error message changes from Tom.
On Sat, Jan 23, 2021 at 8:45 PM Tom Lane <[hidden email]> wrote:
> Magnus Hagander <[hidden email]> writes:
> > All runs since the 11th have failed with a dead regression test:
> > test connect/test1 ... stderr FAILED 202 ms
> > I notice that the coverage scripts explicitly run:
> > make -C src/interfaces/ecpg/test checktcp
> Ugh, no I never run that test either. Will fix.
Thanks! You missed the cron by a couple of minutes, but it recovered
on the next run.
Is there a particular reason we don't run it on the buildfarm, at
least on some animals? Seems like that would be a much better way of
finding these problems...
Magnus Hagander <[hidden email]> writes:
> On Sat, Jan 23, 2021 at 8:45 PM Tom Lane <[hidden email]> wrote:
>> Ugh, no I never run that test either. Will fix.
> Is there a particular reason we don't run it on the buildfarm, at
> least on some animals? Seems like that would be a much better way of
> finding these problems...
IIUC, the reason it's not run by default is that opening up a port
on localhost is insecure on multi-user systems. While that might
be a problem for buildfarm animals too, the ones that are running
things like the SSL checks already have the same hazard. Seems
like we could add a buildfarm client option to run this test.
Or we could just flush the test. It's not very clear to me what
coverage it's adding.
> Alvaro Herrera <[hidden email]> writes:
>> Is there a convenient way to compare two coverage runs?
> Hmm ... diff'ing the html output is really painful, but it
> looks like using checktcp instead of check causes 48 lines
> in ecpg to change from not-covered to covered.
Enlarging on that a little: the coverage in src/interfaces/ecpg
11860/(11860+12555.) = 0.48577
11908/(11908+12507.) = 0.48773
Meanwhile, the coverage in src/interfaces/libpq changes from
2421/(2421+4488.) = 0.35041
2481/(2481+4428.) = 0.35910
and the whole-system coverage from
256632/(256632+163564.) = 0.61074
256839/(256839+163357.) = 0.61124
This is comparing coverage reports for the core regression tests
plus ecpg "make check" versus core plus ecpg "make checktcp".
The increase in coverage outside ecpg is presumably because
we otherwise aren't hitting TCP-socket code paths at all,
while Unix-socket code is covered in both cases. So it's only
fair to include the non-ecpg coverage increase as a benefit
if you suppose that no other tests will hit the TCP code paths.
That might well have been true awhile ago, but I think that the
SSL and Kerberos tests are now covering much of that territory
(if coverage.postgresql.org is running those?)
Alvaro Herrera <[hidden email]> writes:
> Hm, a 0.2% increase ... sounds marginal.
> The other two increases are more interesting, but if the code is already
> covered by ssl/ldap/kerberos, then the checktcp test isn't buying much there
Since those are all non-default build options, I suppose that the
checktcp target does have some reason to live: it's the only test
suite that will exercise TCP-related code paths in a minimal build.
(On non-Windows platforms, anyway.)
However, if nobody knows about it and no buildfarm animals test it,
that argument seems pretty hypothetical :-(