coverage.postgresql.org not being refreshed

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

coverage.postgresql.org not being refreshed

Peter Geoghegan-4
coverage.postgresql.org reports that its "current" coverage report is
from 2021-01-11 19:06:43. I imagine that the backend infrastructure
that runs lcov is having issues.


--
Peter Geoghegan


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Magnus Hagander-2
On Sat, Jan 23, 2021 at 6:51 PM Peter Geoghegan <[hidden email]> wrote:
>
> coverage.postgresql.org reports that its "current" coverage report is
> from 2021-01-11 19:06:43. I imagine that the backend infrastructure
> that runs lcov is having issues.

It's indeed failing, and it seems we don't have any monitoring on it :/

All runs since the 11th have failed with a dead regression test:
test connect/test1                ... stderr FAILED      202 ms

I notice that the coverage scripts explicitly run:
make -C src/interfaces/ecpg/test checktcp

And that's what's been failing. And it is also failing for me locally.

So it seems we have a test that's failing on debian at least. But we
never run this on the buildfarm anywhere?

Looking at the git log of that directory, I'm guessing it's related to
some of the error message changes from Tom.

(And I've now added monitoring)

--
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Tom Lane-2
Magnus Hagander <[hidden email]> writes:
> All runs since the 11th have failed with a dead regression test:
> test connect/test1                ... stderr FAILED      202 ms

> I notice that the coverage scripts explicitly run:
> make -C src/interfaces/ecpg/test checktcp

Ugh, no I never run that test either.  Will fix.

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Peter Geoghegan-4
On Sat, Jan 23, 2021 at 11:45 AM Tom Lane <[hidden email]> wrote:
> Ugh, no I never run that test either.  Will fix.

Thanks for taking care of this so quickly, Tom and Magnus.

--
Peter Geoghegan


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Magnus Hagander-2
In reply to this post by Tom Lane-2
On Sat, Jan 23, 2021 at 8:45 PM Tom Lane <[hidden email]> wrote:
>
> Magnus Hagander <[hidden email]> writes:
> > All runs since the 11th have failed with a dead regression test:
> > test connect/test1                ... stderr FAILED      202 ms
>
> > I notice that the coverage scripts explicitly run:
> > make -C src/interfaces/ecpg/test checktcp
>
> Ugh, no I never run that test either.  Will fix.

Thanks! You missed the cron by a couple of minutes, but it recovered
on the next run.

Is there a particular reason we don't run it on the buildfarm, at
least on some animals? Seems like that would be a much better way of
finding these problems...

--
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Tom Lane-2
Magnus Hagander <[hidden email]> writes:
> On Sat, Jan 23, 2021 at 8:45 PM Tom Lane <[hidden email]> wrote:
>> Ugh, no I never run that test either.  Will fix.

> Is there a particular reason we don't run it on the buildfarm, at
> least on some animals? Seems like that would be a much better way of
> finding these problems...

IIUC, the reason it's not run by default is that opening up a port
on localhost is insecure on multi-user systems.  While that might
be a problem for buildfarm animals too, the ones that are running
things like the SSL checks already have the same hazard.  Seems
like we could add a buildfarm client option to run this test.

Or we could just flush the test.  It's not very clear to me what
coverage it's adding.

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Álvaro Herrera
On 2021-Jan-23, Tom Lane wrote:

> Or we could just flush the test.  It's not very clear to me what
> coverage it's adding.

Hmm, I know I enabled it because it did cover something, though I no
longer remember what that is.


Is there a convenient way to compare two coverage runs?

--
Álvaro Herrera       Valdivia, Chile


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Tom Lane-2
Alvaro Herrera <[hidden email]> writes:
> Is there a convenient way to compare two coverage runs?

Hmm ... diff'ing the html output is really painful, but it
looks like using checktcp instead of check causes 48 lines
in ecpg to change from not-covered to covered.

$ grep -r lineNoCov coverage.std/src/interfaces/ecpg | wc
  12555  120279 2212944
$ grep -r lineNoCov coverage.tcp/src/interfaces/ecpg | wc
  12507  119723 2153410

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Tom Lane-2
I wrote:
> Alvaro Herrera <[hidden email]> writes:
>> Is there a convenient way to compare two coverage runs?

> Hmm ... diff'ing the html output is really painful, but it
> looks like using checktcp instead of check causes 48 lines
> in ecpg to change from not-covered to covered.

Enlarging on that a little: the coverage in src/interfaces/ecpg
changes from
        11860/(11860+12555.) = 0.48577
to
        11908/(11908+12507.) = 0.48773

Meanwhile, the coverage in src/interfaces/libpq changes from
        2421/(2421+4488.) = 0.35041
to
        2481/(2481+4428.) = 0.35910

and the whole-system coverage from
        256632/(256632+163564.) = 0.61074
to
        256839/(256839+163357.) = 0.61124

This is comparing coverage reports for the core regression tests
plus ecpg "make check" versus core plus ecpg "make checktcp".

The increase in coverage outside ecpg is presumably because
we otherwise aren't hitting TCP-socket code paths at all,
while Unix-socket code is covered in both cases.  So it's only
fair to include the non-ecpg coverage increase as a benefit
if you suppose that no other tests will hit the TCP code paths.
That might well have been true awhile ago, but I think that the
SSL and Kerberos tests are now covering much of that territory
(if coverage.postgresql.org is running those?)

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Álvaro Herrera
On 2021-Jan-23, Tom Lane wrote:

> Enlarging on that a little: the coverage in src/interfaces/ecpg
> changes from
> 11860/(11860+12555.) = 0.48577
> to
> 11908/(11908+12507.) = 0.48773

Hm, a 0.2% increase ... sounds marginal.

The other two increases are more interesting, but if the code is already
covered by ssl/ldap/kerberos, then the checktcp test isn't buying much there
either.

So the script has this:

make -j12
make -j12 -C contrib
make -j12 -Otarget check-world PG_TEST_EXTRA="ssl ldap kerberos"
make -C src/interfaces/ecpg/test checktcp
make coverage-html

I mentioned in https://postgr.es/m/20190530202311.GA22421@...
that ecpg coverage was bad, and that it increased with "checktcp".  I
didn't quote exactly what was the change, though :-(

If what this test mode adds is really just a 0.2% increase, then that
doesn't seem worth the risk.  I think the only extra test it runs is
connect/test1.

--
Álvaro Herrera       Valdivia, Chile
"Ed is the standard text editor."
      http://groups.google.com/group/alt.religion.emacs/msg/8d94ddab6a9b0ad3


Reply | Threaded
Open this post in threaded view
|

Re: coverage.postgresql.org not being refreshed

Tom Lane-2
Alvaro Herrera <[hidden email]> writes:
> Hm, a 0.2% increase ... sounds marginal.
> The other two increases are more interesting, but if the code is already
> covered by ssl/ldap/kerberos, then the checktcp test isn't buying much there
> either.

Since those are all non-default build options, I suppose that the
checktcp target does have some reason to live: it's the only test
suite that will exercise TCP-related code paths in a minimal build.
(On non-Windows platforms, anyway.)

However, if nobody knows about it and no buildfarm animals test it,
that argument seems pretty hypothetical :-(

                        regards, tom lane