GCC 8.3.0 vs. 9.0.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

GCC 8.3.0 vs. 9.0.1

Steven Winfield
Hi,

(Apologies if this isn't the right place to post this)

A few days ago a blog post appeared on phoronix.com[1] comparing GCC 8.3.0 against 9.0.1 on Intel cascadelake processors.
A notable difference was seen in the PostgreSQL benchmark (v10.3, pgbench, read/write, more detail below), both when compiling with -march=native and -march=skylake:

GCC version | -march= | TPS
8.3.0 | skylake | 5667
9.0.1 | skylake | 11684 (2.06x speed up)
8.3.0 | native | 8075
9.0.1 | native | 11274 (1.40x speed up)

I'm interested to know the devs' take on this is - does GCC 9 contain some new feature(s) that are particularly well suited to compiling and optimising Postgres? Or was GCC 8 particularly bad?


The test script seems to be this one[2], and goes something like this:

- Postgres 10.3 is configure using --without-readline and --without-zlib (after patching it so that it can run as root). The remaining compiler options seem to be (implicitly?) "-fno-strict-aliasing -fwrapv -O3 -lpgcommon -lpq -lpthread -lrt -lcrypt -ldl -lm", plus the -march setting under test.

- initdb is run with --encoding=SQL_ASCII --locale=C

- the db is started with "pg_ctl start -o '-c autovacuum=false'"

- createdb pgbench

- pgbench -i -s <system memory in MB * 0.003> pgbench

- pgbench -j <number of cores> -c <number of cores * 4> -T 60 pgbench


Cheers,
Steven.

[1] https://www.phoronix.com/scan.php?page=news_item&px=Intel-Cascade-Lake-GCC9
[2] https://openbenchmarking.org/innhold/b53a0ca6dcfdc9b8597a7b144fae2110fa6af1fb




This email is confidential. If you are not the intended recipient, please advise us immediately and delete this message. The registered name of Cantab- part of GAM Systematic is Cantab Capital Partners LLP. See - http://www.gam.com/en/Legal/Email+disclosures+EU for further information on confidentiality, the risks of non-secure electronic communication, and certain disclosures which we are required to make in accordance with applicable legislation and regulations. If you cannot access this link, please notify us by reply message and we will send the contents to you.

GAM Holding AG and its subsidiaries (Cantab – GAM Systematic) will collect and use information about you in the course of your interactions with us. Full details about the data types we collect and what we use this for and your related rights is set out in our online privacy policy at https://www.gam.com/en/legal/privacy-policy. Please familiarise yourself with this policy and check it from time to time for updates as it supplements this notice
Reply | Threaded
Open this post in threaded view
|

Re: GCC 8.3.0 vs. 9.0.1

Tom Lane-2
Steven Winfield <[hidden email]> writes:
> A few days ago a blog post appeared on phoronix.com[1] comparing GCC 8.3.0 against 9.0.1 on Intel cascadelake processors.
> A notable difference was seen in the PostgreSQL benchmark (v10.3, pgbench, read/write, more detail below), both when compiling with -march=native and -march=skylake:
> I'm interested to know the devs' take on this is - does GCC 9 contain some new feature(s) that are particularly well suited to compiling and optimising Postgres? Or was GCC 8 particularly bad?

Given the described test setup, I'd put basically no stock in these
numbers.  It's unlikely that this test case's performance is CPU-bound
per se; more likely, I/O and lock contention are dominant factors.
So I'm afraid whatever they're measuring is a more-or-less chance
effect rather than a real system-wide code improvement.

It is an interesting report, all the same.

                        regards, tom lane


Reply | Threaded
Open this post in threaded view
|

Re: GCC 8.3.0 vs. 9.0.1

Peter Geoghegan-4
On Tue, May 7, 2019 at 10:06 AM Tom Lane <[hidden email]> wrote:
> Given the described test setup, I'd put basically no stock in these
> numbers.  It's unlikely that this test case's performance is CPU-bound
> per se; more likely, I/O and lock contention are dominant factors.
> So I'm afraid whatever they're measuring is a more-or-less chance
> effect rather than a real system-wide code improvement.

Or a compiler bug. Link-time optimizations give the compiler a view of
the program as a whole, not just a single TU at a time. This enables
it to perform additional aggressive optimization.

--
Peter Geoghegan


Reply | Threaded
Open this post in threaded view
|

Re: GCC 8.3.0 vs. 9.0.1

Andres Freund
In reply to this post by Steven Winfield
Hi,

On 2019-05-07 16:14:43 +0000, Steven Winfield wrote:
> (Apologies if this isn't the right place to post this)

Seems right.


> A few days ago a blog post appeared on phoronix.com[1] comparing GCC 8.3.0 against 9.0.1 on Intel cascadelake processors.
> A notable difference was seen in the PostgreSQL benchmark (v10.3, pgbench, read/write, more detail below), both when compiling with -march=native and -march=skylake:
>
> GCC version | -march= |   TPS
>       8.3.0 | skylake |  5667
>       9.0.1 | skylake | 11684 (2.06x speed up)
>       8.3.0 | native  |  8075
>       9.0.1 | native  | 11274 (1.40x speed up)
>
> I'm interested to know the devs' take on this is - does GCC 9 contain some new feature(s) that are particularly well suited to compiling and optimising Postgres? Or was GCC 8 particularly bad?

I think those numbers are just plain bogus. read/write pgbench is
commonly IO bound.  My suspicion is much more that the tests for gcc 8
and 9 were executed in the same postgres cluster (in which case the
second will be faster, because it'll have pre-initialized WAL files).
Or something of that vein.


> (after patching it so that it can run as root)

That, uh, seems odd.


> - pgbench -i -s <system memory in MB * 0.003> pgbench

That's pretty small, but whatever.


Here's my results:

I ran:

pgbench -i -q -s 96 && pgbench -n -c 8 -j 8 -T 100 -P1


gcc 8.3, march=native (on skylake):

first run:
tps = 14436.465265 (excluding connections establishing)

second run:
tps = 13293.266789 (excluding connections establishing)

third run after postgres restart (and thus a checkpoint):
tps = 14270.248273 (excluding connections establishing)


gcc 9.1, march=native (on skylake):

first run:
tps = 13836.231981 (excluding connections establishing)

second run:
tps = 13304.975550 (excluding connections establishing)

third run after postgres restart (and thus a checkpoint):
tps = 14390.246324 (excluding connections establishing)


As you can see the test results are somewhat unstable - the test
duration of 60s is just not long enough. But there's no meaningful
evidence of a large speedup here.




> This email is confidential. If you are not the intended recipient, please advise us immediately and delete this message.
> The registered name of Cantab- part of GAM Systematic is Cantab Capital Partners LLP.
> See - http://www.gam.com/en/Legal/Email+disclosures+EU for further information on confidentiality, the risks of non-secure electronic communication, and certain disclosures which we are required to make in accordance with applicable legislation and regulations.
> If you cannot access this link, please notify us by reply message and we will send the contents to you.
>
> GAM Holding AG and its subsidiaries (Cantab – GAM Systematic) will collect and use information about you in the course of your interactions with us.
> Full details about the data types we collect and what we use this for and your related rights is set out in our online privacy policy at https://www.gam.com/en/legal/privacy-policy.
> Please familiarise yourself with this policy and check it from time to time for updates as it supplements this notice.

This is a public list.

Greetings,

Andres Freund


Reply | Threaded
Open this post in threaded view
|

Re: GCC 8.3.0 vs. 9.0.1

Andres Freund
In reply to this post by Peter Geoghegan-4
Hi,

On 2019-05-07 10:28:16 -0700, Peter Geoghegan wrote:

> On Tue, May 7, 2019 at 10:06 AM Tom Lane <[hidden email]> wrote:
> > Given the described test setup, I'd put basically no stock in these
> > numbers.  It's unlikely that this test case's performance is CPU-bound
> > per se; more likely, I/O and lock contention are dominant factors.
> > So I'm afraid whatever they're measuring is a more-or-less chance
> > effect rather than a real system-wide code improvement.
>
> Or a compiler bug. Link-time optimizations give the compiler a view of
> the program as a whole, not just a single TU at a time. This enables
> it to perform additional aggressive optimization.

Note that the flags described don't enable LTO.

Greetings,

Andres Freund


Reply | Threaded
Open this post in threaded view
|

Re: GCC 8.3.0 vs. 9.0.1

Andres Freund
In reply to this post by Andres Freund
Hi,

On 2019-05-07 10:32:45 -0700, Andres Freund wrote:
> pgbench -i -q -s 96 && pgbench -n -c 8 -j 8 -T 100 -P1

possibly also worthwhile to note: Adding -M prepared (which I think
phoronix doesn't specify) makes this considerably faster...

Greetings,

Andres Freund


Reply | Threaded
Open this post in threaded view
|

RE: GCC 8.3.0 vs. 9.0.1

Steven Winfield
Thanks, everyone, for your comments.
I guess if something looks too good to be true then it usually is!

Steven.

(P.S. Apologies for the email disclaimer - it is added by our mail server, not my mail client, and its exclusion list is on the fritz)