benchmarking proposed config

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

benchmarking proposed config

Scott Ribe
Does anyone have suggestions for a utility that would easily let us mirror all queries to a "shadow" PG cluster, so that we can compare performance of our current production config against a new proposed config with our actual production traffic?

(We could live with the new one falling behind as long as queries were queued not lost--in fact we know this will happen.)

--
Scott Ribe
[hidden email]
https://www.linkedin.com/in/scottribe/





Reply | Threaded
Open this post in threaded view
|

Re: benchmarking proposed config

Jakub Olczyk
Hi Scott,

On Thu, 20 Feb 2020 09:12:09 -0700
Scott Ribe <[hidden email]> wrote:

> Does anyone have suggestions for a utility that would easily let us
> mirror all queries to a "shadow" PG cluster, so that we can compare
> performance of our current production config against a new proposed
> config with our actual production traffic?

I am not sure if this would be an acceptable solution for you but there
is an open source utility called pgreplay[1]. Here is the description:

> pgreplay reads a PostgreSQL log file (*not* a WAL file), extracts the
> SQL statements and executes them in the same order and relative time
> against a PostgreSQL database cluster.

This of course means that you'd need to have a fairly "chatty" log
configuration, but maybe it could work for your needs?


Cheers,
jwo


[1]: https://github.com/laurenz/pgreplay


Reply | Threaded
Open this post in threaded view
|

Re: benchmarking proposed config

Scott Ribe-2
On Feb 21, 2020, at 2:55 AM, Jakub Olczyk <[hidden email]> wrote:
>
> This of course means that you'd need to have a fairly "chatty" log
> configuration, but maybe it could work for your needs?

It sure might work; we'll give it a try--hopefully we can log all SQL without filling disk or undue performance hit, we'll see.

Reply | Threaded
Open this post in threaded view
|

Re: benchmarking proposed config

bricklen
In reply to this post by Scott Ribe
On Thu, Feb 20, 2020 at 12:10 PM Scott Ribe <[hidden email]> wrote:
Does anyone have suggestions for a utility that would easily let us mirror all queries to a "shadow" PG cluster, so that we can compare performance of our current production config against a new proposed config with our actual production traffic?

(We could live with the new one falling behind as long as queries were queued not lost--in fact we know this will happen.)

PgShark might work for this