How to get the content of Bind variables

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get the content of Bind variables

Did16

Hi

               In the log file of my PostgreSQL cluster, I find :

>> 

Statement: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8

<< 

 

è how to get the content of the bind variables ?

 

Thanks in advance

 

Best Regards

cid:image002.png@01D14E0E.8515EB90


Didier ROS

Expertise SGBD

EDF - DTEO - DSIT - IT DMA

Département Solutions Groupe

Groupe Performance Applicative

32 avenue Pablo Picasso

92000 NANTERRE

 

[hidden email]

Tél. : +33 6 49 51 11 88

[hidden email]<a href="sip:prenom.nom@edf.fr">cid:image004.png@01D4BE20.1EAF68B0

 

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Michaeldba@sqlexec.com
If you set log_min_duration_statement low enough for your particular query, you will see another line below it showing what values are associated with each bind variable like this:

2019-02-28 00:07:55 CST 2019-02-28 00:02:09 CST ihr2 10.86.42.184(43460) SELECT LOG:  duration: 26078.308 ms  execute <unnamed>: select pg_advisory_lock($1)

2019-02-28 00:07:55 CST 2019-02-28 00:02:09 CST ihr2 10.86.42.184(43460) SELECT DETAIL:  parameters: $1 = '3428922050323511872'

Regards,
Michael Vitale
Thursday, February 28, 2019 7:21 AM

Hi

               In the log file of my PostgreSQL cluster, I find :

>> 

Statement: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8

<< 

 

è how to get the content of the bind variables ?

 

Thanks in advance

 

Best Regards

cid:image002.png@01D14E0E.8515EB90



Didier ROS

Expertise SGBD

EDF - DTEO - DSIT - IT DMA

Département Solutions Groupe

Groupe Performance Applicative

32 avenue Pablo Picasso

92000 NANTERRE

 

[hidden email]

Tél. : +33 6 49 51 11 88

[hidden email]<a moz-do-not-send="true" href="sip:prenom.nom@edf.fr">cid:image004.png@01D4BE20.1EAF68B0

 

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.


Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16

Hi

               Thanks for the answer.

 

I have  in my postgresql.conf :

log_min_duration_statement=0

and the content of bind variables is not showed in the log file.

What can I do to get the content of the bind variables ?

 

Best Regard

cid:image002.png@01D14E0E.8515EB90


Didier ROS

Expertise SGBD

EDF - DTEO - DSIT - IT DMA

Département Solutions Groupe

Groupe Performance Applicative

32 avenue Pablo Picasso

92000 NANTERRE

 

[hidden email]

Tél. : +33 6 49 51 11 88

[hidden email]<a href="sip:prenom.nom@edf.fr">cid:image004.png@01D4BE20.1EAF68B0

 

 

De : [hidden email] [mailto:[hidden email]]
Envoyé : jeudi 28 février 2019 13:37
À : ROS Didier <[hidden email]>
Cc : [hidden email]
Objet : Re: How to get the content of Bind variables

 

If you set log_min_duration_statement low enough for your particular query, you will see another line below it showing what values are associated with each bind variable like this:

2019-02-28 00:07:55 CST 2019-02-28 00:02:09 CST ihr2 10.86.42.184(43460) SELECT LOG:  duration: 26078.308 ms  execute <unnamed>: select pg_advisory_lock($1)

2019-02-28 00:07:55 CST 2019-02-28 00:02:09 CST ihr2 10.86.42.184(43460) SELECT DETAIL:  parameters: $1 = '3428922050323511872'

Regards,
Michael Vitale

Thursday, February 28, 2019 7:21 AM

Hi

               In the log file of my PostgreSQL cluster, I find :

>> 

Statement: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8

<< 

 

è how to get the content of the bind variables ?

 

Thanks in advance

 

Best Regards

cid:image002.png@01D14E0E.8515EB90


Didier ROS

Expertise SGBD

EDF - DTEO - DSIT - IT DMA

Département Solutions Groupe

Groupe Performance Applicative

32 avenue Pablo Picasso

92000 NANTERRE

 

[hidden email]

Tél. : +33 6 49 51 11 88

[hidden email]<a href="sip:prenom.nom@edf.fr">cid:image004.png@01D4BE20.1EAF68B0

 

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Laurenz Albe
In reply to this post by Did16
ROS Didier wrote:
>                In the log file of my PostgreSQL cluster, I find :
> >>
> Statement: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
> <<
>  
> how to get the content of the bind variables ?

Can we see the whole log entry and the following one?

Perhaps there was a syntax error or similar, and the statement was never executed.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com


Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16
Hi Laurent

Here is a biggest part of my log file :

>>
 2019-02-27 14:41:28 CET [16239]: [5696-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  duration: 1.604 ms
2019-02-27 14:41:28 CET [16239]: [5697-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  duration: 0.084 ms  parse <unnamed>: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
2019-02-27 14:41:28 CET [16239]: [5698-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  plan:
2019-02-27 14:41:28 CET [16239]: [5699-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainSTATEMENT:  update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
2019-02-27 14:41:28 CET [16239]: [5700-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  duration: 0.288 ms  bind <unnamed>: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
2019-02-27 14:41:28 CET [16239]: [5701-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  execute <unnamed>: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
<<
The statement has been executed
It is the same problem for all the statements.
I can not get the content of the bind variables.


Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA
Département Solutions Groupe
Groupe Performance Applicative
32 avenue Pablo Picasso
92000 NANTERRE
 
[hidden email]
Tél. : +33 6 49 51 11 88


-----Message d'origine-----
De : [hidden email] [mailto:[hidden email]]
Envoyé : jeudi 28 février 2019 17:01
À : ROS Didier <[hidden email]>; [hidden email]
Objet : Re: How to get the content of Bind variables

ROS Didier wrote:
>                In the log file of my PostgreSQL cluster, I find :
> >>
> Statement: update t_shared_liste_valeurs set deletion_date=$1,
> deletion_login=$2, modification_date=$3, modification_login=$4,
> administrable=$5, libelle=$6, niveau=$7 where code=$8 <<
>  
> how to get the content of the bind variables ?

Can we see the whole log entry and the following one?

Perhaps there was a syntax error or similar, and the statement was never executed.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com




Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Justin Pryzby
In reply to this post by Did16
On Thu, Feb 28, 2019 at 12:21:56PM +0000, ROS Didier wrote:
> Statement: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
>
> è how to get the content of the bind variables ?

What is your setting of log_error_verbosity ?
https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-ERROR-VERBOSITY

Also, I recommend using CSV logs, since they're easier to import into the DB
and then easier to parse.
https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-ERROR-VERBOSITY

Also, note that you can either set log_min_duration_statement=0, which logs all
statement durations, and associated statements (if they haven't been previously
logged).

Or, you can set log_statement=all, which logs all statements (but duration is
only logged according to log_min_duration_statement).

Justin

Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16
Hi
        Here is the information :

postgres=# show log_error_verbosity ;
 log_error_verbosity
---------------------
 default
(1 row)


postgres=# show log_statement ;
 log_statement
---------------
 none
(1 row)

I am trying now to set up log_statement :
log_statement=all ;
log_min_duration_statement=250;


Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA
Département Solutions Groupe
Groupe Performance Applicative
32 avenue Pablo Picasso
92000 NANTERRE
 
[hidden email]
Tél. : +33 6 49 51 11 88




-----Message d'origine-----
De : [hidden email] [mailto:[hidden email]]
Envoyé : jeudi 28 février 2019 17:19
À : ROS Didier <[hidden email]>
Cc : [hidden email]
Objet : Re: How to get the content of Bind variables

On Thu, Feb 28, 2019 at 12:21:56PM +0000, ROS Didier wrote:
> Statement: update t_shared_liste_valeurs set deletion_date=$1,
> deletion_login=$2, modification_date=$3, modification_login=$4,
> administrable=$5, libelle=$6, niveau=$7 where code=$8
>
> è how to get the content of the bind variables ?

What is your setting of log_error_verbosity ?
https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-ERROR-VERBOSITY

Also, I recommend using CSV logs, since they're easier to import into the DB and then easier to parse.
https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-ERROR-VERBOSITY

Also, note that you can either set log_min_duration_statement=0, which logs all statement durations, and associated statements (if they haven't been previously logged).

Or, you can set log_statement=all, which logs all statements (but duration is only logged according to log_min_duration_statement).

Justin



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.


Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Laurenz Albe
In reply to this post by Did16
ROS Didier wrote:

> Here is a biggest part of my log file :
>
>  2019-02-27 14:41:28 CET [16239]: [5696-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  duration: 1.604 ms
> 2019-02-27 14:41:28 CET [16239]: [5697-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  duration: 0.084 ms  parse <unnamed>: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
> 2019-02-27 14:41:28 CET [16239]: [5698-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  plan:
> 2019-02-27 14:41:28 CET [16239]: [5699-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainSTATEMENT:  update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
> 2019-02-27 14:41:28 CET [16239]: [5700-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  duration: 0.288 ms  bind <unnamed>: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
> 2019-02-27 14:41:28 CET [16239]: [5701-1] [10086] user=pgbd_preint_sg2,db=pgbd_preint_sg2,client=localhost.localdomainLOG:  execute <unnamed>: update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8
> <<
> The statement has been executed
> It is the same problem for all the statements.
> I can not get the content of the bind variables.

You should set "log_error_verbosity" back from "terse" to "default".
Then you will see the DETAIL messages.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com


Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Tom Lane-2
In reply to this post by Did16
ROS Didier <[hidden email]> writes:
> postgres=# show log_error_verbosity ;
>  log_error_verbosity
> ---------------------
>  default
> (1 row)

So ... how old is this server?  AFAIK the above should be enough to ensure
you get the DETAIL lines with parameter values.  But the ability to log
those hasn't been there forever.

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16
Hi Tom

        Thanks a lot for your answer.

*) Here is information about my server :
[postgres@noeyypvd pg_log]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.3 (Maipo)

postgres=# select version() ;
                                                 version
---------------------------------------------------------------------------------------------------------
 PostgreSQL 10.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16), 64-bit
(1 row)

  *)  it's very problematic that we can not get the content of bind variables. We can not determine the root query which makes UPDATE statements to crash our production database.
What can explain the lack of information about bind variables?

*) Here is the parameters setting I use :
# postgresql.conf : include_if_exists = '/appli/postgres/pgbd_prod_pda/10/conf/audit.conf'

log_rotation_size = 0
log_destination=stderr
logging_collector=true
client_min_messages=notice
log_min_messages=ERROR
log_min_error_statement=ERROR
log_min_duration_statement=250
debug_print_parse=off
debug_print_rewritten=off
debug_print_plan=on
debug_pretty_print=on
log_checkpoints=on
log_connections=on
log_disconnections=on
log_duration=on
log_error_verbosity=VERBOSE
log_hostname=on
log_lock_waits=on
deadlock_timeout=1s
log_statement=all
log_temp_files=0
log_autovacuum_min_duration = 0
track_activities=on
track_io_timing=on
track_functions=all
log_line_prefix = '%t [%p]: [%l-1] [%x] user=%u,db=%d,client=%h'
lc_messages ='C'
shared_preload_libraries = 'passwordcheck,pg_stat_statements,pgstattuple'
listen_addresses = '*'
pg_stat_statements.track=all
pg_stat_statements.max = 1000
pg_stat_statements.track_utility=on
pg_stat_statements.save=on
   
*) -> suggestion : It would be nice to have the content of bind variable of a query in a table of  pg_catalog. (cf ORACLE)

Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA
Département Solutions Groupe
Groupe Performance Applicative
32 avenue Pablo Picasso
92000 NANTERRE
 
[hidden email]
Tél. : +33 6 49 51 11 88

-----Message d'origine-----
De : [hidden email] [mailto:[hidden email]]
Envoyé : vendredi 1 mars 2019 17:30
À : ROS Didier <[hidden email]>
Cc : [hidden email]; [hidden email]
Objet : Re: How to get the content of Bind variables

ROS Didier <[hidden email]> writes:
> postgres=# show log_error_verbosity ;
>  log_error_verbosity
> ---------------------
>  default
> (1 row)

So ... how old is this server?  AFAIK the above should be enough to ensure you get the DETAIL lines with parameter values.  But the ability to log those hasn't been there forever.

                        regards, tom lane



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.


Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

legrand legrand
Hi Didier,

I imagine that this is the sql executed from a trigger.
Could you provide the trigger pl/pgsql code ?
as the source and target tables (anonymized) definition ?

After a fresh db restart, are thoses logs the same for the 6 first
executions and the following ones ?

Regards
PAscal



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-performance-f2050081.html

Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16

Hi

 

               The SQL is not executed from a trigger.

               Here is an extract of my log file :

>> 

2019-03-01 14:53:37 CET [24803]: [129-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 LOG:  process 24803 still waiting for ShareLock on transaction 3711 after 1000.476 ms

2019-03-01 14:53:37 CET [24803]: [130-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 DETAIL:  Process holding the lock: 24786. Wait queue: 24803.

2019-03-01 14:53:37 CET [24803]: [131-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 CONTEXT:  while rechecking updated tuple (3,33) in relation "t_shared_liste_valeurs"

2019-03-01 14:53:37 CET [24803]: [132-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 STATEMENT:  update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8

<< 

 

After a fresh db restart, the result is the same : no content of Bind variables in the log file.

 

Best Regardscid:image002.png@01D14E0E.8515EB90


Didier ROS

Expertise SGBD

EDF - DTEO - DSIT - IT DMA

Département Solutions Groupe

Groupe Performance Applicative

32 avenue Pablo Picasso

92000 NANTERRE

 

[hidden email]

Tél. : +33 6 49 51 11 88

[hidden email]<a href="sip:prenom.nom@edf.fr">cid:image004.png@01D4BE20.1EAF68B0

 

 

-----Message d'origine-----
De : [hidden email] [mailto:[hidden email]]
Envoyé : vendredi 1 mars 2019 21:42
À : [hidden email]
Objet : RE: How to get the content of Bind variables

 

Hi Didier,

 

I imagine that this is the sql executed from a trigger.

Could you provide the trigger pl/pgsql code ?

as the source and target tables (anonymized) definition ?

 

After a fresh db restart, are thoses logs the same for the 6 first executions and the following ones ?

 

Regards

PAscal

 

 

 

--

Sent from: http://www.postgresql-archive.org/PostgreSQL-performance-f2050081.html

 


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

legrand legrand
Did16 wrote

> Hi
>                The SQL is not executed from a trigger.
>                Here is an extract of my log file :
>>>
>
> 2019-03-01 14:53:37 CET [24803]: [129-1] [3686]
> user=pgbd_preint_sg2,db=pgbd_preint_sg2 LOG:  process 24803 still waiting
> for ShareLock on transaction 3711 after 1000.476 ms
>
> 2019-03-01 14:53:37 CET [24803]: [130-1] [3686]
> user=pgbd_preint_sg2,db=pgbd_preint_sg2 DETAIL:  Process holding the lock:
> 24786. Wait queue: 24803.
>
> 2019-03-01 14:53:37 CET [24803]: [131-1] [3686]
> user=pgbd_preint_sg2,db=pgbd_preint_sg2 CONTEXT:  while rechecking updated
> tuple (3,33) in relation "t_shared_liste_valeurs"
>
> 2019-03-01 14:53:37 CET [24803]: [132-1] [3686]
> user=pgbd_preint_sg2,db=pgbd_preint_sg2 STATEMENT:  update
> t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2,
> modification_date=$3, modification_login=$4, administrable=$5, libelle=$6,
> niveau=$7 where code=$8
>
> <<
>
> After a fresh db restart, the result is the same : no content of Bind
> variables in the log file.
>
> Best Regards
>
> Didier ROS
> Expertise SGBD
> EDF - DTEO - DSIT - IT DMA
> Département Solutions Groupe
> Groupe Performance Applicative
> 32 avenue Pablo Picasso
> 92000 NANTERRE

OK, In case of a trigger or any pl/pgsql program I would have tryed to write
the content of bind variables using RAISE command or someting similar before
executing the UPDATE command.

But your log is now much more explicit: you where waiting on a LOCK ...

Regards
PAscal



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-performance-f2050081.html

Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Justin Pryzby
In reply to this post by Did16
On Fri, Mar 01, 2019 at 06:47:06PM +0000, ROS Didier wrote:
> log_line_prefix = '%t [%p]: [%l-1] [%x] user=%u,db=%d,client=%h'

On Sat, Mar 02, 2019 at 01:14:44PM +0000, ROS Didier wrote:
> 2019-03-01 14:53:37 CET [24803]: [129-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 LOG:  process 24803 still waiting for ShareLock on transaction 3711 after 1000.476 ms
> 2019-03-01 14:53:37 CET [24803]: [130-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 DETAIL:  Process holding the lock: 24786. Wait queue: 24803.
> 2019-03-01 14:53:37 CET [24803]: [131-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 CONTEXT:  while rechecking updated tuple (3,33) in relation "t_shared_liste_valeurs"
> 2019-03-01 14:53:37 CET [24803]: [132-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 STATEMENT:  update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8

I just realized that your log is showing "STATEMENT:  [...]" which I think
means that's using libpq PQexec (simple query protocol), which means it doesn't
use or support bind parameters at all.  If it were using PQexecParams (protocol
2.0 "extended" query), it would show "execute <unnamed>: [...]", with any bind
params in DETAIL.  And if you were using PQexecPrepared, it'd show "execute
FOO: [...]" where FOO is the name of the statement "prepared" by PQprepare
(plus bind params).

https://www.postgresql.org/docs/current/libpq-exec.html
https://www.postgresql.org/docs/current/protocol.html

What client application is this ?  It looks like it's going to set
deletion_date to the literal string "$1" .. except that it's not quoted, so the
statement will just cause an error.  Am I wrong ?

Could you grep the entire logfile for pid 24803 and post the output on dropbox
or pastebin or show 10 lines of context by email ?

I've just used my messages and test cases on this patch as a reference to check
what I wrote above is accurate.
https://www.postgresql.org/message-id/flat/20190210015707.GQ31721%40telsasoft.com#037d17567f4c84a5f436960ef1ed8c49

On Fri, Mar 01, 2019 at 06:47:06PM +0000, ROS Didier wrote:
> *) -> suggestion : It would be nice to have the content of bind variable of a query in a table of pg_catalog. (cf ORACLE)

As I mentioned, you can set log_destination=csvlog,stderr and import them with
COPY (and add indices and analysis and monitoring..).  It look like DETAILs are
being logged, so that's not the issue, but CSV also has the nice benefit of
being easily imported to SQL where escaping and linebreaks and similar are not
confusing the issue, which I think can be the case for text logs.

Justin

Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Tom Lane-2
Justin Pryzby <[hidden email]> writes:
> On Fri, Mar 01, 2019 at 06:47:06PM +0000, ROS Didier wrote:
>> log_line_prefix = '%t [%p]: [%l-1] [%x] user=%u,db=%d,client=%h'

> On Sat, Mar 02, 2019 at 01:14:44PM +0000, ROS Didier wrote:
>> 2019-03-01 14:53:37 CET [24803]: [129-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 LOG:  process 24803 still waiting for ShareLock on transaction 3711 after 1000.476 ms
>> 2019-03-01 14:53:37 CET [24803]: [130-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 DETAIL:  Process holding the lock: 24786. Wait queue: 24803.
>> 2019-03-01 14:53:37 CET [24803]: [131-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 CONTEXT:  while rechecking updated tuple (3,33) in relation "t_shared_liste_valeurs"
>> 2019-03-01 14:53:37 CET [24803]: [132-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 STATEMENT:  update t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2, modification_date=$3, modification_login=$4, administrable=$5, libelle=$6, niveau=$7 where code=$8

> I just realized that your log is showing "STATEMENT:  [...]" which I think
> means that's using libpq PQexec (simple query protocol), which means it doesn't
> use or support bind parameters at all.

No, what's shown above is a case of the current statement being printed
as detail for some log message (a log_lock_waits message in this case).
This has nothing to do with whether statement logging is on overall.

I now realize that what the OP is probably wishing for is that bind
parameter values would be included as a detail line in messages other
than log_all_statements or statement-duration messages.  Sorry, that
feature doesn't exist, and there'd be pretty serious technical
impediments to making it happen.

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: How to get the content of Bind variables

Sergei Kornilov
In reply to this post by Did16
Hello

Postgresql does not log statement parameters on log_lock_wait. Because this is not implemented: https://github.com/postgres/postgres/blob/REL_10_STABLE/src/backend/storage/lmgr/proc.c#L1461
Compare with errdetail_params routine in this file: https://github.com/postgres/postgres/blob/REL_10_STABLE/src/backend/tcop/postgres.c#L1847 

Currently query parameters can be logged only at the end of successful query execution.

regards, Sergei

Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16
In reply to this post by Justin Pryzby
Hi
       
I have executed grep command on the entire logfile for pid 24803. See the attached file
NB : I have no DETAIL section in my entire log file. Is it normal ?

Best Reagrds

Didier ROS

-----Message d'origine-----
De : [hidden email] [mailto:[hidden email]]
Envoyé : samedi 2 mars 2019 16:57
À : ROS Didier <[hidden email]>
Cc : [hidden email]; [hidden email]; [hidden email]
Objet : Re: How to get the content of Bind variables

On Fri, Mar 01, 2019 at 06:47:06PM +0000, ROS Didier wrote:
> log_line_prefix = '%t [%p]: [%l-1] [%x] user=%u,db=%d,client=%h'

On Sat, Mar 02, 2019 at 01:14:44PM +0000, ROS Didier wrote:

> 2019-03-01 14:53:37 CET [24803]: [129-1] [3686]
> user=pgbd_preint_sg2,db=pgbd_preint_sg2 LOG:  process 24803 still
> waiting for ShareLock on transaction 3711 after 1000.476 ms
> 2019-03-01 14:53:37 CET [24803]: [130-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 DETAIL:  Process holding the lock: 24786. Wait queue: 24803.
> 2019-03-01 14:53:37 CET [24803]: [131-1] [3686] user=pgbd_preint_sg2,db=pgbd_preint_sg2 CONTEXT:  while rechecking updated tuple (3,33) in relation "t_shared_liste_valeurs"
> 2019-03-01 14:53:37 CET [24803]: [132-1] [3686]
> user=pgbd_preint_sg2,db=pgbd_preint_sg2 STATEMENT:  update
> t_shared_liste_valeurs set deletion_date=$1, deletion_login=$2,
> modification_date=$3, modification_login=$4, administrable=$5,
> libelle=$6, niveau=$7 where code=$8
I just realized that your log is showing "STATEMENT:  [...]" which I think means that's using libpq PQexec (simple query protocol), which means it doesn't use or support bind parameters at all.  If it were using PQexecParams (protocol
2.0 "extended" query), it would show "execute <unnamed>: [...]", with any bind params in DETAIL.  And if you were using PQexecPrepared, it'd show "execute
FOO: [...]" where FOO is the name of the statement "prepared" by PQprepare (plus bind params).

https://www.postgresql.org/docs/current/libpq-exec.html
https://www.postgresql.org/docs/current/protocol.html

What client application is this ?  It looks like it's going to set deletion_date to the literal string "$1" .. except that it's not quoted, so the statement will just cause an error.  Am I wrong ?

Could you grep the entire logfile for pid 24803 and post the output on dropbox or pastebin or show 10 lines of context by email ?

I've just used my messages and test cases on this patch as a reference to check what I wrote above is accurate.
https://www.postgresql.org/message-id/flat/20190210015707.GQ31721%40telsasoft.com#037d17567f4c84a5f436960ef1ed8c49

On Fri, Mar 01, 2019 at 06:47:06PM +0000, ROS Didier wrote:
> *) -> suggestion : It would be nice to have the content of bind
> variable of a query in a table of pg_catalog. (cf ORACLE)

As I mentioned, you can set log_destination=csvlog,stderr and import them with COPY (and add indices and analysis and monitoring..).  It look like DETAILs are being logged, so that's not the issue, but CSV also has the nice benefit of being easily imported to SQL where escaping and linebreaks and similar are not confusing the issue, which I think can be the case for text logs.

Justin



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

log_tirs_perf.txt (101K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: How to get the content of Bind variables

Did16
In reply to this post by Sergei Kornilov
Hi Sergei

        Thank you for your explanation. I can understand for the lock wait message, but I have no DETAIL section in my entire log file. Why ?
              I have plenty of STATEMENT sections ...

        Thanks in advance

Best Regards
Didier ROS

-----Message d'origine-----
De : [hidden email] [mailto:[hidden email]]
Envoyé : samedi 2 mars 2019 17:34
À : ROS Didier <[hidden email]>; [hidden email]; [hidden email]
Objet : Re: How to get the content of Bind variables

Hello

Postgresql does not log statement parameters on log_lock_wait. Because this is not implemented: https://github.com/postgres/postgres/blob/REL_10_STABLE/src/backend/storage/lmgr/proc.c#L1461
Compare with errdetail_params routine in this file: https://github.com/postgres/postgres/blob/REL_10_STABLE/src/backend/tcop/postgres.c#L1847 

Currently query parameters can be logged only at the end of successful query execution.

regards, Sergei



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.