Pg_restore failing due to data inconsistency in dump file

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Pg_restore failing due to data inconsistency in dump file

soumitra
Hello Team , 

I have encountered following issue while restoring the Postgres dump using pg_restore . Please provide any suggestion to the same .
  1. This dump was taken using bdr_dump as bdr enabled database and all the BDR configuration and extension was excluded .
  2. DUMP was taken in a running DB server where BDR was running .
  3. Postgres Dump was taken in .dump format .
  4. During restoration of the same using pg_restore getting following error . 
Error : 

pg_restore: [archiver (db)] Error from TOC entry 3061; 2606 17551 FK CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_attribute" violates foreign key constraint "fk_5hrm2vlf9ql5fu043kqepovbr"
DETAIL: Key (user_id)=(1f34ec07-7087-45ff-8a56-f36177752499) is not present in table "user_entity".
Command was: ALTER TABLE ONLY user_attribute
ADD CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr FOREIGN KEY (user_id) REFERENCES user_entity...
pg_restore: [archiver (db)] Error from TOC entry 3065; 2606 17601 FK CONSTRAINT fk_c4fqv34p1mbylloxang7b1q3l DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_role_mapping" violates foreign key constraint "fk_c4fqv34p1mbylloxang7b1q3l"
DETAIL: Key (user_id)=(24d1ee2a-b05e-4c30-a292-84045c47611e) is not present in table "user_entity".


Thanks , 

Soumitra 


Soumitra Bhandary Postgres Database Administrator
Reply | Threaded
Open this post in threaded view
|

Re: Pg_restore failing due to data inconsistency in dump file

Shreeyansh dba
Hi Soumitra,

Normally this type of error encounter while insert or update data on the table, which is dependable on the other table data.

To avoid such error you should disable foreign keys before insert data and enable it later.



On Thu, Dec 13, 2018 at 4:50 PM soumitra bhandary <[hidden email]> wrote:
Hello Team , 

I have encountered following issue while restoring the Postgres dump using pg_restore . Please provide any suggestion to the same .
  1. This dump was taken using bdr_dump as bdr enabled database and all the BDR configuration and extension was excluded .
  2. DUMP was taken in a running DB server where BDR was running .
  3. Postgres Dump was taken in .dump format .
  4. During restoration of the same using pg_restore getting following error . 
Error : 

pg_restore: [archiver (db)] Error from TOC entry 3061; 2606 17551 FK CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_attribute" violates foreign key constraint "fk_5hrm2vlf9ql5fu043kqepovbr"
DETAIL: Key (user_id)=(1f34ec07-7087-45ff-8a56-f36177752499) is not present in table "user_entity".
Command was: ALTER TABLE ONLY user_attribute
ADD CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr FOREIGN KEY (user_id) REFERENCES user_entity...
pg_restore: [archiver (db)] Error from TOC entry 3065; 2606 17601 FK CONSTRAINT fk_c4fqv34p1mbylloxang7b1q3l DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_role_mapping" violates foreign key constraint "fk_c4fqv34p1mbylloxang7b1q3l"
DETAIL: Key (user_id)=(24d1ee2a-b05e-4c30-a292-84045c47611e) is not present in table "user_entity".


Thanks , 

Soumitra 


Reply | Threaded
Open this post in threaded view
|

Re: Pg_restore failing due to data inconsistency in dump file

soumitra
Hi , 

Thanks for the response . 

But this one is third party application and dump is restored from dump file not plain text file . So is there any way to control it ?

Thanks,
Soumitra 

From: Shreeyansh Dba <[hidden email]>
Sent: Thursday, December 13, 2018 5:51 PM
To: [hidden email]
Cc: pgsql-admin
Subject: Re: Pg_restore failing due to data inconsistency in dump file
 
Hi Soumitra,

Normally this type of error encounter while insert or update data on the table, which is dependable on the other table data.

To avoid such error you should disable foreign keys before insert data and enable it later.



On Thu, Dec 13, 2018 at 4:50 PM soumitra bhandary <[hidden email]> wrote:
Hello Team , 

I have encountered following issue while restoring the Postgres dump using pg_restore . Please provide any suggestion to the same .
  1. This dump was taken using bdr_dump as bdr enabled database and all the BDR configuration and extension was excluded .
  2. DUMP was taken in a running DB server where BDR was running .
  3. Postgres Dump was taken in .dump format .
  4. During restoration of the same using pg_restore getting following error . 
Error : 

pg_restore: [archiver (db)] Error from TOC entry 3061; 2606 17551 FK CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_attribute" violates foreign key constraint "fk_5hrm2vlf9ql5fu043kqepovbr"
DETAIL: Key (user_id)=(1f34ec07-7087-45ff-8a56-f36177752499) is not present in table "user_entity".
Command was: ALTER TABLE ONLY user_attribute
ADD CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr FOREIGN KEY (user_id) REFERENCES user_entity...
pg_restore: [archiver (db)] Error from TOC entry 3065; 2606 17601 FK CONSTRAINT fk_c4fqv34p1mbylloxang7b1q3l DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_role_mapping" violates foreign key constraint "fk_c4fqv34p1mbylloxang7b1q3l"
DETAIL: Key (user_id)=(24d1ee2a-b05e-4c30-a292-84045c47611e) is not present in table "user_entity".


Thanks , 

Soumitra 


Soumitra Bhandary Postgres Database Administrator
Reply | Threaded
Open this post in threaded view
|

Re: Pg_restore failing due to data inconsistency in dump file

Shreeyansh dba
You can use --disable-trigger option with pg_restore utility while performing the restoration.




On Thu, Dec 13, 2018 at 5:56 PM soumitra bhandary <[hidden email]> wrote:
Hi , 

Thanks for the response . 

But this one is third party application and dump is restored from dump file not plain text file . So is there any way to control it ?

Thanks,
Soumitra 

From: Shreeyansh Dba <[hidden email]>
Sent: Thursday, December 13, 2018 5:51 PM
To: [hidden email]
Cc: pgsql-admin
Subject: Re: Pg_restore failing due to data inconsistency in dump file
 
Hi Soumitra,

Normally this type of error encounter while insert or update data on the table, which is dependable on the other table data.

To avoid such error you should disable foreign keys before insert data and enable it later.



On Thu, Dec 13, 2018 at 4:50 PM soumitra bhandary <[hidden email]> wrote:
Hello Team , 

I have encountered following issue while restoring the Postgres dump using pg_restore . Please provide any suggestion to the same .
  1. This dump was taken using bdr_dump as bdr enabled database and all the BDR configuration and extension was excluded .
  2. DUMP was taken in a running DB server where BDR was running .
  3. Postgres Dump was taken in .dump format .
  4. During restoration of the same using pg_restore getting following error . 
Error : 

pg_restore: [archiver (db)] Error from TOC entry 3061; 2606 17551 FK CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_attribute" violates foreign key constraint "fk_5hrm2vlf9ql5fu043kqepovbr"
DETAIL: Key (user_id)=(1f34ec07-7087-45ff-8a56-f36177752499) is not present in table "user_entity".
Command was: ALTER TABLE ONLY user_attribute
ADD CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr FOREIGN KEY (user_id) REFERENCES user_entity...
pg_restore: [archiver (db)] Error from TOC entry 3065; 2606 17601 FK CONSTRAINT fk_c4fqv34p1mbylloxang7b1q3l DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_role_mapping" violates foreign key constraint "fk_c4fqv34p1mbylloxang7b1q3l"
DETAIL: Key (user_id)=(24d1ee2a-b05e-4c30-a292-84045c47611e) is not present in table "user_entity".


Thanks , 

Soumitra 


Reply | Threaded
Open this post in threaded view
|

Re: Pg_restore failing due to data inconsistency in dump file

Ron-2
In reply to this post by Shreeyansh dba
No.  That is not the first thing to do.

OP needs to find out how the corruption got into the source in the first place.   There's probably corruption in one of the indexes in user_role_mapping and user_entity which support fk_5hrm2vlf9ql5fu043kqepovbr.


On 12/13/2018 06:21 AM, Shreeyansh Dba wrote:
Hi Soumitra,

Normally this type of error encounter while insert or update data on the table, which is dependable on the other table data.

To avoid such error you should disable foreign keys before insert data and enable it later.



On Thu, Dec 13, 2018 at 4:50 PM soumitra bhandary <[hidden email]> wrote:
Hello Team , 

I have encountered following issue while restoring the Postgres dump using pg_restore . Please provide any suggestion to the same .
  1. This dump was taken using bdr_dump as bdr enabled database and all the BDR configuration and extension was excluded .
  2. DUMP was taken in a running DB server where BDR was running .
  3. Postgres Dump was taken in .dump format .
  4. During restoration of the same using pg_restore getting following error . 
Error : 

pg_restore: [archiver (db)] Error from TOC entry 3061; 2606 17551 FK CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_attribute" violates foreign key constraint "fk_5hrm2vlf9ql5fu043kqepovbr"
DETAIL: Key (user_id)=(1f34ec07-7087-45ff-8a56-f36177752499) is not present in table "user_entity".
Command was: ALTER TABLE ONLY user_attribute
ADD CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr FOREIGN KEY (user_id) REFERENCES user_entity...
pg_restore: [archiver (db)] Error from TOC entry 3065; 2606 17601 FK CONSTRAINT fk_c4fqv34p1mbylloxang7b1q3l DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_role_mapping" violates foreign key constraint "fk_c4fqv34p1mbylloxang7b1q3l"
DETAIL: Key (user_id)=(24d1ee2a-b05e-4c30-a292-84045c47611e) is not present in table "user_entity".


Thanks , 

Soumitra 



--
Angular momentum makes the world go 'round.
Reply | Threaded
Open this post in threaded view
|

Re: Pg_restore failing due to data inconsistency in dump file

soumitra
In reply to this post by Shreeyansh dba
Hi 

Tried with that still it is failing and restore process is getting aborted. 

Thanks
Soumitra 

Sent from my iPhone

On 13-Dec-2018, at 6:07 PM, Shreeyansh Dba <[hidden email]> wrote:

You can use --disable-trigger option with pg_restore utility while performing the restoration.




On Thu, Dec 13, 2018 at 5:56 PM soumitra bhandary <[hidden email]> wrote:
Hi , 

Thanks for the response . 

But this one is third party application and dump is restored from dump file not plain text file . So is there any way to control it ?

Thanks,
Soumitra 

From: Shreeyansh Dba <[hidden email]>
Sent: Thursday, December 13, 2018 5:51 PM
To: [hidden email]
Cc: pgsql-admin
Subject: Re: Pg_restore failing due to data inconsistency in dump file
 
Hi Soumitra,

Normally this type of error encounter while insert or update data on the table, which is dependable on the other table data.

To avoid such error you should disable foreign keys before insert data and enable it later.



On Thu, Dec 13, 2018 at 4:50 PM soumitra bhandary <[hidden email]> wrote:
Hello Team , 

I have encountered following issue while restoring the Postgres dump using pg_restore . Please provide any suggestion to the same .
  1. This dump was taken using bdr_dump as bdr enabled database and all the BDR configuration and extension was excluded .
  2. DUMP was taken in a running DB server where BDR was running .
  3. Postgres Dump was taken in .dump format .
  4. During restoration of the same using pg_restore getting following error . 
Error : 

pg_restore: [archiver (db)] Error from TOC entry 3061; 2606 17551 FK CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_attribute" violates foreign key constraint "fk_5hrm2vlf9ql5fu043kqepovbr"
DETAIL: Key (user_id)=(1f34ec07-7087-45ff-8a56-f36177752499) is not present in table "user_entity".
Command was: ALTER TABLE ONLY user_attribute
ADD CONSTRAINT fk_5hrm2vlf9ql5fu043kqepovbr FOREIGN KEY (user_id) REFERENCES user_entity...
pg_restore: [archiver (db)] Error from TOC entry 3065; 2606 17601 FK CONSTRAINT fk_c4fqv34p1mbylloxang7b1q3l DBNAME
pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "user_role_mapping" violates foreign key constraint "fk_c4fqv34p1mbylloxang7b1q3l"
DETAIL: Key (user_id)=(24d1ee2a-b05e-4c30-a292-84045c47611e) is not present in table "user_entity".


Thanks , 

Soumitra 


Soumitra Bhandary Postgres Database Administrator
Reply | Threaded
Open this post in threaded view
|

Re: Pg_restore failing due to data inconsistency in dump file

David G Johnston
In reply to this post by soumitra
On Thu, Dec 13, 2018 at 7:26 AM soumitra bhandary <[hidden email]> wrote:
But this one is third party application and dump is restored from dump file not plain text file . So is there any way to control it ?

Your backup file (and the source database is comes from) is corrupted.

You need to decide whether you want to restore that backup anyway; or just throw it away, fix the source database, and generate a new backup file.

If you want to do the former you will want to restore schema only.  Modify the schema by dropping the foreign key.  Restore the data.  Fix the newly restored data.  Recreate the foreign key.

David J.