Aurora PostgreSQL Support

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Aurora PostgreSQL Support

Rakesh T
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

dbatoCloud Solution
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

Rakesh T
Thank Ashok. I will note this point.

In fact it would be great if you can highlight the pain points? Also additionally what additional support factors do you think we can help with BU considering majority of things are automated in Aurora PostgreSQL?

Thanks again!


Thanks,
Goti


On Thu, Feb 18, 2021 at 11:33 AM dbatoCloud Solution <[hidden email]> wrote:
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

dbatoCloud Solution
I am working on performance tuning activities. Our people migrated data from AWS redshift to Aurora PostgreSQL.  Please give me some time I will highlight the parameters for now I am saying you will always get error like that you don't have access to RDS_SUPERUSER all the time.


On Thu, Feb 18, 2021, 11:39 Rakesh T <[hidden email]> wrote:
Thank Ashok. I will note this point.

In fact it would be great if you can highlight the pain points? Also additionally what additional support factors do you think we can help with BU considering majority of things are automated in Aurora PostgreSQL?

Thanks again!


Thanks,
Goti


On Thu, Feb 18, 2021 at 11:33 AM dbatoCloud Solution <[hidden email]> wrote:
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

Prince Pathria
In reply to this post by dbatoCloud Solution
One of the issues I faced in the past is limited(not expandable) tmp storage. If your query is huge, it may fail.
AWS docs don't tell such things explicitly so have intensive testing on Aurora before planning to migrate.
RDS seems to be a safe choice.

Quoting from the doc - "Each Aurora DB instance contains a limited amount of local storage that is determined by the DB instance class. Typically, the amount of local storage is twice the amount of memory on the DB instance."
ref. https://aws.amazon.com/premiumsupport/knowledge-center/postgresql-aurora-storage-issue/ 

Happy to help :)
Prince Pathria Cloud Engineer II| Certified Kubernetes Administrator | AWS Certified Solutions Architect Evive +91 9478670472 goevive.com


On Thu, Feb 18, 2021 at 11:33 AM dbatoCloud Solution <[hidden email]> wrote:
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

dbatoCloud Solution
Dear Rakesh

Prince is correct. 

@Rakesh T # The below parameter value cannot be changed in aurora database. I tried to change in parameter group level but there is no option even I checked with AWS 
It is not possible to change this setting. In our environment is Dataware where we are having huge DML,Select & I/O happening I asked AWS to support/give Dataware house best practice
so that I can follow but no support from AWS.  

min_wal_size
max_wal_size
effective_io_concurrency
checkpoint_completion_target

 

It can be done in the community version.


My suggestion better to go-ahead with RDS-PostgreSQL, or EC2-> PostgreSQL but not Aurora postgreSQL.



 

Thanks & Best Wishes,

Ashok

 

---------------------------------------------------------------------------------------------------------------------

Ashokkumar Mani  (OCP12c/11g/10g/9i, AWS SAA, M103)

Dubai , UAE | BLR , INDIA

M: +971 54 723 0075 | +91 90086 70302 | WApp : +91 81975 99922

W: https://dbatocloudsolution.blogspot.in/ | E: [hidden email]



On Thu, Feb 18, 2021 at 11:47 AM Prince Pathria <[hidden email]> wrote:
One of the issues I faced in the past is limited(not expandable) tmp storage. If your query is huge, it may fail.
AWS docs don't tell such things explicitly so have intensive testing on Aurora before planning to migrate.
RDS seems to be a safe choice.

Quoting from the doc - "Each Aurora DB instance contains a limited amount of local storage that is determined by the DB instance class. Typically, the amount of local storage is twice the amount of memory on the DB instance."
ref. https://aws.amazon.com/premiumsupport/knowledge-center/postgresql-aurora-storage-issue/ 

Happy to help :)
Prince Pathria Cloud Engineer II| Certified Kubernetes Administrator | AWS Certified Solutions Architect Evive +91 9478670472 goevive.com


On Thu, Feb 18, 2021 at 11:33 AM dbatoCloud Solution <[hidden email]> wrote:
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

Avinash Kumar
If you are ready for some Bugs that you don't find in Community PostgreSQL, then you may feel free to think of it. 

One of the latest unfixed Aurora PostgreSQL 12 bugs is reproduced and documented in this blog post. 





On Fri, Feb 19, 2021 at 8:34 AM dbatoCloud Solution <[hidden email]> wrote:
Dear Rakesh

Prince is correct. 

@Rakesh T # The below parameter value cannot be changed in aurora database. I tried to change in parameter group level but there is no option even I checked with AWS 
It is not possible to change this setting. In our environment is Dataware where we are having huge DML,Select & I/O happening I asked AWS to support/give Dataware house best practice
so that I can follow but no support from AWS.  

min_wal_size
max_wal_size
effective_io_concurrency
checkpoint_completion_target

 

It can be done in the community version.


My suggestion better to go-ahead with RDS-PostgreSQL, or EC2-> PostgreSQL but not Aurora postgreSQL.



 

Thanks & Best Wishes,

Ashok

 

---------------------------------------------------------------------------------------------------------------------

Ashokkumar Mani  (OCP12c/11g/10g/9i, AWS SAA, M103)

Dubai , UAE | BLR , INDIA

M: +971 54 723 0075 | +91 90086 70302 | WApp : +91 81975 99922

W: https://dbatocloudsolution.blogspot.in/ | E: [hidden email]



On Thu, Feb 18, 2021 at 11:47 AM Prince Pathria <[hidden email]> wrote:
One of the issues I faced in the past is limited(not expandable) tmp storage. If your query is huge, it may fail.
AWS docs don't tell such things explicitly so have intensive testing on Aurora before planning to migrate.
RDS seems to be a safe choice.

Quoting from the doc - "Each Aurora DB instance contains a limited amount of local storage that is determined by the DB instance class. Typically, the amount of local storage is twice the amount of memory on the DB instance."
ref. https://aws.amazon.com/premiumsupport/knowledge-center/postgresql-aurora-storage-issue/ 

Happy to help :)
Prince Pathria Cloud Engineer II| Certified Kubernetes Administrator | AWS Certified Solutions Architect Evive +91 9478670472 goevive.com


On Thu, Feb 18, 2021 at 11:33 AM dbatoCloud Solution <[hidden email]> wrote:
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti


--
Regards,
Avinash Vallarapu
MigOps Inc.
CEO. 
Reply | Threaded
Open this post in threaded view
|

Re: Aurora PostgreSQL Support

Dan Smith
Hi Goti,

I would suggest generating a report using the aws schema migration tool to see what ddl might need to be changed or rewritten).

Some Aurora configuration is determined by instance size such as number of max connections.  The rest will be in the cluster and instance parameter groups (which are tied to Postgres compatibility mode).

I would absolutely recommend creating cluster and instance parameter groups first and with automation.  This costs nothing and will help you determine what parameters you might be interested in changing.

As to other comments in this thread about migrating a distributed columnar store with huge page sizes (aka Redshift) to Aurora Postgres...  I can only speculate wrong tool for the wrong job.  Aurora is optimized for OLTP type workloads.  So, Aurora would excel for a write heavy workload such as collecting IoT events but I would expect it to underperform Redshift for analytics at scale.

There seem to be some Aurora Postgres haters out there, but, I'm not one at all.  I have been using and administrating Postgres for a number of years and have few complaints about Postgres or Postgres Aurora.  Easy to automate cluster build with Terraform to include configuration, and some level of monitoring on infrastructure (GruntWorks modules are an ok starting point).

Some additional observability and monitoring is needed after provisioning. 

The only issues I have ever had with Aurora Postgres were: 
1) Failed to upgrade major version.  Fixed by adding new nodes to cluster, removing old nodes and running upgrade.

2) Configuration drift in Cluster Parameter group where old default value impacted some client connections default_tablespace if memory serves). Fix, set value to blank as this is valid default for Postgres.

All other issues were run of the mill issues where I was quickly able to help developers out.  Hope this helps.


Best regards,

Dan Smith

On Fri, Feb 19, 2021, 12:52 Avinash Kumar <[hidden email]> wrote:
If you are ready for some Bugs that you don't find in Community PostgreSQL, then you may feel free to think of it. 

One of the latest unfixed Aurora PostgreSQL 12 bugs is reproduced and documented in this blog post. 





On Fri, Feb 19, 2021 at 8:34 AM dbatoCloud Solution <[hidden email]> wrote:
Dear Rakesh

Prince is correct. 

@Rakesh T # The below parameter value cannot be changed in aurora database. I tried to change in parameter group level but there is no option even I checked with AWS 
It is not possible to change this setting. In our environment is Dataware where we are having huge DML,Select & I/O happening I asked AWS to support/give Dataware house best practice
so that I can follow but no support from AWS.  

min_wal_size
max_wal_size
effective_io_concurrency
checkpoint_completion_target

 

It can be done in the community version.


My suggestion better to go-ahead with RDS-PostgreSQL, or EC2-> PostgreSQL but not Aurora postgreSQL.



 

Thanks & Best Wishes,

Ashok

 

---------------------------------------------------------------------------------------------------------------------

Ashokkumar Mani  (OCP12c/11g/10g/9i, AWS SAA, M103)

Dubai , UAE | BLR , INDIA

M: +971 54 723 0075 | +91 90086 70302 | WApp : +91 81975 99922

W: https://dbatocloudsolution.blogspot.in/ | E: [hidden email]



On Thu, Feb 18, 2021 at 11:47 AM Prince Pathria <[hidden email]> wrote:
One of the issues I faced in the past is limited(not expandable) tmp storage. If your query is huge, it may fail.
AWS docs don't tell such things explicitly so have intensive testing on Aurora before planning to migrate.
RDS seems to be a safe choice.

Quoting from the doc - "Each Aurora DB instance contains a limited amount of local storage that is determined by the DB instance class. Typically, the amount of local storage is twice the amount of memory on the DB instance."
ref. https://aws.amazon.com/premiumsupport/knowledge-center/postgresql-aurora-storage-issue/ 

Happy to help :)
Prince Pathria Cloud Engineer II| Certified Kubernetes Administrator | AWS Certified Solutions Architect Evive +91 9478670472 goevive.com


On Thu, Feb 18, 2021 at 11:33 AM dbatoCloud Solution <[hidden email]> wrote:
Hi Rakesh
I would suggest to go for EC2 then install PostgreSQL then use it. I am facing lot of problems in Aurora PostgreSQL

Thanks
Ashok

On Thu, Feb 18, 2021, 10:04 Rakesh T <[hidden email]> wrote:
Hi Listeres,

I am looking for suggestions from people who have experience with Aurora PostgreSQL. 

I work as a database admin in my company. We are migrating some of our Oracle databases to Aurora PostgreSQL. Since this falls under the RDS category I understand that the majority of the admin work will be managed by AWS team itself. But from a business unit perspective, they are looking for support from my team with regards to general BU work and performance and tuning. Can someone shed some light on possible areas which we can support in the RDS Aurora PostgreSQL world?

Thanks in Advance!

Thanks,
Goti


--
Regards,
Avinash Vallarapu
MigOps Inc.
CEO.