Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl

January 2019
Automation & Management of
PostgreSQL with ClusterControl
Sebastian Insausti
Presenter
sebastian@severalnines.com

Copyright 2018 Severalnines AB
I'm Jean-Jérôme from the Severalnines Team and
I'm your host for today's webinar!
Feel free to ask any questions in the Questions
section of this application or via the Chat box.
You can also contact me directly via the chat box
or via email: info@severalnines.com during or
after the webinar.
Your host & some logistics

Free to download
Initial 30 days Enterprise trial
Converts into free Community Edition
Enterprise / paid versions available

Automation & Management
Deployment (Free Community)
● Deploy a Cluster in Minutes
○ On-Prem
○ Cloud (AWS/Azure/Google) - paid
Monitoring (Free Community)
● Systems View with 1 sec Resolution
● DB / OS stats & Performance Advisors
● Configurable Dashboards
● Query Analyzer
● Agentless & agent-based
Management (Paid Features)
● Backup Management
● Upgrades & Patching
● Security & Compliance
● Operational Reports
● Automatic Recovery & Repair
● Performance Management
● Automatic Performance Advisors

Supported Databases

Our Customers

Poll - multiple choice
Which method do you use to perform your DBA tasks?
● Manual actions
● Custom scripts
● External tools and utilities
● ClusterControl
● Others (please specify in the chat box)

Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Managing PostgreSQL
● Requirements for managing production
databases
“The Old Way”
● Labour intensive, manual operations
● Scripts, tools and utilities
● What happens when the DBA leaves?
Agenda
Automation and Management with ClusterControl
● Deployment
● Monitoring & Alerting
● Backup management
● High Availability setups
● Single VIP endpoint with Load Balancer
● Automatic Failover
Live Demo

Requirements for managing production
databases

Deployment
● Install
○ The first step to begin the game
● Secure
○ Keep your data safe and secure
● Configure
○ Environment requirements

Monitoring & Alerting
● Define metrics
○ Database
○ Operating System
● Configure monitoring
○ Custom Monitoring
○ System Monitoring
● Configure alerts
○ Email alerts
○ Tool Integrations

● Important but complex task
● PostgreSQL is configured with compatibility and stability by default
● Depends on the hardware and on the system itself
● Performance queries
Performance Management

● Backups Methods:
○ Logical backup: Plain text files with SQL statements
○ Physical backup: A copy of the datafiles
● Backup tasks:
○ Schedule
○ Rotation / Retention
○ Secure Backup
○ Verify Backup
○ Point-in-time recovery backup
○ Upload to external datacenter, e.g., cloud storage
○ Reporting
Backup Management

● Define topology
○ Master-Slave
○ Master-Master
● Define replication type
○ Synchronous: There is no loss of data even if our master node is lost. The commits
in the master must wait for a confirmation from the slave.
○ Asynchronous: There is a possibility of data loss in case we lose our master node. If
the replica is not updated at the time of the incident, the information that has not
been copied may be lost.
○ Logical Replication: Performed via SQL statements.
○ Physical Replication: Performed via internal data structure.
● Failover
● Single endpoint for application
High Availability setups

Poll - multiple choice
According to you, what are your most challenging tasks in production?
● A well tuned and secure configuration
● Monitoring uptime & Alerting
● Managing performance / queries
● Backup management
● Failover & High Availability

"The Old Way"

Deployment
● Installation:
○ Repositories or packages
○ Some packages dependencies could be required
● Secure installation:
○ Configuration files (pg_hba.conf or postgresql.conf)
○ Authentication
● Basic configuration:
○ For example: Listen address or wal_level
○ Require DBA or Sysadmin skills

What to monitor
● Database
○ Queries
○ Sessions
○ Locks
○ IO
○ Replication
○ Backups
● System
○ CPU
○ Memory
○ Load Average
○ Network

● Time consuming task
● There is not a standard configuration for this task
● Check charts
○ e.g: max_connections
● Database commands
○ EXPLAIN: Check the query plan
○ VACUUM: For several maintenance tasks
● Parameters
○ shared_buffers: Amount of memory that the database uses for shared memory
○ work_mem: Amount of memory used by internal operations like ORDER BY, DISTINCT, JOIN,
among others
● Require deep DBA knowledge

Backup Management
● Typically done using custom scripts
● Some just copy the datafiles (!)
● Schedule using cron
● Manual rotation
● No backup verification or PITR backup

● Master-Slave Topology
● Asynchronous replication
● Manual setup of slave
○ Install a new server
○ Copy data from master
○ Configure replication
High Availability Setups

Automatic Failover
● We need to know when we need the failover
● PostgreSQL itself does not include an automatic failover mechanism
● After failover, the applications need to know about the new master

● Load Balancer:
○ Balancing traffic
○ Redirect traffic to available nodes
○ Specify ports for different roles
○ It's an external component that requires installation,
configuration and monitoring
● Virtual IP:
○ Keepalived is a linux service that allows to configure a
virtual IP within an active/passive group of servers
○ Another external component that requires installation,
configuration and monitoring
Single VIP endpoint with Load Balancer

● Where is the documentation?
● How do all these scripts hang together?
● It is often hard to know how the database is managed
● There is nobody to take over what you did
● It can be a mess
What happens when the DBA leaves?

ClusterControl - PostgreSQL automation
and management

ClusterControl

Deployment

Backup Management

High Availability Setups

Single VIP endpoint with Load Balancer

Automatic Failover

ClusterControl CLI
$ s9s cluster --list --long
ID STATE TYPE OWNER GROUP NAME COMMENT
1 FAILURE galera system admins Galera1 Cluster failed to start.
2 STARTED postgresql_single system admins PostgreSQL1 All nodes are operational.
5 FAILURE replication system admins MySQL1 Cluster failed to start.
Total: 3
$ s9s node --list --cluster-id=2 --long
STAT VERSION CID CLUSTER HOST PORT COMMENT
coC- 1.7.1.2965 2 PostgreSQL1 192.168.100.110 9500 Up and running.
?o-- 2.3.2 2 PostgreSQL1 192.168.100.110 9090 Process 'prometheus' is running.
ho-- 1.5.18 2 PostgreSQL1 192.168.100.114 9600 Process 'haproxy' is running.
koM- 1.3 2 PostgreSQL1 192.168.100.114 112 Process 'keepalived' is running.
ho-- 1.5.18 2 PostgreSQL1 192.168.100.115 9600 Process 'haproxy' is running.
ko-- 1.3 2 PostgreSQL1 192.168.100.115 112 Process 'keepalived' is running.
poM- 10.5 2 PostgreSQL1 192.168.100.123 5432 Up and running.
poS- 10.5 2 PostgreSQL1 192.168.100.124 5432 Up and running.
poS- 10.5 2 PostgreSQL1 192.168.100.125 5432 Up and running.
Total: 9

Live Demo

● Deployment
● Add replication slave
● Add load balancer
● Deploy keepalived
● Monitoring
● Performance
● Backup
● Topology view
● CLI
Cluster Control PostgreSQL Management
Demo

Poll - multiple choice (choose one of below)
Where do you need the most improvements in your environment?
● Monitoring & Alerting
● Backups
● High Availability & Automatic Failover
● Other (please specify in the chat box)

Q & A

Links & Resources
● ClusterControl for PostgreSQL
● Download / install ClusterControl
● ClusterControl Community Edition Page
● Contact us: info@severalnines.com

Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl

Similar to Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl (20)

More from Severalnines

More from Severalnines (12)

Recently uploaded

Recently uploaded (20)

Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl