Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch metrics for multiple databases #88

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Fetch metrics for multiple databases #88

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
01474e1
allow to get metrics for multiple databases
ydkn Apr 30, 2019
fd1a376
* add usage examples for multiple databases
ydkn Apr 30, 2019
3e2aae4
* use "," as list separator for database names
ydkn Jan 14, 2020
404fb81
fix missing wrong number of arguments for statsio and statstable metrics
ydkn Jan 14, 2020
53334f5
Merge remote-tracking branch 'upstream/master' into multiple_database…
ydkn Jan 14, 2020
2b7e80e
Update CHANGELOG.md
ydkn Jan 15, 2020
fb711d3
Update CHANGELOG.md
ydkn Jan 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ This project adheres to [Semantic Versioning](http://semver.org/).
This CHANGELOG follows the format listed [here](https://github.com/sensu-plugins/community/blob/master/HOW_WE_CHANGELOG.md).

## [Unreleased]
### Changed
- Option `database (-d, --db)` for metrics now supports semicolon separated list of databases to fetch metrics of multiple databases at once.
ydkn marked this conversation as resolved.
Show resolved Hide resolved
### Breaking Changes
- If no `database (-d, --db)` argument is supplied for the metric bins through the command line now metrics for all databases are returned.
ydkn marked this conversation as resolved.
Show resolved Hide resolved

## [4.0.0] - 2020-01-09

Expand Down
55 changes: 31 additions & 24 deletions bin/metric-postgres-connections.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
#
# USAGE:
# ./metric-postgres-connections.rb -u db_user -p db_pass -h db_host -d db
# ./metric-postgres-connections.rb -u db_user -p db_pass -h db_host -d 'db1,db2'
#
# NOTES:
#
Expand All @@ -31,6 +32,7 @@
#

require 'sensu-plugins-postgres/pgpass'
require 'sensu-plugins-postgres/pgdatabases'
require 'sensu-plugin/metric/cli'
require 'pg'
require 'socket'
Expand Down Expand Up @@ -62,10 +64,11 @@ class PostgresStatsDBMetrics < Sensu::Plugin::Metric::CLI::Graphite
short: '-P PORT',
long: '--port PORT'

option :database,
description: 'Database name',
option :databases,
description: 'Database names, separated by ","',
short: '-d DB',
long: '--db DB'
long: '--db DB',
default: nil

option :scheme,
description: 'Metric naming scheme, text to prepend to $queue_name.$metric',
Expand All @@ -79,12 +82,14 @@ class PostgresStatsDBMetrics < Sensu::Plugin::Metric::CLI::Graphite
default: nil

include Pgpass
include Pgdatabases

def run
timestamp = Time.now.to_i
pgpass
databases = pgdatabases
con = PG.connect(host: config[:hostname],
dbname: config[:database],
dbname: databases.first,
user: config[:user],
password: config[:password],
port: config[:port],
Expand All @@ -98,30 +103,32 @@ def run
]
wait_col = con.exec(request.join(' ')).first['wait_col']

request = [
"select count(*), #{wait_col} as waiting from pg_stat_activity",
"where datname = '#{config[:database]}' group by #{wait_col}"
]

metrics = {
active: 0,
waiting: 0,
total: 0
}
con.exec(request.join(' ')) do |result|
result.each do |row|
if row['waiting'] == 't'
metrics[:waiting] = row['count']
elsif row['waiting'] == 'f'
metrics[:active] = row['count']
databases.each do |database|
request = [
"select count(*), #{wait_col} as waiting from pg_stat_activity",
"where datname = '#{con.escape_string(database)}' group by #{wait_col}"
]

metrics = {
active: 0,
waiting: 0,
total: 0
}
con.exec(request.join(' ')) do |result|
result.each do |row|
if row['waiting'] == 't'
metrics[:waiting] = row['count']
elsif row['waiting'] == 'f'
metrics[:active] = row['count']
end
end
end
end

metrics[:total] = (metrics[:waiting].to_i + metrics[:active].to_i)
metrics[:total] = (metrics[:waiting].to_i + metrics[:active].to_i)

metrics.each do |metric, value|
output "#{config[:scheme]}.connections.#{config[:database]}.#{metric}", value, timestamp
metrics.each do |metric, value|
output "#{config[:scheme]}.connections.#{database}.#{metric}", value, timestamp
end
end

ok
Expand Down
24 changes: 14 additions & 10 deletions bin/metric-postgres-dbsize.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
#
# USAGE:
# ./metric-postgres-dbsize.rb -u db_user -p db_pass -h db_host -d db
# ./metric-postgres-dbsize.rb -u db_user -p db_pass -h db_host -d 'db1,db2'
#
# NOTES:
#
Expand All @@ -31,6 +32,7 @@
#

require 'sensu-plugins-postgres/pgpass'
require 'sensu-plugins-postgres/pgdatabases'
require 'sensu-plugin/metric/cli'
require 'pg'
require 'socket'
Expand Down Expand Up @@ -62,10 +64,11 @@ class PostgresStatsDBMetrics < Sensu::Plugin::Metric::CLI::Graphite
short: '-P PORT',
long: '--port PORT'

option :database,
description: 'Database name',
option :databases,
description: 'Database names, separated by ","',
short: '-d DB',
long: '--db DB'
long: '--db DB',
default: nil

option :scheme,
description: 'Metric naming scheme, text to prepend to $queue_name.$metric',
Expand All @@ -79,23 +82,24 @@ class PostgresStatsDBMetrics < Sensu::Plugin::Metric::CLI::Graphite
default: nil

include Pgpass
include Pgdatabases

def run
timestamp = Time.now.to_i
pgpass
databases = pgdatabases
con = PG.connect(host: config[:hostname],
dbname: config[:database],
dbname: databases.first,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this still needs to be updated right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in which way? It connects to the first DB and then queries the information for all databases. There is no need to connect to each DB individually.

user: config[:user],
password: config[:password],
port: config[:port],
connect_timeout: config[:timeout])
request = [
"select pg_database_size('#{config[:database]}')"
]

con.exec(request.join(' ')) do |result|
result.each do |row|
output "#{config[:scheme]}.size.#{config[:database]}", row['pg_database_size'], timestamp
databases.each do |database|
con.exec_params('select pg_database_size($1)', [database]) do |result|
result.each do |row|
output "#{config[:scheme]}.size.#{database}", row['pg_database_size'], timestamp
end
end
end

Expand Down
43 changes: 26 additions & 17 deletions bin/metric-postgres-locks.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
#
# USAGE:
# ./metric-postgres-locks.rb -u db_user -p db_pass -h db_host -d db
# ./metric-postgres-locks.rb -u db_user -p db_pass -h db_host -d 'db1,db2'
#
# NOTES:
#
Expand All @@ -31,6 +32,7 @@
#

require 'sensu-plugins-postgres/pgpass'
require 'sensu-plugins-postgres/pgdatabases'
require 'sensu-plugin/metric/cli'
require 'pg'
require 'socket'
Expand Down Expand Up @@ -62,10 +64,11 @@ class PostgresStatsDBMetrics < Sensu::Plugin::Metric::CLI::Graphite
short: '-P PORT',
long: '--port PORT'

option :database,
description: 'Database name',
option :databases,
description: 'Database names, separated by ","',
short: '-d DB',
long: '--db DB'
long: '--db DB',
default: nil

option :scheme,
description: 'Metric naming scheme, text to prepend to $queue_name.$metric',
Expand All @@ -79,33 +82,39 @@ class PostgresStatsDBMetrics < Sensu::Plugin::Metric::CLI::Graphite
default: nil

include Pgpass
include Pgdatabases

def run
timestamp = Time.now.to_i

locks_per_type = Hash.new(0)
pgpass
databases = pgdatabases
# connect only to first database and get information for all databases through SQL queries
con = PG.connect(host: config[:hostname],
dbname: config[:database],
dbname: databases.first,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this still needs to be updated right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in which way? It connects to the first DB and then queries the information for all databases. There is no need to connect to each DB individually.

user: config[:user],
password: config[:password],
port: config[:port],
connect_timeout: config[:timeout])
request = [
'SELECT mode, count(mode) AS count FROM pg_locks',
"WHERE database = (SELECT oid FROM pg_database WHERE datname = '#{config[:database]}')",
'GROUP BY mode'
]

con.exec(request.join(' ')) do |result|
result.each do |row|
lock_name = row['mode'].downcase.to_sym
locks_per_type[lock_name] = row['count']

databases.each do |database|
request = [
'SELECT mode, count(mode) AS count FROM pg_locks',
'WHERE database = (SELECT oid FROM pg_database WHERE datname = $1)',
'GROUP BY mode'
]

con.exec_params(request.join(' '), [database]) do |result|
result.each do |row|
lock_name = row['mode'].downcase.to_sym
locks_per_type[lock_name] = row['count']
end
end
end

locks_per_type.each do |lock_type, count|
output "#{config[:scheme]}.locks.#{config[:database]}.#{lock_type}", count, timestamp
locks_per_type.each do |lock_type, count|
output "#{config[:scheme]}.locks.#{database}.#{lock_type}", count, timestamp
end
end

ok
Expand Down
1 change: 1 addition & 0 deletions bin/metric-postgres-statsdb.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
#
# USAGE:
# ./metric-postgres-statsdb.rb -u db_user -p db_pass -h db_host -d db
# ./metric-postgres-statsdb.rb -u db_user -p db_pass -h db_host -d 'db1;db2;db3'
#
# NOTES:
# Requires PSQL `track_counts` `track_io_timing` for some metrics enabled
Expand Down
43 changes: 29 additions & 14 deletions bin/metric-postgres-statsio.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
#
# USAGE:
# ./metric-postgres-statsio.rb -u db_user -p db_pass -h db_host -d db -s scope
# ./metric-postgres-statsio.rb -u db_user -p db_pass -h db_host -d 'db1,db2' -s scope
#
# NOTES:
# Requires PSQL `track_io_timing` enabled
Expand All @@ -32,6 +33,7 @@
#

require 'sensu-plugins-postgres/pgpass'
require 'sensu-plugins-postgres/pgdatabases'
require 'sensu-plugin/metric/cli'
require 'pg'
require 'socket'
Expand Down Expand Up @@ -63,10 +65,11 @@ class PostgresStatsIOMetrics < Sensu::Plugin::Metric::CLI::Graphite
short: '-P PORT',
long: '--port PORT'

option :database,
description: 'Database name',
option :databases,
description: 'Database names, separated by ","',
short: '-d DB',
long: '--db DB'
long: '--db DB',
default: nil

option :scope,
description: 'Scope, see http://www.postgresql.org/docs/9.2/static/monitoring-stats.html',
Expand All @@ -86,12 +89,24 @@ class PostgresStatsIOMetrics < Sensu::Plugin::Metric::CLI::Graphite
default: nil

include Pgpass
include Pgdatabases

def run
timestamp = Time.now.to_i
pgpass

pgdatabases.each do |database|
output_database(database)
end

ok
end

private

def output_database(database)
timestamp = Time.now.to_i
con = PG.connect(host: config[:hostname],
dbname: config[:database],
dbname: database,
user: config[:user],
password: config[:password],
port: config[:port],
Expand All @@ -105,17 +120,17 @@ def run
]
con.exec(request.join(' ')) do |result|
result.each do |row|
output "#{config[:scheme]}.statsio.#{config[:database]}.heap_blks_read", row['heap_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.heap_blks_hit", row['heap_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.idx_blks_read", row['idx_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.idx_blks_hit", row['idx_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.toast_blks_read", row['toast_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.toast_blks_hit", row['toast_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.tidx_blks_read", row['tidx_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{config[:database]}.tidx_blks_hit", row['tidx_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{database}.heap_blks_read", row['heap_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{database}.heap_blks_hit", row['heap_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{database}.idx_blks_read", row['idx_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{database}.idx_blks_hit", row['idx_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{database}.toast_blks_read", row['toast_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{database}.toast_blks_hit", row['toast_blks_hit'], timestamp
output "#{config[:scheme]}.statsio.#{database}.tidx_blks_read", row['tidx_blks_read'], timestamp
output "#{config[:scheme]}.statsio.#{database}.tidx_blks_hit", row['tidx_blks_hit'], timestamp
end
end

ok
con.close
end
end
Loading