Dalibo released yesterday (14.11.2022) temBoard version 8.0, which is quite a big one. Even the documentation page has changed. 🙂 Here are the most important changes in this new release compared to version 7.11.
Breaking changes
- temBoard UI dropped support for Internet Explorer 8. You may have issues with browsers older than 5 years.
- Dropped support for PostgreSQL 9.5 and 9.4. For both UI and agent.
- Dropped key-only authentication on agent. Access to UI grants full access on managed agent.
- New CLI for both UI and agent. temboard and temboard-agent are the single CLI entrypoints.
- Removed temboard –debug CLI option.
- Dropped commands temboard-agent-adduser and temboard-agent-password.
- Commands temboard-migratedb, temboard-agent-register are moved as subcommands of temboard and temboard-agent.
- temBoard UI dropped push-metrics handler from pre-6.0 push metric collect.
- temBoard UI RPM does not execute auto_configure.sh upon installation.
- temBoard UI RPM does not create temboard UNIX user. Use auto_configure.sh instead.
- temBoard Agent RPM does not create postgres UNIX user. Use PostgreSQL packages instead.
- Packages does not provide logrotate configuration anymore. temBoard can still log to file.
- pg_ctl agent parameter must not be quoted now, in temboard-agent.conf.
- Agent auto_configure.sh now requires a parameter: the UI url.
- temBoard agent does not provide legacy single-installation systemd unit file temboard-agent.service in favor of temboard-agent@.service.
- temBoard agent debian package does not provide legacy temboard-agent.init SysV script. Use sytemd instead.
- temBoard agent auto_configure.sh now generates a single configuration file instead of temboard-agent.conf.d/auto.conf.
- temBoard agent auto_configure.sh does not configure file logging anymore.
- Dropped temBoard agent HTTP endpoint /monitoring/probe/*, some /dashboard/ probes and more. Use temboard routes and temboard-agent routes to inspect availables HTTP URLs.
- Docker image for agent now configures agent with auto_configure.sh. temBoard agent configuration moved from /etc/temboard-agent to /etc/temboard-agent/.
Deprecation
- Running temBoard UI with Python 2.7 and Python 3.5 is deprecated. All RPM and deb packages ships with Python 3.
Debian stretch support is deprecated (EOL June 2022).
New Features
- PostgreSQL 15 support.
- Unified authentication. Signing in UI open full DBA access to agents, without double login.
- Register instance without querying UI API using temboard register-instance.
- Download instance inventory as CSV.
- Automatically refresh introspection data from agent.
- New About temBoard page with detailed installation’s informations.
- New About instance page with PostgreSQL, system and agent informations.
- OpenMetrics endpoint. Accessible using temBoard UI as authenticating proxy to agent.
- RHEL9, Debian Bookworm (testing) packages. Increase agent OOM score with systemd.
- Restyled documentation with improved search and navigation.
Changes
- Each temBoard release has it’s own docker tag. e.g. dalibo/temboard:8.0, dalibo/temboard:8.0rc1, etc. See Docker Hub repositories dalibo/temboard and dalibo/temboard-agent.
- Streamlined docker images, basing on Debian Bullseye, with temBoard installed with APT instead of pip.
- Restart scheduler and worker pool background processes on crash.
- Improved error logging. Log format is now Postgres-like.
UI changes
- Sign agent requests with an asymetric cryptographic key. New agent does not require double authentification anymore.
- New command temboard generate-key for generating signing key.
- New database migration engine. Dropped dependency on Alembic, Mako, etc.
- Move temboard-migratedb as temboard migratedb subcommand.
- New command temboard tasks flush to flush old tasks when upgrading.
- Fast collect of monitoring and statements metrics upon agent registration.
- Format PostgreSQL start time in dashboard as relative date.
- temBoard UI waits for locks in monitoring collect. Abort long collect task. New parameter [monitoring] collect_max_duration.
- temBoard UI now has API key authorization, use for /metrics proxy to agent.
- New temBoard UI parameter [auth] allowed_ip to restrict API Key authorization. By default, only 127/8 is allowed.
- Improved refresh error handling in dashboard, activity and home page. Error are now inlined instead of modal.
- temBoard UI now accept to serve plain HTTP. For development purpose.
- New commands temboard query-agent, temboard routes, temboard tasks run, temboard tasks schedule and temboard web for debugging.
- Handle SIGCHLD in temboard UI too. No more zombies.
- Limit activity view to 300 longest queries.
Agent changes
- Unified authentification: agent now uses UI as source of identity.
- Dropped users file and related configuration.
- New command temboard-agent fetch-key to accept UI signing key.
- New option: [temboard] ui_url, pointing to UI URL.
- auto_configure.sh conditionnaly enable statements plugins.
- Move temboard-agent-register as temboard-agent register subcommand.
- New command temboard-agent discover to introspect temBoard agent, system and PostgreSQL.
- Refresh discover data on startup and Postgres connection recovery.
- temBoard agent pools connections to PostgreSQL used by web API, reducing connections stress on PostgreSQL.
- Heavily reduction of connection opened by dashboard.
- Drop agent configuration postgresql:instance in favor of Postgres setting cluster_name.
- Add new unified sessions and detailed locks endpoint.
- temBoard agent now depends on cryptography and bottle.
- New subcommands temboard-agent routes, temboard-agent tasks run and temboard-agent web for debugging.
- Monitoring purge_after default value is now set to 730 (2 years), it was empty before (no limit).
In case you want to take a closer look at the change log, click on the following link.
https://temboard.readthedocs.io/en/latest/changelog/
You can compare changes between release 7.11 and 8.0 and see how many changes went into this release
https://github.com/dalibo/temboard/compare/7.11…v8.0
However, in this blog post I would like to concentrate on the upgrade procedure to the version 8.0.
Before you start, read the upgrade docs carefully. You can find them here.
https://temboard.readthedocs.io/en/latest/server_upgrade/
From a high level view, the most important steps are:
- Backup your repository database
- Backup temboard.conf, temboard-agent.conf and other temboard related files
- Upgrade first the agents to 7.11 if not already done
- Upgrade temboard and the agent to 8.0
In this blog post, I will jump right into the temboard upgrade process.
Stop the temBoard server and agent and start with a clean log file.
$ sudo systemctl stop temboard.service $ sudo systemctl stop temboard-agent@12-pg58888.service $ sudo systemctl reset-failed $ sudo systemctl is-system-running running $ cd /var/log/temboard/ $ sudo mv temboard.log temboard.log.before_upgrade_to_8.0 $
Upgrade the temBoard (UI) to 8.0
$ sudo yum localupdate temboard-8.0-1.el7.noarch.rpm Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager Examining temboard-8.0-1.el7.noarch.rpm: temboard-8.0-1.el7.noarch Marking temboard-8.0-1.el7.noarch.rpm as an update to temboard-7.11-1.el7.noarch Resolving Dependencies --> Running transaction check ---> Package temboard.noarch 0:7.11-1.el7 will be updated ---> Package temboard.noarch 0:8.0-1.el7 will be an update --> Processing Dependency: python3-flask for package: temboard-8.0-1.el7.noarch epel/x86_64/metalink | 18 kB 00:00:00 epel | 4.7 kB 00:00:00 rhel-7-server-extras-rpms | 1.9 kB 00:00:00 rhel-7-server-optional-rpms | 2.2 kB 00:00:00 rhel-7-server-rh-common-rpms | 2.0 kB 00:00:00 rhel-7-server-rpms | 2.2 kB 00:00:00 rhel-7-server-supplementary-rpms | 2.6 kB 00:00:00 rhel-server-rhscl-7-rpms | 2.3 kB 00:00:00 (1/2): epel/x86_64/updateinfo | 1.0 MB 00:00:02 (2/2): epel/x86_64/primary_db | 7.0 MB 00:00:01 --> Processing Dependency: python3-future for package: temboard-8.0-1.el7.noarch --> Processing Dependency: python3-tornado for package: temboard-8.0-1.el7.noarch --> Processing Dependency: python36-cryptography for package: temboard-8.0-1.el7.noarch --> Processing Dependency: python36-dateutil for package: temboard-8.0-1.el7.noarch --> Processing Dependency: python36-sqlalchemy for package: temboard-8.0-1.el7.noarch --> Running transaction check ---> Package python36-cryptography.x86_64 0:2.3-2.el7 will be installed --> Processing Dependency: python36-asn1crypto >= 0.21 for package: python36-cryptography-2.3-2.el7.x86_64 --> Processing Dependency: python36-cffi >= 1.7 for package: python36-cryptography-2.3-2.el7.x86_64 --> Processing Dependency: python36-idna >= 2.1 for package: python36-cryptography-2.3-2.el7.x86_64 --> Processing Dependency: python36-six >= 1.4.1 for package: python36-cryptography-2.3-2.el7.x86_64 ---> Package python36-dateutil.noarch 1:2.4.2-5.el7 will be installed ---> Package python36-flask.noarch 0:1.1.2-4.el7 will be installed --> Processing Dependency: python36-click >= 5.1 for package: python36-flask-1.1.2-4.el7.noarch --> Processing Dependency: python36-itsdangerous >= 0.24 for package: python36-flask-1.1.2-4.el7.noarch --> Processing Dependency: python36-jinja2 >= 2.10.1 for package: python36-flask-1.1.2-4.el7.noarch --> Processing Dependency: python36-werkzeug >= 0.15 for package: python36-flask-1.1.2-4.el7.noarch ---> Package python36-future.noarch 0:0.18.2-2.el7 will be installed ---> Package python36-sqlalchemy.x86_64 0:1.1.3-3.el7 will be installed ---> Package python36-tornado.x86_64 0:4.5.3-1.el7 will be installed --> Processing Dependency: python36-pycurl for package: python36-tornado-4.5.3-1.el7.x86_64 --> Running transaction check ---> Package python36-asn1crypto.noarch 0:0.24.0-7.el7 will be installed ---> Package python36-cffi.x86_64 0:1.9.1-3.el7 will be installed --> Processing Dependency: python36-pycparser for package: python36-cffi-1.9.1-3.el7.x86_64 ---> Package python36-click.noarch 0:6.7-8.el7 will be installed ---> Package python36-idna.noarch 0:2.10-1.el7 will be installed ---> Package python36-itsdangerous.noarch 0:1.1.0-1.el7 will be installed ---> Package python36-jinja2.noarch 0:2.11.1-1.el7 will be installed --> Processing Dependency: python36-markupsafe >= 0.23 for package: python36-jinja2-2.11.1-1.el7.noarch ---> Package python36-pycurl.x86_64 0:7.43.0-7.el7 will be installed ---> Package python36-six.noarch 0:1.14.0-3.el7 will be installed ---> Package python36-werkzeug.noarch 0:1.0.1-1.el7 will be installed --> Running transaction check ---> Package python36-markupsafe.x86_64 0:0.23-3.el7 will be installed ---> Package python36-pycparser.noarch 0:2.14-2.el7 will be installed --> Processing Dependency: python36-ply for package: python36-pycparser-2.14-2.el7.noarch --> Running transaction check ---> Package python36-ply.noarch 0:3.9-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================== Updating: temboard noarch 8.0-1.el7 /temboard-8.0-1.el7.noarch 6.7 M Installing for dependencies: python36-asn1crypto noarch 0.24.0-7.el7 epel 180 k python36-cffi x86_64 1.9.1-3.el7 epel 219 k python36-click noarch 6.7-8.el7 epel 127 k python36-cryptography x86_64 2.3-2.el7 epel 509 k python36-dateutil noarch 1:2.4.2-5.el7 epel 81 k python36-flask noarch 1.1.2-4.el7 epel 158 k python36-future noarch 0.18.2-2.el7 epel 775 k python36-idna noarch 2.10-1.el7 epel 99 k python36-itsdangerous noarch 1.1.0-1.el7 epel 33 k python36-jinja2 noarch 2.11.1-1.el7 epel 238 k python36-markupsafe x86_64 0.23-3.el7 epel 32 k python36-ply noarch 3.9-2.el7 epel 103 k python36-pycparser noarch 2.14-2.el7 epel 95 k python36-pycurl x86_64 7.43.0-7.el7 epel 223 k python36-six noarch 1.14.0-3.el7 epel 34 k python36-sqlalchemy x86_64 1.1.3-3.el7 epel 1.6 M python36-tornado x86_64 4.5.3-1.el7 epel 718 k python36-werkzeug noarch 1.0.1-1.el7 epel 443 k Transaction Summary ======================================================================================================================================================== Install ( 18 Dependent packages) Upgrade 1 Package Total size: 12 M Total download size: 5.6 M Is this ok [y/d/N]: y Downloading packages: (1/17): python36-asn1crypto-0.24.0-7.el7.noarch.rpm | 180 kB 00:00:03 (2/17): python36-cffi-1.9.1-3.el7.x86_64.rpm | 219 kB 00:00:01 (3/17): python36-click-6.7-8.el7.noarch.rpm | 127 kB 00:00:01 (4/17): python36-cryptography-2.3-2.el7.x86_64.rpm | 509 kB 00:00:05 (5/17): python36-dateutil-2.4.2-5.el7.noarch.rpm | 81 kB 00:00:00 (6/17): python36-flask-1.1.2-4.el7.noarch.rpm | 158 kB 00:00:01 (7/17): python36-future-0.18.2-2.el7.noarch.rpm | 775 kB 00:00:13 (8/17): python36-idna-2.10-1.el7.noarch.rpm | 99 kB 00:00:00 (9/17): python36-itsdangerous-1.1.0-1.el7.noarch.rpm | 33 kB 00:00:00 (10/17): python36-jinja2-2.11.1-1.el7.noarch.rpm | 238 kB 00:00:01 (11/17): python36-markupsafe-0.23-3.el7.x86_64.rpm | 32 kB 00:00:00 (12/17): python36-ply-3.9-2.el7.noarch.rpm | 103 kB 00:00:00 (13/17): python36-pycparser-2.14-2.el7.noarch.rpm | 95 kB 00:00:00 (14/17): python36-pycurl-7.43.0-7.el7.x86_64.rpm | 223 kB 00:00:04 (15/17): python36-sqlalchemy-1.1.3-3.el7.x86_64.rpm | 1.6 MB 00:00:10 (16/17): python36-tornado-4.5.3-1.el7.x86_64.rpm | 718 kB 00:00:07 (17/17): python36-werkzeug-1.0.1-1.el7.noarch.rpm | 443 kB 00:00:02 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 95 kB/s | 5.6 MB 00:00:59 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : python36-pycurl-7.43.0-7.el7.x86_64 1/19 Installing : python36-tornado-4.5.3-1.el7.x86_64 2/19 Installing : python36-itsdangerous-1.1.0-1.el7.noarch 3/19 Installing : python36-markupsafe-0.23-3.el7.x86_64 4/19 Installing : python36-jinja2-2.11.1-1.el7.noarch 5/19 Installing : python36-future-0.18.2-2.el7.noarch 6/19 Installing : python36-asn1crypto-0.24.0-7.el7.noarch 7/19 Installing : python36-idna-2.10-1.el7.noarch 8/19 Installing : python36-click-6.7-8.el7.noarch 9/19 Installing : 1:python36-dateutil-2.4.2-5.el7.noarch 10/19 Installing : python36-ply-3.9-2.el7.noarch 11/19 Installing : python36-pycparser-2.14-2.el7.noarch 12/19 Installing : python36-cffi-1.9.1-3.el7.x86_64 13/19 Installing : python36-cryptography-2.3-2.el7.x86_64 14/19 Installing : python36-sqlalchemy-1.1.3-3.el7.x86_64 15/19 Installing : python36-werkzeug-1.0.1-1.el7.noarch 16/19 Installing : python36-flask-1.1.2-4.el7.noarch 17/19 Updating : temboard-8.0-1.el7.noarch 18/19 Cleanup : temboard-7.11-1.el7.noarch 19/19 Uploading Package Profile Loaded plugins: langpacks, product-id, subscription-manager Verifying : python36-flask-1.1.2-4.el7.noarch 1/19 Verifying : python36-werkzeug-1.0.1-1.el7.noarch 2/19 Verifying : python36-sqlalchemy-1.1.3-3.el7.x86_64 3/19 Verifying : python36-pycparser-2.14-2.el7.noarch 4/19 Verifying : python36-tornado-4.5.3-1.el7.x86_64 5/19 Verifying : python36-ply-3.9-2.el7.noarch 6/19 Verifying : 1:python36-dateutil-2.4.2-5.el7.noarch 7/19 Verifying : python36-click-6.7-8.el7.noarch 8/19 Verifying : python36-idna-2.10-1.el7.noarch 9/19 Verifying : temboard-8.0-1.el7.noarch 10/19 Verifying : python36-cryptography-2.3-2.el7.x86_64 11/19 Verifying : python36-asn1crypto-0.24.0-7.el7.noarch 12/19 Verifying : python36-cffi-1.9.1-3.el7.x86_64 13/19 Verifying : python36-future-0.18.2-2.el7.noarch 14/19 Verifying : python36-markupsafe-0.23-3.el7.x86_64 15/19 Verifying : python36-itsdangerous-1.1.0-1.el7.noarch 16/19 Verifying : python36-jinja2-2.11.1-1.el7.noarch 17/19 Verifying : python36-pycurl-7.43.0-7.el7.x86_64 18/19 Verifying : temboard-7.11-1.el7.noarch 19/19 Dependency Installed: python36-asn1crypto.noarch 0:0.24.0-7.el7 python36-cffi.x86_64 0:1.9.1-3.el7 python36-click.noarch 0:6.7-8.el7 python36-cryptography.x86_64 0:2.3-2.el7 python36-dateutil.noarch 1:2.4.2-5.el7 python36-flask.noarch 0:1.1.2-4.el7 python36-future.noarch 0:0.18.2-2.el7 python36-idna.noarch 0:2.10-1.el7 python36-itsdangerous.noarch 0:1.1.0-1.el7 python36-jinja2.noarch 0:2.11.1-1.el7 python36-markupsafe.x86_64 0:0.23-3.el7 python36-ply.noarch 0:3.9-2.el7 python36-pycparser.noarch 0:2.14-2.el7 python36-pycurl.x86_64 0:7.43.0-7.el7 python36-sqlalchemy.x86_64 0:1.1.3-3.el7 python36-tornado.x86_64 0:4.5.3-1.el7 python36-werkzeug.noarch 0:1.0.1-1.el7 Updated: temboard.noarch 0:8.0-1.el7 Complete! Uploading Enabled Repositories Report Loaded plugins: langpacks, product-id, subscription-manager
Upgrade the temBoard agent to 8.0
$ sudo yum localupdate temboard-agent-8.0-1.el7.noarch.rpm Loaded plugins: enabled_repos_upload, langpacks, package_upload, product-id, search-disabled-repos, subscription-manager Examining temboard-agent-8.0-1.el7.noarch.rpm: temboard-agent-8.0-1.el7.noarch Marking temboard-agent-8.0-1.el7.noarch.rpm as an update to temboard-agent-7.11-1.el7.noarch Resolving Dependencies --> Running transaction check ---> Package temboard-agent.noarch 0:7.11-1.el7 will be updated ---> Package temboard-agent.noarch 0:8.0-1.el7 will be an update --> Processing Dependency: python3-bottle for package: temboard-agent-8.0-1.el7.noarch rhel-7-server-extras-rpms | 1.9 kB 00:00:00 rhel-7-server-optional-rpms | 2.2 kB 00:00:00 rhel-7-server-rh-common-rpms | 2.0 kB 00:00:00 rhel-7-server-rpms | 2.2 kB 00:00:00 rhel-7-server-supplementary-rpms | 2.6 kB 00:00:00 --> Running transaction check ---> Package python36-bottle.noarch 0:0.12.21-1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =========================================================================================================================================================================== Package Arch Version Repository Size =========================================================================================================================================================================== Updating: temboard-agent noarch 8.0-1.el7 /temboard-agent-8.0-1.el7.noarch 944 k Installing for dependencies: python36-bottle noarch 0.12.21-1.el7 epel 93 k Transaction Summary =========================================================================================================================================================================== Install ( 1 Dependent package) Upgrade 1 Package Total size: 1.0 M Total download size: 93 k Is this ok [y/d/N]: y Downloading packages: python36-bottle-0.12.21-1.el7.noarch.rpm | 93 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : python36-bottle-0.12.21-1.el7.noarch 1/3 Updating : temboard-agent-8.0-1.el7.noarch 2/3 You must restart manually temboard-agent services. Cleanup : temboard-agent-7.11-1.el7.noarch 3/3 Uploading Package Profile Loaded plugins: langpacks, product-id, subscription-manager Verifying : temboard-agent-8.0-1.el7.noarch 1/3 Verifying : python36-bottle-0.12.21-1.el7.noarch 2/3 Verifying : temboard-agent-7.11-1.el7.noarch 3/3 Dependency Installed: python36-bottle.noarch 0:0.12.21-1.el7 Updated: temboard-agent.noarch 0:8.0-1.el7 Complete! Uploading Enabled Repositories Report Loaded plugins: langpacks, product-id, subscription-manager
Check if the new temboard ui and temboard agent version is correct.
$ temboard --version temBoard 8.0 (/usr/bin/temboard) System Red Hat Enterprise Linux Server 7.9 (Maipo) Python 3.6.8 (/usr/bin/python3) cryptography 2.3 Tornado 4.5.3 libpq 9.2.24 psycopg2 2.7.7 (dt dec pq3 ext) SQLAlchemy 1.1.3 $ temboard-agent --version temBoard agent 8.0 (/usr/bin/temboard-agent) System Red Hat Enterprise Linux Server 7.9 (Maipo) Python 3.6.8 (/usr/bin/python3) bottle 0.12.21 cryptography 2.3 libpq 9.2.24 psycopg2 2.7.7 (dt dec pq3 ext)
Upgrade Database Schema
$ su - temboard Password: Last login: Thu May 19 10:34:56 CEST 2022 on pts/0 $ temboard migratedb upgrade INFO: app: Using config file /etc/temboard/temboard.conf. INFO: migratedb: Upgrading database to version 007_drop-alembic.sql. INFO: migratedb: Upgrading database to version 008_monitoring-archive-wait-lock.sql. INFO: migratedb: Upgrading database to version 009_apikey.sql. INFO: migratedb: Upgrading database to version 010_discover.sql. INFO: migratedb: Database up to date.
… entries from the postgresql log
2022-11-15 13:09:27.287 CET [22178] LOG: statement: DROP TABLE IF EXISTS "application"."alembic_version";
CREATE TABLE IF NOT EXISTS "application"."schema_migration_log" (
version TEXT UNIQUE NOT NULL PRIMARY KEY,
comment TEXT,
cdate TIMESTAMP DEFAULT NOW()
);
2022-11-15 13:09:27.418 CET [22178] LOG: statement: CREATE OR REPLACE FUNCTION archive_current_metrics(table_name TEXT, record_type TEXT, query TEXT) RETURNS TABLE(tblname TEXT, nb_rows INTEGER)
LANGUAGE plpgsql
AS $$
DECLARE
v_table_current TEXT;
v_table_history TEXT;
v_query TEXT;
i INTEGER;
BEGIN
v_table_current := table_name || '_current';
v_table_history := table_name || '_history';
-- Lock _current table to prevent concurrent updates
EXECUTE 'LOCK TABLE ' || v_table_current || ' IN SHARE MODE';
v_query := replace(query, '#history_table#', v_table_history);
v_query := replace(v_query, '#current_table#', v_table_current);
v_query := replace(v_query, '#record_type#', record_type);
-- Move data into _history table
EXECUTE v_query;
GET DIAGNOSTICS i = ROW_COUNT;
-- Truncate _current table
EXECUTE 'TRUNCATE '||v_table_current;
-- Return each history table name and the number of rows inserted
RETURN QUERY SELECT v_table_history, i;
END;
$$;
2022-11-15 13:09:27.430 CET [22178] LOG: statement: CREATE TABLE "application"."apikeys" (
id SERIAL PRIMARY KEY,
secret TEXT NOT NULL UNIQUE,
comment TEXT,
cdate TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
edate TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW() + '6 months'::interval
);
2022-11-15 13:09:27.443 CET [22178] LOG: statement: ALTER TABLE "application"."instances"
ADD COLUMN "cdate" TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
ADD COLUMN "discover" JSONB,
ADD COLUMN "discover_date" TIMESTAMP WITH TIME ZONE,
ADD COLUMN "discover_etag" TEXT;
UPDATE "application"."instances"
SET "discover" = json_build_object(
'system', json_build_object(
'cpu_count', "cpu",
'memory', "memory_size",
'fqdn', "hostname"
),
'postgres', json_build_object(
'version', "pg_version",
'version_summary', "pg_version_summary",
'port', "pg_port",
'data_directory', "pg_data"
),
'temboard', json_build_object()
);
ALTER TABLE "application"."instances"
DROP COLUMN "cpu",
DROP COLUMN "memory_size",
DROP COLUMN "pg_data",
DROP COLUMN "pg_version",
DROP COLUMN "pg_version_summary";
Flush background tasks with the following command:
$ temboard tasks flush INFO: app: Using config file /etc/temboard/temboard.conf. INFO: taskmanager: Flushed 8 tasks.
Generate Signing Key
$ temboard generate-key INFO: app: Using config file /etc/temboard/temboard.conf. INFO: generate_key: Generating RSA key with openssl at /etc/temboard/signing-private.pem. Generating RSA private key, 4096 bit long modulus ......................................++ .++ e is 65537 (0x10001) INFO: generate_key: Exporting public key at /etc/temboard/signing-public.pem.
Add the Environment to the service file of tembaord server and agent.
$ cd /etc/systemd/system/multi-user.target.wants/ $ sudo vi temboard-agent@12-pg58888.service [Unit] Description=PostgreSQL Remote Control Agent %I After=network.target postgresql@%i.service AssertPathExists=/etc/temboard-agent/%I/temboard-agent.conf [Service] Type=simple User=postgres Group=postgres Environment="SYSTEMD=1" Environment="LD_LIBRARY_PATH=/app/lib/postgres/pgproduct/pg-12.12/lib" ExecStart=/usr/bin/temboard-agent -c /etc/temboard-agent/%I/temboard-agent.conf # Increase OOM Score to ensure agent is killed before Postgres. OOMScoreAdjust=15 [Install] WantedBy=multi-user.target $ sudo vi temboard.service [Unit] Description=temBoard Server After=network.target [Service] Type=simple User=temboard Group=temboard Environment="SYSTEMD=1" Environment="LD_LIBRARY_PATH=/app/lib/postgres/pgproduct/pg-12.12/lib" ExecStart=/usr/bin/temboard -c /etc/temboard/temboard.conf serve ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target $ sudo systemctl daemon-reload $ sudo systemctl reset-failed $ sudo systemctl is-system-running running
I usually adjust the temboard service as well with the nice value of -19. (Nice=-19). This is an optional step and not mandatory. I do it only to give the temBoard service the highest CPU priority.
$ cd /etc/systemd/system/temboard.service.d/ $ cat override.conf [Service] Nice=-19 #Environment=PERF=y
Startup the temBoard UI and the agent.
$ sudo systemctl daemon-reload $ sudo systemctl start temboard.service $ sudo systemctl start temboard-agent@12-pg58888.service
$ sudo systemctl status temboard.service
● temboard.service - temBoard Server
Loaded: loaded (/usr/lib/systemd/system/temboard.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/temboard.service.d
└─override.conf
Active: active (running) since Tue 2022-11-15 13:17:46 CET; 39s ago
Main PID: 32439 (temboard)
CGroup: /system.slice/temboard.service
├─32439 temboard: web
├─32454 temboard: worker pool
├─32456 temboard: scheduler
├─32466 temboard: task temboardui.plugins.statements.pull_data_worker
├─32467 temboard: task temboardui.plugins.statements.statements_purge_worker
├─32468 temboard: task temboardui.plugins.monitoring.aggregate_data_worker
├─32470 temboard: task temboardui.plugins.monitoring.history_tables_worker
└─32471 temboard: task temboardui.plugins.monitoring.purge_data_worker
Nov 15 13:18:24 host01 temboard[32439]: statements: Successfully pulled statements data for h51lya0.suvanet.ch:44077.
Nov 15 13:18:24 host01 temboard[32439]: statements: Pulling statements from h50ly30.suvanet.ch:44013.
Nov 15 13:18:24 host01 temboard[32439]: statements: Successfully pulled statements data for h50ly30.suvanet.ch:44013.
Nov 15 13:18:24 host01 temboard[32439]: statements: Pulling statements from h53ly30.suvanet.ch:44036.
Nov 15 13:18:24 host01 temboard[32439]: statements: Successfully pulled statements data for h53ly30.suvanet.ch:44036.
...
...
$ sudo systemctl status temboard-agent@12-pg58888.service ● temboard-agent@12-pg58888.service - PostgreSQL Remote Control Agent 12/pg58888 Loaded: loaded (/usr/lib/systemd/system/temboard-agent@.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2022-11-15 13:17:56 CET; 51s ago Process: 32602 ExecStart=/usr/bin/temboard-agent -c /etc/temboard-agent/%I/temboard-agent.conf (code=exited, status=1/FAILURE) Main PID: 32602 (code=exited, status=1/FAILURE) Nov 15 13:17:55 host01 systemd[1]: Started PostgreSQL Remote Control Agent 12/pg58888. Nov 15 13:17:56 host01 temboard-agent[32602]: INFO: app: Using config file /etc/temboard-agent/12/pg58888/temboard-agent.conf. Nov 15 13:17:56 host01 temboard-agent[32602]: CRITICAL: app: You must not quote string in configuration (administration.pg.ctl). Nov 15 13:17:56 host01 systemd[1]: temboard-agent@12-pg58888.service: main process exited, code=exited, status=1/FAILURE Nov 15 13:17:56 host01 systemd[1]: Unit temboard-agent@12-pg58888.service entered failed state. Nov 15 13:17:56 host01 systemd[1]: temboard-agent@12-pg58888.service failed.
Starting the Agent failed because we do have some quoted strings in our temboard-agent.conf file. To fix that, remove the quotes from the pg_ctl line. This is not a bug. This change was clearly documented in the release note.
- pg_ctl agent parameter must not be quoted now, in temboard-agent.conf.
[administration] pg_ctl = '/app/lib/postgres/pgproduct/pg-12.8/bin/pg_ctl %s -D /app/lib/postgres/pgdata/12/PGMP01A' [administration] pg_ctl = /app/lib/postgres/pgproduct/pg-12.12/bin/pg_ctl %s -D /app/lib/postgres/pgdata/12/PGMP01A
Now we can try again.
$ sudo systemctl start temboard-agent@12-pg58888.service $ sudo systemctl status temboard-agent@12-pg58888.service ● temboard-agent@12-pg58888.service - PostgreSQL Remote Control Agent 12/pg58888 Loaded: loaded (/usr/lib/systemd/system/temboard-agent@.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2022-11-15 13:21:47 CET; 4s ago Process: 18696 ExecStart=/usr/bin/temboard-agent -c /etc/temboard-agent/%I/temboard-agent.conf (code=exited, status=1/FAILURE) Main PID: 18696 (code=exited, status=1/FAILURE) Nov 15 13:21:43 host01 systemd[1]: Started PostgreSQL Remote Control Agent 12/pg58888. Nov 15 13:21:46 host01 temboard-agent[18696]: INFO: app: Using config file /etc/temboard-agent/12/pg58888/temboard-agent.conf. Nov 15 13:21:47 host01 temboard-agent[18696]: app: Failed to load signing key: [Errno 2] No such file or directory: '/etc/temboard-agent/12/pg58888/sig...blic.pem' Nov 15 13:21:47 host01 systemd[1]: temboard-agent@12-pg58888.service: main process exited, code=exited, status=1/FAILURE Nov 15 13:21:47 host01 systemd[1]: Unit temboard-agent@12-pg58888.service entered failed state. Nov 15 13:21:47 host01 systemd[1]: temboard-agent@12-pg58888.service failed. Hint: Some lines were ellipsized, use -l to show in full. $ cat /var/log/temboard-agent/12-pg58888.log ... ... 2022-11-15 13:21:47 CET temboardagent[18696] CRITICAL: app: Failed to load signing key: [Errno 2] No such file or directory: '/etc/temboard-agent/12/pg58888/signing-public.pem'
Opsss … again an error. This time, the public key is missing.
$ cd /etc/temboard/ $ cp -p signing-public.pem /etc/temboard-agent/12/pg58888/ $ cat signing-public.pem -----BEGIN PUBLIC KEY----- MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEArlYjR/+Y8I5GH0ZAXafg Yuyj8kM7RoaPODtKTkOw3s2E99v8BjbO/NO0ZjUMpjfNHtwKL0Ahkcvs6gB/f7H2 ... ... e4HucKDQj3ZbYr+CN5ztUXhSvr1OVl4jnTOJVDkJtg6LL6V8/X7Q2X6K0SDB1oqf t1O4jlkbfv1Bn2gV9/MqztZ++gudq3bEtFkLi0ICh5LKnPsDXWgI1rKc2hGFYAq5 lk83GCc7fd9LuKmpTgybwAsYnq5i6FG+uGSfQCDObh/wauqlevSD3EXbKaynk9hM H4eWB9JmniAbMUXjPkfeGasCAwEAAQ== -----END PUBLIC KEY-----
$ sudo systemctl start temboard-agent@12-pg58888.service
$ sudo systemctl status temboard-agent@12-pg58888.service
● temboard-agent@12-pg58888.service - PostgreSQL Remote Control Agent 12/pg58888
Loaded: loaded (/usr/lib/systemd/system/temboard-agent@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-11-15 13:24:34 CET; 5s ago
Main PID: 21132 (temboard-agent)
CGroup: /system.slice/system-temboard\x2dagent.slice/temboard-agent@12-pg58888.service
├─21132 temboard-agent: web
├─21138 temboard-agent: worker pool
├─21139 temboard-agent: scheduler
└─21153 temboard-agent: task temboardagent.plugins.dashboard.dashboard_collector_batch_worker
Nov 15 13:24:34 h50ly20 systemd[1]: Started PostgreSQL Remote Control Agent 12/pg58888.
Nov 15 13:24:34 h50ly20 temboard-agent[21132]: INFO: app: Using config file /etc/temboard-agent/12/pg58888/temboard-agent.conf.
Now, everything is fine and we are ready to go. 🙂
Conclusion
That was quite a huge new release from Dalibo with quite a lot cool new stuff. I will write a few blog posts about the new features in temBoard version 8.0 very soon. Have fun with the new version.