This is a package of tools in use in Skype for replication and failover. Also it includes a generic queuing mechanism PgQ and utility library for Python scripts.
Overview
It contains following tools:
PgQ
This is the queue mechanism we use. It consists of PL/pgsql, and C code in database, with Python framework on top of it. It is based on snapshot based event handling ideas from Slony-I, written for general usage.
Features:
-
There can be several queues in database.
-
There can be several producers than can insert into any queue.
-
There can be several consumers on one queue and all consumers see all events.
Documentation:
-
PgQ admin tool (pgqadm) usage: pgq-admin.html
-
PgQ SQL API overview: pgq-sql.html
-
PgQ SQL reference: http://skytools.projects.postgresql.org/pgq/
Londiste
Replication tool written in Python, using PgQ as event transport.
Features:
-
Tables can be added one-by-one into set.
-
Initial COPY for one table does not block event replay for other tables.
-
Can compare tables on both sides.
Documentation:
-
Londiste script usage: londiste.cmdline.html (also available as man 1 londiste)
-
Londiste configuration: londiste.config.html (also available as man 5 londiste)
-
Londiste reference: londiste.ref.html
walmgr
This script will setup WAL archiving, does initial backup and runtime WAL archive and restore.
Source tree contents
doc/ Documentation in asciidoc format. Source for both html and man pages.
python/ Python modules and primary executables - walmgr, londiste, qadmin, pgqadm.
python/pgq/ Python framework for PgQ.
python/londiste/ Londiste replication.
python/skytools/ Low-level utilities for writing database scripts in Python.
sql/ Database modules.
sql/pgq/ Table definitions and functions for PgQ queueing.
sql/pgq_coop/ Functions for cooperative consuming.
sql/londiste/ Table definitions and functions for Londiste replication.
sql/ticker/ PgQ ticker written in C.
scripts/ Python scripts with lesser priority.
lib/ libusual C libary, for pgqd.
debian/ Debian packaging. This is for creating private packages, official Debian packages uses it's own packagin code.
misc/ Random scripts used for building.