Skip to content
This repository was archived by the owner on Apr 21, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
7504c20
Fix README.md (#1250)
iRockyZhou Mar 28, 2017
71a1364
Improve Readme.md for not so advanced users (#1235)
javiertury Mar 28, 2017
0e2625b
Add semicolons to Pooling example in README.md (#1266)
Apr 17, 2017
4505ae9
support ssl params for pg-native (#1169)
arypurnomoz Apr 19, 2017
4f790de
Support for logical streaming replication (#1271)
Apr 24, 2017
80d136a
Add test & documentation for replicationStart message
brianc Apr 24, 2017
f42924b
Bump version
brianc Apr 24, 2017
db5f4ae
Upgrade packet reader (#1287)
brianc May 15, 2017
4659d5d
Bump version
brianc May 15, 2017
ee81936
Libpq connection string escaping (#1285)
sehrope May 15, 2017
c32316d
Bump version
brianc May 15, 2017
e5f0e5d
s/2016/2017/ (#1291)
tjschuck May 17, 2017
4cd56cc
Make pool name consistent on missing config params (#1279)
May 24, 2017
3757ff7
Bump version
brianc May 24, 2017
f2b87e0
Add client connectionString tests (#1310)
brianc Jun 8, 2017
76c59a0
Emit error when backend unexpectedly disconnects
brianc Jun 9, 2017
5869121
Bump version
brianc Jun 9, 2017
5421e9d
Added MIT License
amilajack Jun 11, 2017
bbb759f
Create LICENSE
brianc Jun 12, 2017
b5b49eb
Add deprecations
brianc Jun 18, 2017
1e04fdb
Add changes for v6.3.0
brianc Jun 19, 2017
f7a9461
Bump version
brianc Jun 19, 2017
842803c
Fix over-eager deprecation warnings (#1333)
brianc Jun 20, 2017
e446934
Fix deprecation warnings in native driver
brianc Jun 20, 2017
afe2498
Bump version
brianc Jun 20, 2017
860cccd
fix for server enconding when using SQL_ASCII and latin1 enconding
ederavilaprado Jun 20, 2017
055e708
Bump version
brianc Jun 21, 2017
7636f36
Update changelog
brianc Jun 21, 2017
a0a0507
Bump version
brianc Jun 21, 2017
dbf3bd3
use consistent syntax for semver ranges
Jun 26, 2017
c2af53a
Properly insert buffers in arrays.
2Pacalypse- May 27, 2017
e44d83f
Add the test for arrays of buffers.
2Pacalypse- Jun 10, 2017
e52512c
Adjust the test for arrays of buffers to work across all node versions.
2Pacalypse- Jun 12, 2017
bc0b03e
Bump version
brianc Jul 14, 2017
9bcf55d
Fix vulnerability
brianc Aug 12, 2017
d4aa616
Bump version
brianc Aug 12, 2017
389c25f
Merge tag 'v6.4.2' into cdb-6.1
Algunenano May 23, 2018
8ce7e13
Make tests compatible with PostgreSQL 10
charmander Nov 15, 2017
b5be2ee
Remove deprecation warnings
Algunenano May 23, 2018
bc0fe87
Adapt travis matrix
Algunenano May 23, 2018
5e70051
Do no test native module
Algunenano May 23, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 18 additions & 36 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,25 @@ language: node_js
sudo: false
dist: trusty
before_script:
- node script/create-test-tables.js pg://postgres@127.0.0.1:5432/postgres
- node script/create-test-tables.js pg://postgres@127.0.0.1:$PGPORT/postgres
env:
- CC=clang CXX=clang++ npm_config_clang=1 PGUSER=postgres PGDATABASE=postgres

node_js: "6"
addons:
postgresql: "9.6"
env:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't know that travis could handle these environment combinations:

  • PG 9.5 - node6
  • PG 9.5 - node8
  • PG 10 - node6
  • PG 10 - node8

Great!

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it weren't because PG10 is not fully integrated the script could be simpler (no manual install). For example:

postgresql:
  - "9.5"
  - "9.6"

node_js:
  - "6"
  - "8"

Will create a matrix of 4 configurations automatically with those packages setup properly.

- POSTGRESQL_VERSION="9.5"
- POSTGRESQL_VERSION="10"

node_js:
- "6"
- "8"


before_install:
- sudo service postgresql stop;
- sudo apt-get install -y --allow-unauthenticated --no-install-recommends --no-install-suggests postgresql-$POSTGRESQL_VERSION postgresql-client-$POSTGRESQL_VERSION postgresql-server-dev-$POSTGRESQL_VERSION
- echo -e "# TYPE DATABASE USER ADDRESS METHOD \nlocal all postgres trust\nlocal all all trust\nhost all all 127.0.0.1/32 trust" | sudo tee /etc/postgresql/$POSTGRESQL_VERSION/main/pg_hba.conf
- export PGPORT=`grep ^port /etc/postgresql/$POSTGRESQL_VERSION/main/postgresql.conf | awk '{print $3}'`
- export PGUSER=postgres
- export PGDATABASE=postgres
- sudo service postgresql restart $POSTGRESQL_VERSION;

matrix:
include:
- node_js: "0.10"
addons:
postgresql: "9.6"
env: []
- node_js: "0.12"
addons:
postgresql: "9.6"
env: []
- node_js: "4"
addons:
postgresql: "9.6"
- node_js: "5"
addons:
postgresql: "9.6"
- node_js: "6"
addons:
postgresql: "9.1"
dist: precise
- node_js: "6"
addons:
postgresql: "9.2"
- node_js: "6"
addons:
postgresql: "9.3"
- node_js: "6"
addons:
postgresql: "9.4"
- node_js: "6"
addons:
postgresql: "9.5"
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@ For richer information consult the commit log on github with referenced pull req

We do not include break-fix version release in this file.

### v6.4.0

- Add support for passing `client_encoding` as a connection parameter. Used when decoding strings in the JavaScript driver. The default is still `utf8`.

### v6.3.0

- Deprecate `pg.connect` `pg.end` and `pg.cancel` - favor using `new pg.Pool()` instead of pg singleton.
- Deprecate undocumented but possibly used `query.promise()` method. Use the promise returned directly from `client.query` / `pool.query`.
- Deprecate returning an automatically created query result from `client.query`. Instead return more idomatic responses for callback/promise methods.

### v6.2.0

- Add support for [parsing `replicationStart` messages](https://github.com/brianc/node-postgres/pull/1271/files).

### v6.1.0

- Add optional callback parameter to the pure JavaScript `client.end` method. The native client already supported this.
Expand Down
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2010 - 2017 Brian Carlson

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ test: test-unit

test-all: jshint test-missing-native test-unit test-integration test-native test-binary

test-all-nonative: jshint test-unit test-integration test-binary

update-npm:
@npm i npm --global
Expand Down
145 changes: 98 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#node-postgres
# node-postgres

[![Build Status](https://secure.travis-ci.org/brianc/node-postgres.svg?branch=master)](http://travis-ci.org/brianc/node-postgres)
[![Dependency Status](https://david-dm.org/brianc/node-postgres.svg)](https://david-dm.org/brianc/node-postgres)
Expand All @@ -15,46 +15,36 @@ $ npm install pg

## Intro & Examples

### Simple example
There are 3 ways of executing queries

```js
var pg = require('pg');
1. Passing the query to a pool
2. Borrowing a client from a pool and executing the query with it
3. Obtaining an exclusive client and executing the query with it

// instantiate a new client
// the client will read connection information from
// the same environment variables used by postgres cli tools
var client = new pg.Client();
It is recommended to pass the query to a pool as often as possible. If that isn't possible, because of long and complex transactions for example, borrow a client from a pool. Just remember to initialize the pool only once in your code so you maximize reusability of connections.

// connect to our database
client.connect(function (err) {
if (err) throw err;
### Why pooling?

// execute a query on our database
client.query('SELECT $1::text as name', ['brianc'], function (err, result) {
if (err) throw err;
If you're working on something like a web application which makes frequent queries you'll want to access the PostgreSQL server through a pool of clients. Why? For one thing, there is ~20-30 millisecond delay (YMMV) when connecting a new client to the PostgreSQL server because of the startup handshake. Furthermore, PostgreSQL can support only a limited number of clients...it depends on the amount of ram on your database server, but generally more than 100 clients at a time is a __very bad thing__. :tm: Additionally, PostgreSQL can only execute 1 query at a time per connected client, so pipelining all queries for all requests through a single, long-lived client will likely introduce a bottleneck into your application if you need high concurrency.

// just print the result to the console
console.log(result.rows[0]); // outputs: { name: 'brianc' }
With that in mind we can imagine a situation where you have a web server which connects and disconnects a new client for every web request or every query (don't do this!). If you get only 1 request at a time everything will seem to work fine, though it will be a touch slower due to the connection overhead. Once you get >100 simultaneous requests your web server will attempt to open 100 connections to the PostgreSQL backend and :boom: you'll run out of memory on the PostgreSQL server, your database will become unresponsive, your app will seem to hang, and everything will break. Boooo!

// disconnect the client
client.end(function (err) {
if (err) throw err;
});
});
});
__Good news__: node-postgres ships with built in client pooling. Client pooling allows your application to use a pool of already connected clients and reuse them for each request to your application. If your app needs to make more queries than there are available clients in the pool the queries will queue instead of overwhelming your database & causing a cascading failure. :thumbsup:

```
node-postgres uses [pg-pool](https://github.com/brianc/node-pg-pool.git) to manage pooling. It bundles it and exports it for convenience. If you want, you can `require('pg-pool')` and use it directly - it's the same as the constructor exported at `pg.Pool`.

### Client pooling
It's __highly recommended__ you read the documentation for [pg-pool](https://github.com/brianc/node-pg-pool.git).

If you're working on something like a web application which makes frequent queries you'll want to access the PostgreSQL server through a pool of clients. Why? For one thing, there is ~20-30 millisecond delay (YMMV) when connecting a new client to the PostgreSQL server because of the startup handshake. Furthermore, PostgreSQL can support only a limited number of clients...it depends on the amount of ram on your database server, but generally more than 100 clients at a time is a __very bad thing__. :tm: Additionally, PostgreSQL can only execute 1 query at a time per connected client, so pipelining all queries for all requests through a single, long-lived client will likely introduce a bottleneck into your application if you need high concurrency.
[Here is an up & running quickly example](https://github.com/brianc/node-postgres/wiki/Example)

With that in mind we can imagine a situation where you have a web server which connects and disconnects a new client for every web request or every query (don't do this!). If you get only 1 request at a time everything will seem to work fine, though it will be a touch slower due to the connection overhead. Once you get >100 simultaneous requests your web server will attempt to open 100 connections to the PostgreSQL backend and :boom: you'll run out of memory on the PostgreSQL server, your database will become unresponsive, your app will seem to hang, and everything will break. Boooo!
For more information about `config.ssl` check [TLS (SSL) of nodejs](https://nodejs.org/dist/latest-v4.x/docs/api/tls.html)

__Good news__: node-postgres ships with built in client pooling. Client pooling allows your application to use a pool of already connected clients and reuse them for each request to your application. If your app needs to make more queries than there are available clients in the pool the queries will queue instead of overwhelming your database & causing a cascading failure. :thumbsup:
### Pooling example

Let's create a pool in `./lib/db.js` which will be reused across the whole project

```javascript
var pg = require('pg');
const pg = require('pg');

// create a config to configure both pooling behavior
// and client options
Expand All @@ -70,18 +60,63 @@ var config = {
idleTimeoutMillis: 30000, // how long a client is allowed to remain idle before being closed
};


//this initializes a connection pool
//it will keep idle connections open for 30 seconds
//and set a limit of maximum 10 idle clients
var pool = new pg.Pool(config);
const pool = new pg.Pool(config);

pool.on('error', function (err, client) {
// if an error is encountered by a client while it sits idle in the pool
// the pool itself will emit an error event with both the error and
// the client which emitted the original error
// this is a rare occurrence but can happen if there is a network partition
// between your application and the database, the database restarts, etc.
// and so you might want to handle it and at least log it out
console.error('idle client error', err.message, err.stack);
});

//export the query method for passing queries to the pool
module.exports.query = function (text, values, callback) {
console.log('query:', text, values);
return pool.query(text, values, callback);
};

// the pool also supports checking out a client for
// multiple operations, such as a transaction
module.exports.connect = function (callback) {
return pool.connect(callback);
};
```

Now if in `./foo.js` you want to pass a query to the pool

// to run a query we can acquire a client from the pool,
// run a query on the client, and then return the client to the pool
```js
const pool = require('./lib/db');

//to run a query we just pass it to the pool
//after we're done nothing has to be taken care of
//we don't have to return any client to the pool or close a connection
pool.query('SELECT $1::int AS number', ['2'], function(err, res) {
if(err) {
return console.error('error running query', err);
}

console.log('number:', res.rows[0].number);
});
```

Or if in `./bar.js` you want borrow a client from the pool

```js
const pool = require('./lib/db');

//ask for a client from the pool
pool.connect(function(err, client, done) {
if(err) {
return console.error('error fetching client from pool', err);
}

//use the client for executing the query
client.query('SELECT $1::int AS number', ['1'], function(err, result) {
//call `done(err)` to release the client back to the pool (or destroy it if there is an error)
done(err);
Expand All @@ -93,27 +128,39 @@ pool.connect(function(err, client, done) {
//output: 1
});
});

pool.on('error', function (err, client) {
// if an error is encountered by a client while it sits idle in the pool
// the pool itself will emit an error event with both the error and
// the client which emitted the original error
// this is a rare occurrence but can happen if there is a network partition
// between your application and the database, the database restarts, etc.
// and so you might want to handle it and at least log it out
console.error('idle client error', err.message, err.stack)
})
```

node-postgres uses [pg-pool](https://github.com/brianc/node-pg-pool.git) to manage pooling. It bundles it and exports it for convenience. If you want, you can `require('pg-pool')` and use it directly - it's the same as the constructor exported at `pg.Pool`.
For more examples, including how to use a connection pool with promises and async/await see the [example](https://github.com/brianc/node-postgres/wiki/Example) page in the wiki.

It's __highly recommended__ you read the documentation for [pg-pool](https://github.com/brianc/node-pg-pool.git).
### Obtaining an exclusive client, example

```js
var pg = require('pg');

[Here is an up & running quickly example](https://github.com/brianc/node-postgres/wiki/Example)
// instantiate a new client
// the client will read connection information from
// the same environment variables used by postgres cli tools
var client = new pg.Client();

// connect to our database
client.connect(function (err) {
if (err) throw err;

// execute a query on our database
client.query('SELECT $1::text as name', ['brianc'], function (err, result) {
if (err) throw err;

For more information about `config.ssl` check [TLS (SSL) of nodejs](https://nodejs.org/dist/latest-v4.x/docs/api/tls.html)
// just print the result to the console
console.log(result.rows[0]); // outputs: { name: 'brianc' }

// disconnect the client
client.end(function (err) {
if (err) throw err;
});
});
});

```

## [More Documentation](https://github.com/brianc/node-postgres/wiki)

Expand Down Expand Up @@ -183,6 +230,10 @@ Information about the testing processes is in the [wiki](https://github.com/bria

Open source belongs to all of us, and we're all invited to participate!

## Troubleshooting and FAQ

The causes and solutions to common errors can be found among the [Frequently Asked Questions(FAQ)](https://github.com/brianc/node-postgres/wiki/FAQ)

## Support

If at all possible when you open an issue please provide
Expand All @@ -202,7 +253,7 @@ Follow me [@briancarlson](https://twitter.com/briancarlson) to keep up to date.

## License

Copyright (c) 2010-2016 Brian Carlson (brian.m.carlson@gmail.com)
Copyright (c) 2010-2017 Brian Carlson (brian.m.carlson@gmail.com)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
Loading