Tag Archives: postgresql

Why Not Auto Increment in MongoDB

Posted on August 4, 2010 by kgorman

I came across this blog post with a nice pattern for auto-increment in MongoDB. It’s a great post, but there is something to think about beyond how to logically perform the operation; performance. The idea presented in the blog is … Continue reading →

Posted in Data Architecture, Database Engineering, Mongodb, Python | Tagged , , , , | View Comments

Wayback Machine: snapshots still valid technique

Posted on May 10, 2010 by kgorman

I came across this old article I wrote for the NOCOUG newsletter in 2002 about using OS snapshots for backups. This technique is still very much a valid and widely used technique to perform backups. The idea is simple: – … Continue reading →

Posted in Database Engineering, Mongodb, Oracle, PostgreSQL | Tagged , , , | View Comments

Dropping ACID

Posted on March 15, 2010 by kgorman

The de-facto durability story in MongoDB is essentially… there is none. Or at least single server durability. OMFG! No ACID WTF! &^%#^#?! For the next generation of internet scale, downtime intolerant systems, ACID may not be a desirable property. Traditional … Continue reading →

Posted in Data Architecture, Database Engineering, Mongodb, MySQL, Oracle | Tagged , , , , , , , , | View Comments

pgstat 1.0 released

Posted on November 10, 2009 by kgorman

I added a couple of fixes to the code and released it as 1.0. We have been using it here at hi5 for some time w/o problems. Thanks everyone who has helped with feedback. Also thanks to Devrim GUNDUZ for … Continue reading →

Posted in PostgreSQL | Tagged , , | View Comments

pg_reorg 1.0.4

Posted on June 25, 2009 by kgorman

At Hi5, we currently use pg_reorg1.0.3 in order to organize data in a clustered fashion. I posted previously about the strategy. Our version is slightly modified, the modifications I made to the C code essentially allow pg_reorg to spin/wait for … Continue reading →

Posted in Database Engineering, PostgreSQL | Tagged , , | View Comments

Fusion-io SSD

Posted on May 27, 2009 by kgorman

I got the opportunity to test out some of the new Fusion-io Solid State ioDrive, and I thought I would post some results. Fusion-io has created a SSD product called ioDrive that is based on PCIe cards vs replacing SAS … Continue reading →

Posted in Database Engineering, PostgreSQL | Tagged , , , , | View Comments

Tuning for very large shared_buffers, using Veritas VxFS

Posted on December 12, 2008 by kgorman

The debate about the optimum shared_buffers size in PostgreSQL is clearly far from over. However, I have not seen any results where the buffer cache and the FS cache were tuned in unison. Because PostgreSQL knows what buffers are in … Continue reading →

Posted in Database Engineering, PostgreSQL | Tagged , , , | View Comments

Graphing pgstat output

Posted on November 5, 2008 by kgorman

If you’re using the pgd pgstat utility I posted about previously, you can graph the output with very little effort using gnuplot. In my case I use pgd pgstat for capturing output for various PostgreSQL performance tests, and of course … Continue reading →

Posted in PostgreSQL | Tagged , , | View Comments

pgstat: a database utility like iostat

Posted on October 21, 2008 by kgorman

I needed a utility for capturing various data-points about a PostgreSQL database as I performed load tests. I copied a utility I have used previously on Oracle that worked quite well. The new utility is called pgd pgstat. This utility … Continue reading →

Posted in PostgreSQL | Tagged , , | View Comments

Unique identifier for a database without connecting to the database?

Posted on September 16, 2008 by kgorman

In PostgreSQL, the working directory is a unique identifier for a database, and sometimes you want to use that working dir in your script(s). But what if you don’t want to actually connect and query the database? Is there a … Continue reading →

Posted in PostgreSQL | Tagged , | View Comments