The People of Postgres: Tom Lane

This post was originally posted on Medium, a new blogging platform made up mostly of people who aren’t necessarily subscribed to Planet. So, please forgive the obvious statements, as the target audience are people who don’t know very much about Postgres. Tom Lane, taken by Oleg Bartunov

Wednesday May 23, with no fanfare, Tom Lane’s move to was made public on the Postgres developer wiki.

For 15 years, Tom has contributed code to Postgres, an advanced open source relational database that started development around the same time as MySQL but has lagged behind it in adoption amongst web developers. Tom’s move is part of a significant pattern of investment by large corporations in the future of Postgres.

For the past few years, Postgres development has accelerated. Built with developer addons in mind, things like PLV8 and an extensible replication system have held the interest of companies like NTT and captured the imagination of Heroku.

Tom has acted as a tireless sentry for this community. His role for many years, in addition to hacking on the most important core bits, was to defend quality and a “policy of least surprise” when implementing new features.

Development for this community is done primarily on a mailing list. Tom responds to so many contributor discussions that he’s been the top overall poster on those mailing lists since 2000, with over 85k messages.

Really, he’s a cultural touchstone for a community of developers that loves beautiful, correct code.

Someone asked: “What does [Tom’s move] mean for Postgres?”

You probably don’t remember this: bases its entire cloud on Oracle database,” Ellison said, “but its database platform offering is PostgreSQL. I find that interesting.

When I read that last October, I was filled with glee, quickly followed by terror. I love my small database community, my friends and my job. What if Oracle shifted its attention to our community and attacked it, directly? So far, that hasn’t happened.

Instead, Salesforce advertised they were hiring “5 new engineers…and 40 to 50 more people next year” for a “huge PostgreSQL project.

Tom’s move probably won’t change much for the day-to-day operation of Postgres itself. Hopefully, things are about to get real at Salesforce.

I’m a major contributor to Postgres. I started in 2006, learning about relational databases through work at a small bike parts manufacturer and ERP. My contributions include code, starting conferences, encouraging user group leaders and introducing Postgres to communities that otherwise would never hear from us. I’m a data architect at Mozilla.

Distributed databases: a series of posts including 2-phase commit in Postgres

There’s a fantastic set of blog posts about distributed databases and network partitioning, starting with this post explaining the perils of trying to “communicate with someone who doesn’t know you’re alive.”

The next post is about Postgres and 2-phase commit. And there are four additional posts in the series.

The whole series worth reading for anyone interested in data stores, consistency and Postgres! 🙂

Migrations with Alembic: a lightspeed tour

I’ve got a Beer & Tell to give about alembic. Alembic is a migration tool that works with SQLAlchemy. I’m using it for database migrations with PostgreSQL.

So, here’s what I want to say today:

The most difficult thing to deal with so far are the many User Defined Functions that we use in Socorro. This isn’t something that any migration tools I tested deal well with.

Happy to answer questions! And I’ll see about making a longer talk about this transition soon.

PostgreSQL security releases now available: versions 9.2.4, 9.1.9, 9.0.13 and 8.4.17

PostgreSQL Global Development Group has just released updates for all currently supported versions of PostgreSQL.

From the release announcement:

The PostgreSQL Global Development Group has released a security update to all current versions of the PostgreSQL database system, including versions 9.2.4, 9.1.9, 9.0.13, and 8.4.17. This update fixes a high-exposure security vulnerability in versions 9.0 and later. All users of the affected versions are strongly urged to apply the update immediately.

A major security issue fixed in this release, CVE-2013-1899, makes it possible for a connection request containing a database name that begins with “-” to be crafted that can damage or destroy files within a server’s data directory. Anyone with access to the port the PostgreSQL server listens on can initiate this request. This issue was discovered by Mitsumasa Kondo and Kyotaro Horiguchi of NTT Open Source Software Center.

I wanted to highlight a couple things from the FAQ we developed for this release.

  1. There are no known exploits for the major security issue fixed by this release. The vulnerability was discovered through security testing conducted by NTT.
  2. Only users of 9.0 PostgreSQL and higher are affected by the major vulnerability.
  3. Affected users are those who allow unrestricted access to the network port PostgreSQL listens on. If you allow anyone, without IP address whitelisting, firewalling or some other kind of network-based access control, to connect to your network port, you are especially vulnerable.

Upgrading from minor version (9.2.3 to 9.2.4, for example) only requires that you install the new binaries and then restart PostgreSQL.

Additionally, if you are using GiST indexes, read the detailed notes in the release announcement to see if you are using features that require you to REINDEX your GiST indexes.

Please update as soon as possible!

Many thanks to our volunteer packagers who worked hard for the past several weeks to make this release possible. All PostgreSQL software releases are managed by volunteers.

A rosetta stone for Mac OS X installers for PostgreSQL

I’m no longer using Mac OS X for my primary desktop, but many of my coworkers and friends do. Particularly developers writing applications that use PostgreSQL (aka Postgres) for their data storage.

I’ve spent a lot time over the last few years troubleshooting people’s Postgres installs in the following, very common, situations:

  • A developer installed Postgres on their Mac laptop >1 year ago
  • Now they need to upgrade their Postgres to help me, or support a new application that needs new features
  • They have an old database they’d like to migrate to the new version
  • They have no idea which particular Mac OS X installer they used last time

For this exact situation, I have documented some features of the Mac OS X Installers for Postgres.

And, I felt so good to see this right after I posted the wiki page earlier today:

@zacduncan: “@selenamarie This is helpful to me at this very moment. Thank you. ”


Save the Ada Initiative

If you believe that women are a crucial part of the future of free and open source software, you should give to the Ada Initiative.

If you think we should have more women contributing, talking about and using free and open source software, you should donate to the Ada Initiative today.

I spent this past summer working with Mary, Valerie and the many supporters and contributors to the Ada Initiative. I talked to past donors, and spent a lot of time writing and thinking about how the Ada Initiative has evolved.

I met hundreds of people in person and online who believe not only that the Ada Initiative is a crucial advocate for change in the world of open source, but that establishing gender balance in open source through their work is a worthwhile, achievable goal. That work includes research, writing, training and creating culture and community specifically designed for women to flourish.

They’ve created strong relationships across project, business and ideological boundaries, through their board, advisors and AdaCamps.

I’m a member of the Advisors board, a major contributor to PostgreSQL and a data architect at Mozilla. These relationships have formed into a strong, diverse and visible alliance of women in open technology.

Because of the Ada Initiative’s work, I have seen an important shift from identifying problems to seeking solutions among my colleagues in open source. This work is made possible because TAI provides full-time employment to focus, write about and act on these solutions. Their work cannot continue without your support.

Between now and October 31, you can be the crucial donors who made this organization succeed in 2012. If you work for Microsoft, Google or Red Hat your donation with be doubled thanks to charitable giving matching programs. And individuals like Sumana Harihareswara and Leonard Richardson are sponsoring matching grants.

Social change is never easy, and the organizations like the Ada Initiative, who chose to step into the void, need our support.

Take a few minutes and give to the Ada Initiative, to Mary and Val, and help their work continue in 2013.

Wrapping up Postgres Open, new job, shift away from twitter

Last week in Chicago was amazing! 37 speakers, an incredible keynote by Jacob Kaplan-Moss (video coming soon!) and re-connecting with all the great people in Chicago. We announced a new conference committee for next year’s conference, and said we’d do it again in September in Chicago! That group is just getting started now, and will have some announcements for everyone in the coming weeks.

I’m going to be busy with a new job at Mozilla starting Monday, working on databases with the WebTools team.

Another small change is: I’m writing a few times a day to my tumblr and I’ve just stopped using twitter for the next few weeks. In the last day, I have really only thought about one or two things to share that would have been more than fleetingly useful. As I come across things, I’ll be sending them to the tumblr instead.

I’m also looking for patches and projects to work on for Postgres itself. During Thursday’s code sprint, I picked up an old patch for config directories, and today I spent some time re-generating a list of contributor names for the last 5 major versions of Postgres.

As usual, I feel so energized from hanging out with my favorite Postgres people. I’m only sad that I won’t see most of them in person again until next year.

What features do developers get excited about in Postgres?

I’m here at DjangoCon in Washington, DC and thinking about what it is that developers are currently excited about in Postgres.

Postgres hackers are often very focused on solving our own problems, problems people bring up on our mailing lists and dealing with database scaling, replication and data management.

Developers using Postgres seem more interested in the features which make creating applications easier and removing complexity from architecture.

So, what are they interested in?

The features that I hear mentioned most often include:

(thanks to @ipmb for the list in a lightning talk today!)

What are the features you hear about from developers? Or if you’re a web developer, what are your favorite features in PostgreSQL?

While we’re here, let’s fix computer science education: DjangoCon keynote and resources

My keynote today is done, the resources list is here and the slides are below. I wrote slightly different text to address our experience here in the US, but a mostly-complete transcript of the talk is here.

A ton of people came up to me after the talk and we started talking about all the ways that we might be able to solve problems. I created a mailing list for our first few discussions. If you are a person that doesn’t like google groups, contact me, as I of course can set up something that’s outside of that infrastructure if we have enough people who’d prefer a different place to have this conversation.

We have a plan to contact teachers in our local communities, and ask them what they need that we as open source software developers could help them with. And we all agreed that want to build things, but we’re pausing for a minute to ask the teachers around us what they need first.

For some background, the key bits of reading you should do to get up to speed are the following:
Continue reading