Max in a tree! Talking about GeoCouch

I was at CouchCamp last week out at the Walker Creek Ranch – a bit disconnected (no cel service, and spotty internet), but fully immersed in the CouchDB community.

I was there to give a talk on MVCC in PostgreSQL. I forgot to mention it during my talk, but it was a fitting topic given that I first talked with JChris after a talk he gave in Portland, where I basically trolled him about compaction and MVCC in CouchDB. My goal was to show people the benefits of CouchDB’s built-in MVCC, to point out some places where core developers can learn from PostgreSQL and avoid some of the traps we’ve fallen into over the years. I’ve got more to say about the talk some other day, but I wanted to just reflect on CouchCamp for a moment.

One comment a friend made was, “Wow, these people are just so nice!” And it’s true. Every hacker meetup I attend is full of people who are overwhelmingly kind and thoughtful, and CouchCamp was more of the same.

CouchDB is at a critical point in their development – 1.0 is out the door, and developers are already building cool apps on top of it. CouchApps + Evently are an interesting and fun way to get started building things on top of a couch. And replication parties – seriously awesome. Ward Cunningham is rumored to be considering a CouchDB wiki to drive the patterns repository wiki (And here it is! Thanks, Max!), and CouchCamp was overflowing with ideas and implementations (distributed social, a replacement for email, UbuntuOne).

So what did I learn at CouchCamp? I learned how to hack on a CouchApp (Thanks for the help, JChris!). I learned about what Max Ogden is up to, and am so excited for him and the lucky folks that get to work with him. (and he’s running a hack/project night next weekend you should TOTALLY GO TO!)

I heard about the success and tribulations of running CouchDB on the desktop, and the launch of UbuntuOne from Stuart Langridge. During his talk, Stuart brought up the idea of a general replication API – something that I also believe is important to the growth of open source databases and is critical to enabling data freedom. I met a real, live Pick user/admin/developer, and talked about the inability to move to another system but the possibility of interfacing something like CouchDB to it. I got to chat with Rebecca Murphey about Javascript, MVCC and quality booze. I saw bunnies, foxes, deer, raccoons, and tons of bright stars late at night. And, I saw Damien Katz perform a brief interpretive dance.

I also was pointed to a retrospective on Couch 1.0 development by Ted Leung. I don’t know Noah Slater, but wow, what a testimonial. Noah’s comments about why he continues to contribute to CouchDB mirror a recent thread about PostgreSQL contribution — we work on these open source projects because of the incredible community that develops around them.

Thanks, Mikael, JChris, Jan and Damien, and all the CouchDB folks for creating a community that so many people want to contribute and become a part of. I certainly want to be a part of it, and look forward to finding ways of contributing more.

And thanks for bringing us all together in person. From the squirt guns in the welcome bag, to the campfire and sing-alongs, to the very late night Android libc storytelling by Aaron… These are the moments that glue us all together, and make all that work we do to connect up with one another through software completely worth it.

Thanks to everyone who attended OpenSQLCamp this past weekend in Portland, OR! More than 100 people participated – developers, DBAs and hobbyists from all over the world. Database developers participated from PostgreSQL, MySQL, MariaDB, Drizzle, TokuDB, LucidDB, MongoDB, Cassandra, CouchDB and many more.

The great thing about these events is the opportunity to trade ideas, code and stories. One project I’m very excited about is coming from some Portland State University students and a capstone project to create a new, interactive database client that works with more than just one DBMS. Igal gave a review of non-relational datastores. We had lightning talks about: open source column store databases, a many-master replication system called Trainwreck, open source at Microsoft, how to translate between NoSQL and SQL and many more.

You can see the full list of talks and notes from sessions as people update the wiki.

Joking about NoSQL aside, I was very happy to see many non-relational database developers in attendance, sharing information and participating in interesting discussions about the data management ecosystem. One meme we were happy to spread is that every tool has a purpose and I was happy to see this tweet:

I hope next time we can get a few more core Postgres developers to a Camp. Mark Callaghan expressed interest in a comparison of backend storage mechanisms, and several people were interested in detailed comparisons of replication strategies across many DBMSes.

Thank you to everyone who participated! (sorry I spelled your name wrong in the email, Mark. And left off your name in the list of GoDaddy road-trippers, Dan.) If you were there, please give us feedback!

We’re already looking forward to the next OpenSQL Camp. Some people thought we should do it again in Portland – and we’d be happy to host again next year! Baron also mentioned running an event in Washington, D.C.

For those that don’t follow the End Point blog, I just posted my slides for the Bucardo talk I gave this morning in seattle.

I attended the MySQL Conference all last week, and am feeling very excited about the future of open source databases. I had many interesting discussions and met a ton of Drizzle hackers I was lucky enough to spend Friday with, digging through code.

I was talking with Paul Vallée of the Pythian Group Thursday about Postgres and the future of enterprise support. And he showed me this great graph from indeed.com. It’s acceleration here, not the raw numbers – but still, a neat graph

postgres, mysql, oracle Job Trends

postgres jobs – mysql jobs – oracle jobs

We discussed the issues that enterprise customers with certain types of regulatory obligations encounter — such as contractual obligations for PCI-compliant credit card storage or outsourced management of sensitive data. The standard response developers might give for this is “read the spec, and make sure you implement it properly”. But the truth is, for larger companies, that may not be enough.

So, assuming for a moment that the Postgres community would even want to address this problem as a group — could it be possible for the Postgres community to provide the legal and financial assurances that an incredibly huge corporation (ahem – Sun/Oracle) can?

The short answer for Postgres right now is “no”.

Originally, I had thought just in term of liability, but Paul clarified:

The liability is just one component of what gives the guarantee meaning because there is a consequence to failed delivery. An SLA can also do this. As can a simple lucrative contract that can be lost, or canceled early if delivery does no take place. The key here is to ensure that the technology adopter can legitimately be confident that they are provably being responsible by adopting the platform. “I trusted” doesn’t cut it for many.

My view was that this type of agreement helps to determine who exactly is to blame (and who can be sued) in the event of a software failure. But, Paul said, “It’s more about assurance (with evidence) that obligations realistically will be met.”

I sometimes think that this system of liability and assurances is just ultimately broken. But it is a reality. So, would it be possible for us to come up with a new legal framework for community-driven software?

Paul brought up the idea of a cooperative, and that maybe such a legal entity could provide protection for individuals involved in supporting Postgres, and also shoulder some or all of the liability that a corporation using Postgres would want. I’m not sure that core developers of Postgres would join such a thing, or whether they would be allowed to given existing agreements they have with their own companies. But it is an interesting idea.

Creating a blueprint for this type of organization – hackers cooperatives – could be a way for truly community software to be developed across companies and among individuals in a sustainable, and “trustable” way. Maybe?

Continuing this train of thought – maybe these are non-governmental organizations, whose main purpose is to create and maintain infrastructure software for the good of the world.

Funding for mid-sized free and open source projects seems to be a consistent problem. Perhaps NGOs are a fair model for us.

I am curious about what effort may have already been made in this direction. My next step will be to contact Bradley Kuhn and see if there’s something out there that might address this.

Greg Sabino Mullane will be presenting three talks at PostgreSQL Conference East this weekend in Philadelphia, at Drexel University. The talks are listed on the site, and here’s what he’ll be speaking about:

Bucardo
April 5, Sunday, 10am
Bucardo is a replication system for Postgres that uses triggers to asynchronously copy data from one server to many others (master-slave) or to exchange data between two servers (master-master). We’ll look at replication in general and where Bucardo fits in among other solutions, we’ll take a look at some of its features and use-cases, and discuss where it is going next. We’ll setup a running system along the way to demonstrate how it all works.

Monitoring Postgres with check_postgres.pl
April 4, Saturday, 2:30pm
What should you monitor? And how? We’ll look at the sort of things you should care about when watching over your Postgres databases, as well as ways to graph and analyze metadata about about your database, with a focus on the check_postgres.pl script.

The Power of psql
April 4, Saturday 10:30am
All about everyone’s favorite Postgres utility, psql, the best command-line database interface, period. We’ll cover basic and advanced usage.

I’ve seen a few of Greg’s talks — The Magic of MVCC, Cloning an elephant and a few others. He’s a great speaker and cool guy. And he’s my boss. But I’m not just saying that because he’s my boss! Really!

He doesn’t like to brag about himself, so I’m gonna help him out. He maintains DBD::Pg, check_postgres.pl, Bucardo and has had MANY patches committed to PostgreSQL. He’s also a volunteer for the PostgreSQL sysadmins team, and specifically helps maintain the git repo box. He’s a contributor to the MediaWiki project. He’s on the board of the United States PostgreSQL Association. He’s basically awesome.

If you’re gonna be there, you should check out his talks. And if you can’t make it, here’s hoping Josh Drake records the talks and shares them with us all!

I’m in Brussels for the FOSDEM conference, hanging out at the PostgreSQL booth, meeting my European colleagues, and running into friends.

PostgreSQL has a developer’s room and Simon Riggs just wrapped up a talk about Replication. I sincerely hope that the video of the talk turned out well, because it was the most inspiring and technically interesting talk I have seen in a very long time. Unfortunately, I don’t have a copy of the slides at the moment, but word is that they will be posted on the BSD wiki soon.

Simon focused on new features in 8.4 that affect file-based replication, also mentioning streaming, synchronous replication — which will not be included in 8.4, but is being actively worked on. He explained his rationale for objecting to the inclusion of the synchronous replication patches, mostly, I think, based on the complexity of the WAL archiving required as it was implemented.

Then, Simon launched into an in-depth tour of the issues and solutions brought about during his team’s work on Hot Standby. Hot Standby allows read-only queries to be made against a file-based replication enabled Postgres server, known as Point-in-time recovery and WAL Shipping in the Postgres documentation.

Simon started work on PITR-related patches about five years ago, and continues that work with others today.

One fascinating aspect of the hot standby patches is that they ultimately caused performance improvements in sub-transactions across the board – and will likely cause up to 5% improvement in that code path. There were other performance improvements, but I’ll wait for the slides to mention those. At several times during the talk, Simon pointed out features that Postgres has that no other database has — such as multiple options for dealing with conflicts in hot standby (freezing, conflict resolution and timeout).

At the end of the talk, Simon spent a few minutes talking about how Postgres is capable of being the best database, not just the best open source database. And how all the people in the room were capable of contributing as he had. He claimed that prioritization and aiming to work on the biggest, most interesting problem you can are all you need. And he claimed that all that made him different was that he was a little more persistent about solving problems.

Rock on, Simon.

Update: Fixed the etherboot wiki link.

I attended the Google Summer of Code Mentor Summit this past weekend on behalf of PostgreSQL. We met at the Google campus in Mountain View.

This event was an unconference and so, none of the sessions were determined in advance.

Some of the highlights were:

• Leslie Hawthorn and Chris DiBona went into some detail with the whole group about the selection process for GSOC. This session made me feel as though PostgreSQL had relatively good chances for being accepted again next year. Google, however, does not pre-announce projects/products, so there is no sure thing about our (or any other project’s) involvement.
• I met MusicBrainz guys and was pleased to receive many bars of chocolate they requested to be distributed to SFPUG and PDXPUG members as thanks for making an great database.
• Attended three sessions concerning recruitment and retention of students. This is a topic that many people were interested in, but that few people feel they have a proper strategy for.

I also led a session on recruitment and retention of students to open source projects. Some of the ideas that came out of that and the related sessions were:

• Determine what makes you personally need to be part of Postgres (joy of learning, scratching a technical itch, making a tool for your job, fame). Find out which of those things your student also needs or wants and try to give that or help your student achieve that thing.
• Have a clearly defined method for students to keep journals. Several projects simply used MediaWiki and templates.
• Use git (or other distributed revision control), and have students commit early and often to a branch that mentors have access to.
• The Etherboot project has a great system: http://etherboot.org/wiki/soc/2008/start
• Hold weekly meetings over IRC. These can be brief, but help get students accustomed to your project’s culture and way of doing things.
• Ask the student: “are you on track?”, ask the mentor: “do you think the student is on track?” on a weekly basis
• If you want students to stick around, find incremental responsibilities to assign that are driven by their enthusiasm.
• Interview on the phone all your students ahead of time, not just the ones you think might be a problem.
• Require a phone number on the application for the student.
• Require a secondary contact so that if the student “disappears” there’s a backup person to contact. (and contact that person BEFORE SoC starts)

I made good connections with members of Git, Parrot, WorldForge, Ruby and many other community leaders. I was particularly impressed by the ideas and stories from the current Debian project leader, Steve McIntyre and Gentoo council member Donnie Berkholz. Donnie recommended some books about recruitment that I plan to read and review in the next few weeks.

The issue of mailing list moderation and the number of people required to keep mailing lists functioning properly came up frequently. If you know a moderator for a Postgres mailing list, please consider thanking them for doing a very tedious, extremely important and often thankless job.

I also spent some time discussing with Leslie Hawthorn and Cat Allman how to increase the total number of women mentors and students next year. Leslie and I shared some ideas and I offered to help implement them next year. One thing the crowd asked for was explicit training on how to recruit and manage female students. Realistically, this information will apply to all students, and I hope this training helps us recruit more students overall.

I thought the conference went quite well. I hope PostgreSQL is accepted next year, and that one of our mentors is able to attend this conference. And, if you go, be sure to register for the hotel early, and stay at the Wild Palms.

Heard of geekspeakr.com? Brenda Wallace is created a place to store the names and interesting details of women speakers for conferences.

I took a snapshot of the tag cloud so far:

Yay for PostgreSQL being the only database represented! So far!

Theo Schlossnagle wrote a great blog post about Joshua Drake’s keynote. I wanted to respond to some of his comments here.

I totally agree with Theo that we need to be disruptive! One criticism I have had in the past of the PostgreSQL community is that it has been too centralized, and not willing to experiment with social networking, wikis and other non-hierarchical tools that will get more *end* *users* actively contributing.

I think all of that is changing with the creation of the PostgreSQL-EU and USPgA groups, and the fast uptake in PostgreSQL User Groups in the last few months.

Also, I also respectfully disagree with some comments that were made in other sessions about “us not wanting” the type of users that choose MySQL. *I* want those users to come to my user group meetings. There’s tons we can learn from each other.

We will make PostgreSQL better if we hear and respond directly to criticism from users of the most popular open source database. I’m not saying that Tom Lane and Bruce Momjian need to do that. PUGs should be doing this, filtering out the good/important stuff and communicating information back to -hackers and -core. That’s a great service the user groups can provide to postgresql.org.

Finally, I am so excited about a Baltimore/Washington PUG!! I had several people approach me about a group. I think even the MySQL guy will be interested.

I just got home from the EnterpriseDB party at Jasper’s. What a fantastic day at the conference!

I attended most of Brent Friedman’s Java and PostgreSQL talk. I do a little Java hacking for a legacy app, so the sysadmin-ish details about installation and upgrading were useful as I look forward to upgrading.

I really, really enjoyed Andrew Sullivan’s talk on Rapid Development as Denial of Service Attack. Not only is Andrew witty and sharp, he just nailed the people problems that are inherent in modern software design. I really liked his presentation style as well – tons of useful analogies. Some key bits:

* Be sure to define what “normal” is – so that you can tell when things are not normal
* Build in time for instrumenting – because you’ll either plan to do it up front, or spend unplanned time later
* Databases are really an API, not a black box

Greg Sabino Mullane provided a survey of all the replication options available to PostgreSQL, including the pros and cons. This sparked a 20-minute debate among some long time PostgreSQL contributors and a few newer consultants and conference attendees.

The discussion revolved around what replication technologies to include in the core of PostgreSQL. In particular, the support for statement-level replication was in dispute, as well as the utility of self-modifying code. Greg also mentioned that Bucardo’s configuration engine uses Moose, a Perl object framework.

To wrap up the day, I led a discussion of women in open source, along with Dr. Kathryn Bartol,
Yixin Qiu, Dr. Louiqa Rashid, and Laura Thompson of Mozilla Corporation. My slides are below. Kay and Yixin also presented preliminary results from their research into activities of the KDE Women group. They were studying communication patterns and developing labels for the different types and quality of communication occurring between people on the mailing list.

I look forward to their work being published. I’ve attached my slides below. I have a few links to add before I add the final version to conference site.

A few more photos are also available on the Flickr Pool.

Women in Open Source slides – prelim