Cluster Developer Meeting recap

Cluster Developer Meeting

UPDATE: See bottom of post.

We held a PostgreSQL cluster developer meeting on Thursday, November 19, 2009 in Tokyo. About 25 people were in attendance, and seven projects presented status updates. Projects represented included pgCluster, PostgresForest, Postgres-R, Streaming Replication (slated to be included in core for version 8.5), Postgres-2 (not quite available), GridSQL, the Skype Skytools team (Londiste), Bucardo and Slony. Details of our discussions are being documented on the PostgreSQL wiki, and we’ve started a new mailing list.

The group of developers came up with a list of features that they would appreciate being implemented in Postgres soon, and we will be filling out the details in the coming weeks.

Our first milestone as a group is to create a detailed matrix of features to help users more easily navigate between the different solutions. I’m also going to arrange for a documentation sprint, dedicated to creating introductory documentation for new database administrators interested in clustering technology for Postgres. I’ll report out in December about how this work is going!

Josh has also posted a summary of the cluster meeting, and our next actions.

twittering on 2009-11-18

  • Having a sweet time in Tokyo with @magnushagander @pgsnake. Thanks to Hiroshi, Tado, Ishikawa for fugu dinner where I didn't die. Yet. #
  • @snaga no, tonight we are having a Japanese whiskey. in reply to snaga #
  • Having tea. Wondering who else is coming to the clustering summit from my hotel this morning. #pgcon09j #
  • in #postgres cluster developer meeting, listening to @fuzzychef talk about what we *should* be thinking about implementing #pgcon09j #
  • .@fuzzychef says: three main clustering use cases: transactional, analytic, online. #pgcon09j #
  • online is the use case that most matches the 'eventually consistent' model #pgcon09j #
  • follow what's happening in #pgcon09j: http://tr.im/Fg9C #
  • my notes from josh's presentation http://wiki.postgresql.org/wiki/ClusteringUseCases #pgcon09j #
  • now listening to what skype is up to with replication #pgcon09j #
  • skype working on SPLIT implementation in PL/Proxy (parallel exec of functions with partitioned data) and two-phase commit #pgcon09j #
  • skytools version three coming very soon #pgcon09j #
  • .@snaga actually, @turnstep (Greg Sabino Mullane) is the primary developer of #bucardo #
  • Postgres-2 hoping to open up the code by the end of 2009. and yes, please support APIs and hooks! #pgcon09j #
  • setting the afternoon agenda for the clustering summit discussion: community accessibility, improve user knowledge & access #pgcon09j #
  • @samv another giant leap forward for cross-continental trolling ;) in reply to samv #
  • @robtreat2 that issue was raised by the skype team themselves.. just hasn't been a priority so far, they say. in reply to robtreat2 #

twittering on 2009-11-18

  • Having a sweet time in Tokyo with @magnushagander @pgsnake. Thanks to Hiroshi, Tado, Ishikawa for fugu dinner where I didn't die. Yet. #
  • @snaga no, tonight we are having a Japanese whiskey. in reply to snaga #
  • Having tea. Wondering who else is coming to the clustering summit from my hotel this morning. #pgcon09j #
  • in #postgres cluster developer meeting, listening to @fuzzychef talk about what we *should* be thinking about implementing #pgcon09j #
  • .@fuzzychef says: three main clustering use cases: transactional, analytic, online. #pgcon09j #
  • online is the use case that most matches the 'eventually consistent' model #pgcon09j #
  • follow what's happening in #pgcon09j: http://tr.im/Fg9C #
  • my notes from josh's presentation http://wiki.postgresql.org/wiki/ClusteringUseCases #pgcon09j #
  • now listening to what skype is up to with replication #pgcon09j #
  • skype working on SPLIT implementation in PL/Proxy (parallel exec of functions with partitioned data) and two-phase commit #pgcon09j #
  • skytools version three coming very soon #pgcon09j #
  • .@snaga actually, @turnstep (Greg Sabino Mullane) is the primary developer of #bucardo #
  • Postgres-2 hoping to open up the code by the end of 2009. and yes, please support APIs and hooks! #pgcon09j #
  • setting the afternoon agenda for the clustering summit discussion: community accessibility, improve user knowledge & access #pgcon09j #
  • @samv another giant leap forward for cross-continental trolling ;) in reply to samv #
  • @robtreat2 that issue was raised by the skype team themselves.. just hasn't been a priority so far, they say. in reply to robtreat2 #

twittering on 2009-11-17

twittering on 2009-11-16

OpenSQLCamp was awesome!

Saturday schedule 11/14/09

Thanks to everyone who attended OpenSQLCamp this past weekend in Portland, OR! More than 100 people participated – developers, DBAs and hobbyists from all over the world. Database developers participated from PostgreSQL, MySQL, MariaDB, Drizzle, TokuDB, LucidDB, MongoDB, Cassandra, CouchDB and many more.

The great thing about these events is the opportunity to trade ideas, code and stories. One project I’m very excited about is coming from some Portland State University students and a capstone project to create a new, interactive database client that works with more than just one DBMS. Igal gave a review of non-relational datastores. We had lightning talks about: open source column store databases, a many-master replication system called Trainwreck, open source at Microsoft, how to translate between NoSQL and SQL and many more.

You can see the full list of talks and notes from sessions as people update the wiki.

Joking about NoSQL aside, I was very happy to see many non-relational database developers in attendance, sharing information and participating in interesting discussions about the data management ecosystem. One meme we were happy to spread is that every tool has a purpose and I was happy to see this tweet:

Best thing I learned at #opensqlcamp today: #nosql vs. #sql is a false duality. Different features for different problem domains.

I hope next time we can get a few more core Postgres developers to a Camp. Mark Callaghan expressed interest in a comparison of backend storage mechanisms, and several people were interested in detailed comparisons of replication strategies across many DBMSes.

Thank you to everyone who participated! (sorry I spelled your name wrong in the email, Mark. And left off your name in the list of GoDaddy road-trippers, Dan.) If you were there, please give us feedback!

We’re already looking forward to the next OpenSQL Camp. Some people thought we should do it again in Portland – and we’d be happy to host again next year! Baron also mentioned running an event in Washington, D.C.

Weekly tweet digest for 2009-11-15

twittering on 2009-11-14

twittering on 2009-11-13