UpdatePDX: NOSQL, operational complexity and hiring

Last night, Tim Anglade, Bradford Stephens, Sarah Novotny and Alex Payne put together a three-part discussion to talk complexity, caching and collaboration, and in some cases, skewer popular notions of problem-solving around NOSQL.

Big thanks to Michael Schurter and Rick Turoczy for organizing and providing us space at PIE.

Tim was the thinker and designer behind the event. Many thanks to him for putting so much energy and time into it, and helping many leave inspired.

“Catchy phrases are red herrings.” -@timanglade

The night began with stories of failure: Bradford touched on the horror of 300-line SQL and the value of applying computer science to solve problems; Sarah recalled not having restorable data in an emergency because someone had tested a new backup strategy – for six weeks; Alex talked about the hard problems in computer science (ok, caching) and the value of culture and mentoring.

What I loved about this part of the evening, is that each person had a story. Technical presentations sometimes lack guideposts, but these stories all had villains, heros and a lesson.

“Usually they should just hire a MySQL consultant.” -@timanglade

And then Tim followed with a wickedly funny sendup of “the NOSQL movement”. The thing that struck a nerve and made the audience laugh uproariously was: “Only use NOSQL if you reach a certain point of despair.

His other points included: Never forget the operational complexity; Some things will always be better achieved with an RDBMS; Distribution Model vs. Data Model vs. Disk Data Structure (invoking the Moon Methodology); Hardware will always help, but it will never save you; Given enough time, most NOSQL projects gravitate towards a MapReduce-like model for computations (and querying); Trust no one. That goes double if they talk about CAP.

And then we talked.

The audience had plenty of questions, starting with how do we address collaboration in the context of scalability?

The answers from the panel, disappointingly, seemed to came down to separation of databases and people. While it’s true that it is easier to give out more databases than trying to communicate, it shows how far we have to go as an industry. Another point made was that it’s not typically possible to repurpose a DBA to maintain something like a Hadoop cluster.

In my notes, I wrote: “when problems are of a certain size, and affects the DNA of a company.” And when that happens, the typical separation of responsibilities between developer, DBA and operations break down.

Alex made my night by mentioning that sometimes solving a problem just takes getting a fresh set of eyes on it. He talked about replacing a particular bit of technology by applying some features of Postgres, with the help of a new hire.

I asked the final question – how does a company hire for the types of skills needed to solve these types of problems? Most agreed that finding someone with a passion for learning was critical. There was a dissenting voice – I think Bradford’s. My notes trailed off at that point, unfortunately. My guess is that he spoke up for logic and patience, and that some of the problems companies face have been largely solved, if you spend the time to study the science.

Epilogue

We wrapped up the night out at Little Big Burger and the Teardrop. Several people commented on how lovely Portland was – even in the rain. But damn, they wish they could hire the types of people they needed here.

So, people who are interested in big data: consider that a call to action.

Visit us for OSCON Data this summer. Then, move to Portland.

A year of PDXPUG

Last year was the third year that PDXPUG has been operating in Portland, and I decided to look back at our year of meetings. Here goes:

January 11 – 10 things you can use in PostgreSQL 8.3
February 26 – Extreme Database Makeover: RT
March 20 – Managing Internet Services: Using the right tool for the job
April 17 – Rails on PostgreSQL
May 15 – PostgreSQL for Pythoneers
June 19 – The relational model
July 20 – PDXPUG DAY!, and the schedule
August 21 – Tsearch2 and Materialized Views (Guest speaker from Seattle!!)
September 18 – The Visual Planner
October 16 – Point In Time Recovery
November 20 – Reviewed 8.4 features with the help of depesz’s blog
December – Coder’s Social

Thanks everyone who gave talks and attended meetings! User groups are only as good as the people who participate in them, and this list shows just how talented, diverse and fun the Postgres community is in Portland. I love you guys!

Looking forward – once again, we’ve already scheduled talks through the next four months! I feel like the group is running on its own momentum, and that is a fabulous feeling. We have a data visualization talk, another Extreme Database Makeover, and hopefully a presentation about teaching database theory with PostgreSQL.

Our next meeting is on January 15, 7pm with Stephen Jazdzewski traveling all the way from Eugene to present SplendidCRM, a formerly Microsoft SQL-only system that is now compatible with PostgreSQL. I am happy to see more of our Microsoft colleagues joining and presenting to the user group communities, as I’ve always felt they are underrepresented in our groups. Also, I’m happy to host another out-of-town presenter here in Portland! Hope to see you on the 15th.

Running a Successful User Group

running a successful user group

After the People For Geeks talk, I presented “Running a Successful User Group” with Gabrielle Roth on Wednesday. You can find our slides and our presentation handout over on Bacon and Tech. The handout is pretty cool, take a minute and print it out!