Config Management Camp is becoming a great addition to the post-FOSDEM crowd. Short train ride away from Brussels to Ghent, lots of good content for an overnighter in a hotel (2d/1n). It routinely sells out (the cost is unbeatable, free ;)), so go early. Some quick notes:

Jonathan Boulle – CoreOS – etcd @baronboulle

• /etc but distributed with a clustered key-value store with GET/SET operations
• Support SkyDNS, Kubernetes, etc.
• CoreOS mission is to “secure the internet” and they like to provide automatic updates. Updates = rebooting. They didn’t like ZooKeeper so much.
• etcd: strong consistency guarantees, simple/fast HTTP API, OSS
• Uses the Raft consensus algorithm (replicated log to model a state machine; append only log). Concepts: leaders, elections, terms
• Written in Go, statically linked – etcd daemon and etcdctl cli

Service discovery using SmartStack – John Billings – billings@yelp.com

• http://nerds.airbnb.com/smartstack-service-discovery-cloud/
• Load balancing was done via something called create-lb-config and it was a lot of manual stuff which was deployed via puppet. Very labour intensive.
• They created something called PaasTa as a Paas (https://github.com/Yelp/paasta). And they found out about AirBnB’s SmartStack
• HAProxy allows you to re-dispatch on connection failures and easy to add connection logging as well. Each HAProxy instance also has a status page
• http://engineeringblog.yelp.com/2015/04/true-zero-downtime-haproxy-reloads.html
• Originally Nerve & Synapse made a Zookeper connection per service so it would make Zookeeper slow. Modify Nerve/Synapse to multiplex all operations over a single Zookeeper connection
• Use Zookeeper + AWS without issue for service discovery
• etcd would probably also work and supported by SmartStack but they already know Zookeper. SmartStack is conceptually similar to consul/consul-template

Platform Orchestration with Kubernetes and Docker

• Julian Strobl – julian@endocode.com – experts in linux/OSS dev, trainers, consultants located in berlin
• They have a 4-node Raspberry Pi cluster that one of them made
• There used to be a Master/Minion relationship, but now its a Master/Worker relationship when you are dealing in the Kubernetes world
• https://github.com/saturnism/gcp-live-k8s-visualizer

How CoreOS is built, modified and updated – Brian “redbeard” Harrington, Principal Architect, CoreOS

• @brianredbeard / brianredbeard.com
• repo sync – allows for managing many git repos at once
• cros sdk is used to build Chromium OS, but its truly an SDK (lots of bash scripts that do things the way you want them)
• OMAHA protocol created by Google to handle updates to desktop applications (eg. ChromeOS, Browser, etc.). There are OSS bindings provided by CoreOS in Go.

I have never been to Sofia, Bulgaria till this past February 2016, and boy did I enjoy myself. I visited the Bulgaria Web Summit and spoke there amongst many others. A few notes:

• Almost 800 people (so more than last year); hence the event was sold out
• Missed the RocksDB talk due to the massive Q&A session that went on afterwards.
• Very interesting messaging

LinvoDB

• LinvoDB (embeddable MongoDB alternative) — LinvoDB / www.strem.io
• Library written entirely in JavaScript without any dependencies. Converts any KV store to a MongoDB-like API, with Mongoose-like models, and live queries
• Use case: < 1 million objects (indexes are in memory using a binary search tree; so don’t use it for more). HTML5/Electron/NW.js. Best used with AngularJS/React and maybe Meteor. Can also use NativeScript or React Native. You can use it with node.js but its not recommended for server use cases.
• Works with SQLite or LevelDB (why not RocksDB yet?). Can also use with IndexedDB/LocalStorage
• Implemented almost entirely the MongoDB query language. Gives you automatic indexes.
• FTS in memory (linvodb-fts) – uses trie/metaphone modules for node.js. Can also do p2p replication, persistent indexes, compound indexes

My talk

I enjoyed speaking about MariaDB Server as always, and its clear that many people had a lot of questions about it. Slides. Video. It was tweeted that I had to answer questions for about as long as my talk, afterwards, and it was true :)

I got to meet Robert Nyman at the social event (small world, since he works at the office where Jonas of ex-MySQL fame does). Also met someone very interested in contributing to InfiniDB. It was nice having a beer with my current colleague Salle too. And speaking to the track moderator, Alexander Todorov was also a highlight – since we had many common topics, and he does an amazing amount of work around automation and QA. His blog is worth a read.

While being on the committee for the FOSDEM MySQL & friends devroom, I didn’t speak at that devroom (instead I spoke at the distributions devroom). But when I had time to pop in, I did take some notes on sessions that were interesting to me, so here are the notes. I really did enjoy Yoshinori Matsunobu’s session (out of the devroom) on RocksDB and MyRocks and I highly recommend you to watch the video as the notes can’t be very complete without the great explanation available in the slide deck. Anyway there are videos from the MySQL and friends devroom.

MySQL & Friends Devroom

MySQL Group Replication or how good theory gets into better practice – Tiago Jorge

• Multi-master update everywhere with built-in automatic distributed recovery, conflict detection and group membership
• Group replication added 3 PERFORMANCE_SCHEMA tables
• If a server leaves the group, the others will be automatically informed (either via a crash or if you execute STOP GROUP REPLICATION)
• Cloud friendly, and it is self-healing. Integrated with server core via a well-defined API. GTIDs, row-based replication, PERFORMANCE_SCHEMA. Works with MySQL Router as well.
• Multi-master update everywhere. Conflicts will be detected and dealt with, via the first committer wins rule. Any 2 transactions on different servers can write to the same tuple.
• labs.mysql.com / mysqlhighavailability.com
• Q: When a node leaves a group, will it still accept writes? A: If you leave voluntarily, it can still accept writes as a regular MySQL server (this needs to be checked)
• Online DDL is not supported
• Checkout the video

ANALYZE for statements – Sergei Petrunia

• a lot like EXPLAIN ANALYZE (in PostgreSQL) or PLAN_STATISTICS (in Oracle)
• Looks like explain output with execution statistics
• slides and video

Preparse Query Rewrite Plugins – Sveta Smirnova / Martin Hansson

• martin.hansson@oracle.com
• Query rewwriting with a proxy might be too complex, so they thought of doing it inside the server. There is a pre-parse (string-to-string) and a post-parse (parse tree) API. Pre-parse: low overhead, but no structure. Post-parse: retains structure, but requires re-parsing (no destructive editing), need to traverse parse tree and will only work on select statements
• Query rewrite API builds on top of teh Audit API, and then you’ve got the pre-parse/post-parse APIs on the top that call out to the plugins
• video

Fedora by the Numbers – Remy DeCausemaker

• Over 200,000 downloads a month for Fedora (at least this is where they could installer downloads). Upgrades don’t need an installer download though!
• https://github.com/fedora-infra/fedora-stats-tools

MyRocks: RocksDB Storage Engine for MySQL (LSM Databases at Facebook) – Yoshinori Matsunobu

• SSD/Flash is getting affordable but MLC Flash is still expensive. HDD has large capacity but limited IOPS (reducing rw IOPS is very important and reducing write is harder). SSD/Flash has great read iops but limited space and write endurance (reducing space here is higher priority)
• Punch hole compression in 5.7, it is aligned to the sector size of your device. Flash device is basically 4KB. Not 512 bytes. So you’re basically wasting a lot of space and the compression is inefficient
• LSM tends to have a read penalty compared to B-Tree, like InnoDB. So a good way to reduce the read penalty is to use a Bloom Filter (check key may exist or not without reading data, and skipping read i/o if it definitely does not exist)
• Another penalty is for delete. It puts them into tombstones. So there is the workaround called SingleDelete.
• LSMs are ideal for write heavy applications
• Similar features as InnoDB, transactions: atomicity, MVCC/non-locking consistent read, read committed repeatable read (PostgreSQL-style), Crash safe slave and master. It also has online backup (logical backup by mysqldump and binary backup by myrocks_hotbackup).
• Much smaller space and write amplification compared to InnoDB
• Reverse order index (Reverse Column Family). SingleDelete. Prefix bloom filter. Mem-comparable keys when using case sensitive collations. Optimizer statistics for diving into pages.
• RocksDB is great for scanning forward but ORDER BY DESC queries are slow, hence they use reverse column families to make descending scan a lot faster
• watch the video

The problem with Twitter is that we talk about something and before you know it, people forget. (e.g. does WebScaleSQL have an async client library?) How many blog posts are there about Percona Live Santa Clara 2015? This time (2016), I’m going to endeavour to write more than to just tweet – I want to remember this stuff, and search archives (and also note the changes that happen in this ecosystem). And maybe you do too as well. So look forward to more blogs from Percona Live Data Performance Conference 2016. In the meantime, here’s tweets in chronological order from my Twitter search.

• crowd filling up the keynote room for #perconalive
• beginning shortly, we’ll see @peterzaitsev at #perconalive doing his keynote
• #perconalive has over 1,200 attendees – oracle has 20 folk, with 22 folk from facebook
• #perconalive is going to be in Amsterdam sept 21-22 2015 (not in London this year). And in 2015, April 18-21 2016!
• We have @PeterZaitsev on stage now at #perconalive
• 5 of the 5 top websites are powered by MySQL – an Oracle ad – alexa rankings? http://www.alexa.com/topsites #perconalive
• now we have Harrison Fisk on ployglot persistence at facebook #perconalive
• make it work / make it fast / make it efficient – the facebook hacker way #perconalive
• a lot of FB innovation goes into having large data sizes with short query time response #perconalive
• “small data” to facebook? 10’s of petabytes with <5ms response times. and yes, this all sits in mysql #perconalive
• messages eventually lands in hbase for long term storage for disk #perconalive they like it for LSM
• Harrison introduces @RocksDB to be fast for memory/flash/disk, and its also LSM based. Goto choice for 100’s of services @ FB #perconalive
• Facebook Newsfeed is pulled from RocksDB. 9 billion QPS at peak! #perconalive
• Presto works all in memory on a streaming basis, whereas Hive uses map/reduce. Queries are much faster in Presto #perconalive
• Scuba isn’t opensource – real time analysis tool to debug/understand whats going on @ FB. https://research.facebook.com/publications/456106467831449/scuba-diving-into-data-at-facebook/ … #perconalive
• InnoDB as a read-optimized store and RocksDB as a write-optimized store — so RocksDB as storage engine for MySQL #perconalive
• Presto + MySQL shards is something else FB is focused on – in production @ FB #perconalive
• loving the woz keynote @ #perconalive – wondering if like apple keynotes, we’ll see a “one more thing” after this ;)
• “i’m only a genius at one thing: that’s making people think i’m a genius” — steve wozniak #perconalive
• Happiness = Smiles – Frowns (H=S-F) & Happiness = Food, Fun, Friends (H=F³) Woz’s philosophy on being happy + having fun daily #perconalive
• .@Percona has acquired @Tokutek in a move that provides some consolidation in the MySQL database market and takes..
• MySQL Percona snaps up Tokutek to move onto MongoDB and NoSQL turf http://zd.net/1ct6PEI by @wolpe
• One more thing – congrats @percona @peterzaitsev #perconalive Percona has acquired Tokutek with storage engines for MySQL & MongoDB – @PeterZaitsev #perconalive
• Percona is now a player in the MongoDB space with TokuMX! #perconalive
• The tokumx mongodb logo is a mongoose… #perconalive Percona will continue to support TokuDB/TokuMX to customers + new investments in it
• @Percona “the company driving MySQL today” and “the brains behind MySQL”. New marketing angle? http://www.datanami.com/2015/04/14/mysql-leader-percona-takes-aim-at-mongodb/ …
• We have Steaphan Greene from @facebook talk about @WebScaleSQL at #perconalive
• what is @webscalesql? its a collaboration between Alibaba, Facebook, Google, LinkedIn, and Twitter to hack on mysql #perconalive
• close collaboration with @mariadb @mysql @percona teams on @webscalesql. today? upstream 5.6.24 today #perconalive
• whats new in @WebScaleSQL ? asynchronous mysql client, with support from within HHVM, from FB & LinkedIn #perconalive
• smaller @webscalesql change (w/big difference) – lower innodb buffer pool memory footprint from FB & Google #perconalive
• reduce double-write mode while still preserving safety. query throttling, server side statement timeouts, threadpooling #perconalive
• logical readahead to make full table scans as much as 10x fast. @WebScaleSQL #perconalive
• whats coming to @WebScaleSQL – online innodb defragmentation, DocStore (JSON style document database using mysql) #perconalive
• MySQL & RocksDB coming to @WebScaleSQL thanks to facebook & @MariaDB #perconalive
• So, @webscalesql will skip 5.7 – they will backport interesting features into the 5.6 branch! #perconalive
• likely what will be next to @webscalesql ? will be mysql-5.8, but can’t push major changes upstream. so might not be an option #perconalive
• Why only minor changes from @WebScaleSQL to @MySQL upstream? #perconalive
• Only thing not solved with @webscalesql & upstream @mysql – the Contributor license agreement #perconalive
• All @WebScaleSQL features under Apache CCLA if oracle can accept it. Same with @MariaDB @percona #perconalive
• Steaphan Greene says tell Oracle you want @webscalesql features in @mysql. Pressure in public to use the Apache CLA! #perconalive
• We now have Patrik Sallner CEO from @MariaDB doing the #perconalive keynote ==> 1+1 > 2 (the power of collaboration)
• “contributors make mariadb” – patrik sallner #perconalive
• Patrik Sallner tells the story about the CONNECT storage engine and how the retired Olivier Bertrand writes it #perconalive
• Google contributes table/tablespace encryption to @MariaDB 10.1 #perconalive
• Patrik talks about the threadpool – how #MariaDB made it, #Percona improved it, and all benefit from opensource development #perconalive
• and now we have Tomas Ulin from @mysql @oracle for his #perconalive keynote
• 20 years of MySQL. 10 years of Oracle stewardship of InnoDB. 5 years of Oracle stewardship of @MySQL #perconalive
• Tomas Ulin on the @mysql 5.7 release candidate. It’s gonna be a great release. Congrats Team #MySQL #perconalive
• MySQL 5.7 has new optimizer hint frameworks. New cost based optimiser. Generated (virtual) columns. EXPLAIN for running thread #perconalive
• MySQL 5.7 comes with the query rewrite plugin (pre/post parse). Good for ORMs. “Eliminates many legacy use cases for proxies” #perconalive
• MySQL 5.7 – native JSON datatypes, built-in JSON functions, JSON comparator, indexing of documents using generated columns #perconalive
• InnoDB has native full-text search including full CJK support. Does anyone know how FTS compares to MyISAM in speed? #perconalive
• MySQL 5.7 group replication is unlikely to make it into 5.7 GA. Designed as a plugin #perconalive
• Robert Hodges believes more enterprises will use MySQL thanks to the encryption features (great news for @mariadb) #perconalive
• Domas on FB Messenger powered by MySQL. Goals: response time, reliability, and consistency for mobile messaging #perconalive
• FB Messenger: Iris (in-memory pub-sub service – like a queue with cache semantics). And MySQL as persistence layer #perconalive
• FB focuses on tiered storage: minutes (in memory), days (flash) and longterm (on disks). #perconalive
• Gotta keep I/O devices for 4-5 years, so don’t waste endurance capacity of device (so you don’t write as fast as a benchmark) #perconalive
• Why MySQL+InnoDB? B-Tree: cheap overwrites, I/O has high perf on flash, its also quick and proven @ FB #perconalive
• What did FB face as issues to address with MySQL? Write throughput. Asynchronous replication. and Failover time. #perconalive
• HA at Facebook: <30s failover, <1s switchover, > 99.999% query success rate
• Learning a lot about LSM databases at Facebook from Yoshinori Matsunobu – check out @rocksdb + MyRocks https://github.com/MySQLOnRocksDB/mysql-5.6 …
• The #mysqlawards 2015 winners #PerconaLive
• Percona has a Customer Advisory Board now – Rob Young #perconalive
• craigslist: mysql for active, mongodb for archives. online alter took long. that’s why @mariadb has https://mariadb.com/kb/en/mariadb/progress-reporting/ … #perconalive
• can’t quite believe @percona is using db-engines rankings in a keynote… le sigh #perconalive
• “Innovation distinguishes between a leader and a follower” – Steve Jobs #perconalive
• Percona TokuDB: “only alternative to MySQL + InnoDB” #perconalive
• “Now that we have the rights to TokuDB, we can add all the cool features ontop of Percona XtraDB Cluster (PXC)” – Rob Young #perconalive
• New Percona Cloud Tools. Try it out. Helps remote DBA/support too. Wonder what the folk at VividCortex are thinking about now #perconalive
• So @MariaDB isn’t production ready FOSS? I guess 3/6 top sites on Alexa rank must disagree #perconalive
• Enjoying Encrypting MySQL data at Google by @jeremycole & Jonas — you can try this in @mariadb 10.1.4 https://mariadb.com/kb/en/mariadb/mariadb-1014-release-notes/ … #perconalive
• google encryption: mariadb uses the api to have a plugin to store the keys locally; but you really need a key management server #perconalive
• Google encryption: temporary tables during query execution for the Aria storage engine in #MariaDB #perconalive
• find out more about google mysql encryption — https://code.google.com/p/google-mysql/ or just use it at 10.1.4! https://downloads.mariadb.org/mariadb/10.1.4/ #perconalive
• Encrypting MySQL data at Google – Percona Live 2015 #perconalive http://wp.me/p5WPkh-5F
• The @WebScaleSQL goals are still just to provide access to the code, as opposed to supporting it or making releases #perconalive
• There is a reason DocStore & Oracle/MySQL JSON 5.7 – they were designed together. But @WebScaleSQL goes forward with DocStore #perconalive
• So @WebScaleSQL will skip 5.7, and backport things like live resize of the InnoDB buffer pool #perconalive
• How to view @WebScaleSQL? Default GitHub branch is the active one. Ignore -clean branches, just reference for rebase #perconalive
• All info you need should be in the commit messages @WebScaleSQL #perconalive
• Phabricator is what @WebScaleSQL uses as a code review system. All diffs are public, anyone can follow reviews #perconalive
• automated testing with jenkins/phabricator for @WebScaleSQL – run mtr on ever commit, proposed diffs, & every night #perconalive
• There is feature documentation, and its a work in progress for @WebScaleSQL. Tells you where its included, etc. #perconalive
• Checked out the new ANALYZE statement feature in #MariaDB to analyze JOINs? Sergei Petrunia tells all #perconalive https://mariadb.com/kb/en/mariadb/analyze-statement/ …

I have a few speaking engagements coming up in April 2016, and I hope to see you at some of these events. I’m always available to talk shop (opensource, MariaDB Server, MySQL, etc.) so looking forward to saying hi.

• A short talk at the MariaDB Berlin Meetup on April 12 2016 – this should be fun if you’re in Berlin as you’ll see many people from the MariaDB Server and MariaDB MaxScale world talk about what they’re doing for the next releases.
• rootconf.in – April 14-15 2016, tutorial day on 16 – I’ve not been to India since about 2011, so I’m looking forward to this trip to Bangalore (and my first time to a HasGeek event). Getting the email from the conference chair was very nice, and I believe I’m giving a keynote and a tutorial.
• Percona Live Data Performance Conference 2016 – April 18-21 2016 – this is obviously the event for the MySQL ecosystem, and I’m happy to state that I’m giving a tutorial and a talk at this event.
• Open Source Data Centre Conference – April 26-28 2016 – Its been a few years since I’ve been here, but I’m looking forward to presenting to the audience again.

There’s some prep work for some internal presentations and tutorials that I’ll be running in Berlin at the company meeting as well.

There are a few upcoming speaking gigs this March 2016:

• flossUK Spring Conference – 15-17 March 2016, London
• FOSSASIA – 18-20 March 2016, Singapore

Two talks in London and one keynote in Singapore, while I will also moderate the database track on the last day of the conference as well. I’ve never been to flossUK before, but the schedule looks very interesting and I’m looking forward to it. FOSSASIA is an event I’ve been going to regularly since it was held in varying places (Vietnam, Cambodia, before finally settling down in Singapore) so this is an always exciting time.

Would happily meet you to talk shop, feel free to drop me an email: colin[at]mariadb.com.