Colin Charles Agenda

Here are some random thoughts, notes, observations, etc.

Jay Pipes – he made the conference possible

• Friendster uses Bugzilla internally. Yes they’re still alive, even though MySpace and so forth are around and kicking. Had to Google them (I wanted to find their old talk about their storage engine), and found MySQL Customers – Friendster, instead. From 2005, Dathan’s (now at Flickr) presentation. Never did find the storage engine stuff, beyond random bits in the press.
• Probably the best blog post that hit Planet MySQL, as opposed to the session summaries and so on, comes from Alexy Kovyrin. I quote: “P.S. Just remembered – I saw some women-DBAs today! Really smart girls! I never thought that pretty girl can become such great IT prefessional and now I know – I was really wrong.” Maybe we’ll have the MySQL-Women group, following on the LinuxChix.
• While Adam Donnison from the web team at MySQL gave his talk, I noticed Planet MySQL go down. Looks like the rest of the folk in his talk did too, and Jan Lehnardt says it best in Oh Irony!
• DorsalSource. Congratulations Jeremy Cole, and Solid. I am highly impressed with all the available binaries, from a trusted source. We’ve got the RHEL vs. Fedora vs. CentOS split now :-) (sure, building MySQL isn’t rocket science, as opposed to rebuilding an entire distribution, but consider DorsalSource+patches to be like the kbs-CentOS-Extras repository)
• Microsoft is also wanting to sleep with MySQL, these days. Must be a promiscious world we’re living in (they probably got jealous with IBM/DB2 already being in bed with MySQL). Read The Beautiful Game, and of course MySQL on Windows: A Beautiful Game. I expect this isn’t one off (Port25, Microsoft’s open source effort, has had a lot of open source database articles in recent times).

Some of my photos from the MySQL Conference & Expo – 26 April, 27 April. I wish I had more time & energy (& inspiration) to take more photos. Lenz Grimmer took some of the photos at dinner, as he was taken away by the 50/f1.4 lens and no usage of the magic flash (in fact, I didn’t even bring my external flash unit for the trip).

Mr. Eventum – at the hotel, while we all wound down

Dinner with some of the community members was fun, as always. Next year, more community members should stick around and we should all eat, meet and greet. The Fish Market has got good seafood (save for their lobster, which comes from… Australia. Not something I’d want!)

I was afraid, I thought he’d eat the power squid!

The MySQL community is just great. I’ve been suggesting that we get recordings for the Conference & Expo, alas, its generally not in the roadmap. I mean, look at Apple and their WWDC – all attendees get amazing video recordings that switch between slides and the speaker. Last year, they even delivered it via iTunes! In previous years, they distributed DVDs (valuable, though with WWDC a lot is generally new technology announcements, and I can hardly want to reference what was cool for Panther or Tiger any longer…) Mad props also to the linux.conf.au 2007 team, who also did amazing recordings – sessions were available by the evening they were given!

The MySQL conference is a lot different. There are lots of reusable sessions. Some that you attend, you’ll get knowledge committed for life. The tendency to not see too many roadmap talks makes it very useful for future reference.

Back to why the community rocks. They’ve done exactly what should have been done – record the sessions. Give much applause to:

• Sheeri Kritzer, for a lot of 2007 MySQL User Conference & Expo Presentations & Videos. Sheeri walked around with tripod, and video camera, and did an amazing job. She has MP3 audio and VMV video (it plays on Linux…)
• Baron Schwartz, has a few files, that are available in OGG Vorbis format.
• However, that’s not so good for iPod users, so Kevin Burton decided to make MP3’s of Baron’s recordings!

If that wasn’t enough, let me take a moment to thank all the Planet MySQL bloggers, who pretty much created content so regularly, that you could follow the conference, even while you were not there. Kudos to all!

Technorati Tags: mysql, mysqlconf, mysqlconf07, video, audio, recordings, new media, community

At the MySQL Conference the closing keynote was on Yahoo! Pipes, by Pasha Sadri, a Principal Software Engineer, Advanced Development Division, Yahoo!. I wanted to try it, but I was on Firefox 1.5 on Fedora Core 6 and there was no way I was going to build a pipe during the talk.

Fast forward a week or so later, and a boring Friday night ensued. What better thing to do, than to play with Pipes. In under fifteen minutes, I created the Edwin Pipe. What is it? Its a pipe that is all things MySQL – comprehensive source of news, whats cool, and so forth. There are some limitations – regular expression support is supposedly like Perl’s, but is not quite complete. The Unique operator is pretty cool, filtering is good (can be improved with better regex support), and maybe some sort of fuzzyness in the way data is displayed (I don’t only want all Digg mysql related items popping up at the top, or I don’t only want all mysql job forum details at the bottom, etc.). Language conversion via a Babelfish operator exists, but not language filtering (maybe I only want all English text displayed in my final pipe output).

That aside, the forums are pretty active. Pipes are ridiculously easy to create. Its simply great stuff. Oh, shorter URLs – the URLs are so long and not feasible, in my opinion. Impressive is the support to then get RSS output, and also JSON (so all processing is done on the server side). Happy I am with sites that provide JSON feeds.

Now, for some notes I took during the closing keynote.

• A while ago, he wanted to find an apartment near a park. Go to Craigslist and find apartment lists, then click the map link, and also check distance to a park on the map… This is tedious, and not automated.
• Craigslist apartment RSS feed. Yahoo! Local API to find Parks. Why not tie this in together? It started with about 50 lines of Perl code, and it combined feeds + web services (this is your Web 2.0 mashup).
• Pipes: free online service that lets you remix data and create mashups using a visual editor.
• Pipes treats the web as a big database, as they do joins across different ‘tables’.
• Design principles came from the Unix Pipes. They’re like pipes for the Web. Build useful applications from simple primitives.
• The more open Pipes is, the more useful it will be (so Google goodness will also work).
• Output available in JSON, so it can be used as another application. Get email or SMS from output, even. RSS is obviously available.
• App Examples: Last.fm + Flickr, Babbler (Second Life, language translation) by Max Case.
• Must enable users to solve ad-hoc problems. User generated “features” and disposable applications -> the future.
• Pipes uses MySQL, squid for caching, PHP & Perl (lots of CPAN modules) for serving and back-end processing of the pipes.

Edwin 2.0 is already in the works. It will have more cool feeds, and probably work out all the language issues with more separated regexes. More fuzzy organizing of data, if possible. If you want to see a MySQL Blogger Photo Gallery, bman_seattle created a pipe too.

Technorati Tags: mysqlconf, mysqlconf07, yahoo, pipes, edwin pipe, json, visual editing, mashup

Everyone likes to scale – Peter Van Dijck has got some top 10 presentations listed – Twitter, Flickr, LiveJournal, Six Apart (Vox), Last.fm, SlideShare, etc. Guess what these sites are all generally backed by? You guessed right – go MySQL. I however didn’t know that Bloglines was backed by Sleepycat.

If you’re interested in viewing some rather swanky MySQL-related presentations, check out the mysql tag on SlideShare, as well.

I especially like Brad Fitzpatrick’s LiveJournal: Behind the Scenes talk that he gave at Yet Another Perl Conference Asia 2007. The LiveJournal folk (Brad mostly?) have built some amazing tools that we all take for granted daily – memcached, perlbal, djabberd, OpenID and so much more. In fact, for those thinking about presenting at the MySQL Users Conference Japan 2007 (in September), I’d suggest looking at Brad’s slide deck – look at how he does dual languages! I don’t know if he wrote the Japanese as well, but I think this is definitely a good idea.

Now to go read up on mogilefs and all the other cool tools at Danga Interactive.

Technorati Tags: mysql, scaling, livejournal, slideshare, japanese, presentation

From the “I wish I attended this talk” department. When I attended Dathan’s talk about Federation at Flickr, he did mention that the next day there was going to be a really interesting talk by John Allspaw, who is an Engineering Manager at Flickr. John has posted up his slides, which I might add, are quite an interesting read (look at the speaker notes, they sort of give a heads up as to what you missed).

Its a pity no one made notes of this on Planet MySQL, so if anyone did attend the talk and you did take notes, please do place them online! This talk isn’t so much teaching you about capacity planning, its really more about all the pointers you can take away, about monitoring, graphing statistics, the fun of deployment, and of course some fun Flickr statistics (on slide 6, 18).

Pictures are fun! Capacity is not the same as speed (and it doesn’t mean performance). He mentions that you probably don’t want to read up about queuing theory (and should probably forget about benchmarks), because its mostly irrelevant to the real world. Testing in production is good, so don’t be afraid (I guess this is why Flickr has built-in notifications now).

Tools of interest: Ganglia, for pretty graphs, rrdtool, memcached, GraphClick (OS X only, and I wonder why they only get MRTG information in graphs, with no raw data at Yahoo!?). For deployment they use SystemImager (oh, I remember using this – it gets “fun” when you try to image dual-boot machines, but thats another story), and Subcon (interesting, I wonder how this compares to Slack).

Technorati Tags: mysqlconf, mysqlconf07, flickr, capacity, planning, scaling, LAMP, mysql

I missed a couple of talks that I’d really have liked to attend, for various reasons (probably the fact that at the MySQL conferences, staff also have a tonne of meetings and customers/people to meet). Thanks to the great bloggers, I don’t feel so bad for missing such talks. And for the ones with no blogs and notes, well, I’ll just hope and dream up the fact that sometime in the future, there will be video recorded sessions, available on the same day on the Internet, in OGG (like was done at linux.conf.au 2007).

Eric Lai has an interesting article in ComputerWorld titled: How Digg.com uses the LAMP stack to scale upward. And Mike Kruckenberg had some of his notes, which I think are also useful. Considering Digg.com don’t really publish information about their technology, or anything for that matter, its interesting to see how they’re scaling and running “the modern Slashdot”. Eli White also has the slides he used, at his website for his Technology @ Digg talk (yay! Only in OpenOffice.org, so go download it). He also has a blog thats worth following.

Sheeri Kritzer has some great information about the Japanese Character Set. Concise notes, its like I didn’t miss the talk. Will prove useful, as I plan on going to the MySQL Conf in Tokyo, right before our developer meeting this year.

Brian gives great tips and tricks on data warehousing, and I was a little upset I didn’t visit his talk. Again, Sheeri comes to the rescue: Data Warehousing Tips & Tricks.

Spoke to Eric Bergen a little while ago after drinks, and he was wondering how come I didn’t post about his tutorial with Jeremy Cole, on MySQL Scaling & High Availability Architectures. I do have a tonne of notes, that I’ve got to actually type up (yay, I have work to do on the plane ride home), but it’ll probably be useful to actually get the tutorial notes. My own notes will come later…

Technorati Tags: mysql, mysqlconf, mysqlconf07, digg, lamp, japanese, character set, data warehousing, linkblog