Colin Charles Agenda

I haven’t done any data recovery or data rescue work in sometime (the last time was on Linux, with a combination of dd, ddrescue, and some throwaway code to parse JPGs – it was a Compact Flash card that needed saving). This time, all I had was macOS, a 16GB thumb drive, and the files were someone’s life’s work, which were more than just JPGs but also AI (adobe illustrator), DOC, XLS, PDF, TTF, etc. files.

So via Homebrew, I installed ddrescue again. A command like ddrescue -v -n -c 4096 /dev/disk2 helena.dmg helena.log seemed to work. On macOS, fdisk totally couldn’t get me anything useful and if I ran diskutil list the output would be as follows:

/dev/disk2 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   *15.5 GB    disk2

For good measure I wanted to also make an image via dd, dd if=/dev/disk2 conv=sync,noerror bs=4096 of=helena.img. It was clearly throwing many errors, an example of which:

13399375872 bytes transferred in 1263.864380 secs (10601910 bytes/sec)
dd: /dev/disk2: Input/output error
dd: /dev/disk2: Input/output error

The real problem was mounting either the DMG or the IMG. On Linux you have this option to loopback mount a file; on macOS this isn’t quite there. There is hdiutil but frankly, this doesn’t work if there is no partition record. I tried to mount it using hdiutil attach -noverify -nomount helena.img but that didn’t work to then do a diskutil mountDisk.

Then… I found a tool: PhotoRec. I wouldn’t have to write something to parse the magic numbers and extract files. PhotoRec just works. It parsed the IMG file, and spat out plenty of files to look at. Recovery was generally full.

For reference, on Linux, there are some good resources: Mounting a raw partition file made with dd or dd_rescue in Linux, and Guide to Using DDRescue to Recover Data. From a forensic standpoint, Disk-Arbitrator looks like a good tool as well.

I was thinking about buying a Mac Mini, but the Apple Store in Malaysia has over-priced it, due to the crazy Malaysian Ringgit (prices don’t reflect current realities; custom Mac Mini with all things thrown in is USD$1,499 vs RM6,899, today’s rate being RM6,435).

So I decided to buy an Intel NUC and go the Linux route. I picked up the Intel NUC NUC5i5RYH from CZone for RM1,643 which has an Intel Core i5-5250U processor, WiFi, Ethernet but requires you to provide some RAM, storage and you’re good to go. This is the model that allows an M.2 SSD and another regular 2.5” disk, so I chose to get the Transcend M.2 256GB SATA III 6Gb/s MTS800 to be the disk I’ll used to install an OS on for RM477 and decided that I’ll get another disk for storage/Dropbox purposes — HGST 7000rpm 2.5-Inch 1TB SATA III. RAM was easy – just make sure to get low power DDR3 RAM (DDR3L), and its easy enough to pick up 2*8GB sticks for a total of 16GB of RAM.

All in, I paid RM2,713 for this, and I provided by own MiniDisplayPort to VGA adapter. I see this as a huge savings over the Mac Mini. Sure, I can get a 2TB spinning disk on the Mac Mini (it seems that OEM folk can’t get these fusion drives at that size), but if I really wanted to go all out, I could have gotten a larger M.2 SSD and also went all in with SSD instead of spinning disk. Maybe when the 6th generation NUC comes out.

Configuring Ubuntu was relatively easy. Ubuntu 15.10 did require me to boot with the nomodeset option (so immediately after the visual BIOS splash screen, hit the Shift key, press e to edit the displayed kernel, and when it says ro quiet splash, edit it to say ro nomodeset quiet splash. You install Ubuntu via a USB thumb drive as well.

This is basically a server with X for me. It’s doing tasks like syncing Dropbox, backing up with CrashPlan, and it will allow me to use Docker containers, compile software, etc. while I’m sitting at my desk. It makes for a pretty mean desktop, all packed in a tiny little package. 

Why didn’t I go with the current i7? Seems like there wasn’t too much of a performance boost (good reading: Intel NUC Mini PC Review: Core i5 and i7 Benchmarked). The 6th Gen is also coming, so it will be a much more interesting platform for me (see the NUC6i5SYH; here’s hoping they also have i7 versions).

Facebook recently made opensource, osquery. It gives you operating system data via SQL queries! Its very neat, and you can test this even on MacOSX (it works on that platform & Linux). It is by far the project with the most advanced functionality, linked here in this post.

I noticed that rather quickly, there was a PostgreSQL project, called pgosquery, based on Foreign Data Wrappers with a similar idea. (apparently it was written in less than 15 minutes; so a much lower learning curve than the regular MySQL storage engine interface)

I immediately thought about an older MySQL project, by Chip Turner (then at Google, now at Facebook), called mysql-filesystem-engine. This idea was kicking around in 2008. I was intrigued by hearing about this at a talk (probably at the MySQL Conference & Expo); it’s a pity no one took this further.

On a similar tangent, did you also know that there is the option to use MySQL as storage via FUSE (see: mysqlfs)? An article by Ben Martin shows some practical examples.

At its heyday, MySQL had many storage engines (maybe around 50). Wikipedia has an incomplete list. I see some engines on that list, and think that some of these folk are also creating MongoDB backends – competition. At MariaDB we are probably shipping the most storage engines of any MySQL-based distribution, however I think we could be doing an even better job at working with upstream vendors, and figuring out how to support & augment business around it.

If you use Amazon Elastic Compute Cloud (EC2), you are always given choices of AMIs (by default; there are plenty of other AMIs available for your base-os): Amazon Linux AMI, Red Hat Enterprise Linux, SUSE Enterprise Server and Ubuntu. In terms of cost, the Amazon Linux AMI is the cheapest, followed by SUSE then RHEL. 

I use EC2 a lot for testing, and recently had to pay a “RHEL tax” as I needed to run a RHEL environment. For most uses I’m sure you can be satisfied by the Amazon Linux AMI. The last numbers suggest Amazon Linux is #2 in terms of usage on EC2.

Anyway, recently Amazon Linux AMI came out with the 2014.03 release (see release notes). You can install MySQL 5.1.73 or MySQL 5.5.36 (the latter makes the most sense today) easily without additional repositories.

The most interesting part of the release notes though? When the 2014.09 release comes out, it would mark 3 years since they’ve gone GA with the Amazon Linux AMI. They are likely to remove MySQL 5.1 (its old and deprecated upstream). And:

We are considering switching from MySQL to MariaDB.

This should be interesting going forward. MariaDB in the EC2 AMI would be a welcome addition naturally. I do wonder if the choice will be offered in RDS too. I will be watching the forums closely

Towards the end of last year, I was asked to investigate the Red Hat Software Collections by someone that popped by one of my talks. SkySQL has been working heavily with Red Hat, and with Fedora 19 shipping MariaDB as a default, it seems like MariaDB is getting even more distribution. The Red Hat Software Collections 1.0 Beta is now available for users of Red Hat Enterprise Linux 6.

From a database standpoint, users now get MariaDB 5.5. I encourage all to try it, as it is an in-situ upgrade. It is described as:

MariaDB version 5.5, which introduces an easy-to-adopt alternative for MySQL for Red Hat Enterprise Linux users. Binary compatibility allows MySQL users to drop-in MariaDB without converting data files.

Sweet. But for database users, it also includes MySQL 5.5 (better than 5.1.69), and PostgreSQL 9.2 (better than 8.4.13). After listening to Rasmus Lerdorf talk about PHP 5.4, I’m glad that I can now use it with RHEL6.

I wonder if there will be CentOS Software Collections as well?

Read an article in PCWorld about Software Collections (see a press release too). There’s some developer documentation from Red Hat, and some draft documentation from Fedora too.

Thanks SkySQL, Team MariaDB and Red Hat!

Retreat in sales of PCs turns into rout – FT.com: “‘At the beginning, retailers don’t know how to explain it to customers,’ says Mikako Kitagawa, principal analyst at Gartner. ‘Marketing the new [operating system] to consumers takes extra effort.’”

Well, skip Windows 8 altogether and just switch to Linux. And for more elegant hardware, there’s always the Mac.