Colin Charles Agenda

Being the long-time Linux user that I am, there was no way I was going to be satisfied with Xandros, which is the stock Linux that ASUS ships with the Eee PC. I was stuck for choice between Ubuntu and Fedora, and after some careful evaluation, I decided that Fedora, was right for me.

Getting Eeedora, was pretty straight-forward. The installation wiki is pretty accurate when it comes to the “how” of installing Eeedora. For me, it was made easier that I had a Fedora system already, so I could run the tool to create Live CDs from the livecd-tool package.

One snag I noticed, and this is more with the Live CD script, is that when you don’t have a bootable USB thumb drive, it tells you that, but doesn’t quite tell you it didn’t make an installation on the thumb drive. I guess the script could be more idiot-proof. Anyways, making the drive bootable is easy.

Now, once that was done, it was on to installing on the Eee. I ensured that in the BIOS (accessed via pressing F2), the first boot device was the USB thumb drive attached to the system at boot-up. However, it was never booting into Eeedora, and I only managed to see the Xandros start up. Highly disappointing.

Turns out, that hitting the Esc key was the magic sauce, during boot-up. Only when doing that, was I given a boot menu, and then I could choose if I’d like to boot off the internal SSD, or the USB thumb drive. Once that was sorted, I was pretty happy to see a familiar Fedora-looking screen (sure, it said Eeedora, for legal reasons, but I think its a pretty darn good spin :P).

The installation process went on pretty smoothly. During partitioning, I was pretty much rid of Xandros – the chosen default was actually the most sensible. Just one / partition, filling up the entire disk, with no swap. As usual, anaconda (the Fedora installer) will warn you that not having a swap partition will be detrimental to performance – I wonder how many newbies might decide to create a swap partition (kind of a big no-no, on these SSD based devices).

One thing I did notice was the use of ext2 partitioning as opposed to a journalled filesystem like ext3. The natural question then comes to mind, as to why not just use jffs2?

Once the filesystem partitioning was sorted, and GRUB was chosen as the default bootloader, anaconda proceeded to install packages. This was a fairly speedy process, and once that was complete, it was a simple reboot, and I was booting off the SSD before I knew it.

You’re logged in by default as the eeedora user. You get a stock XFCE4 environment (or stock, that I would think anyway), and it comes with the basic utilities you need to get working (Firefox, wireless, etc.).

There were some things I obviously did not like, so decided to poke a little further. First up, was creating a user, that was not”eeedora”.

Dissecting the .bash_profile of the eeedora user, I noticed some things:

• pulseaudio -D – pulseaudio as a daemon for the user? Why, sound seems to work just fine on my Eee without it
• xsetroot -solid steelblue – seems OK, but not actually required anyway
• startxfce4 – if you want a GUI the moment you login, this is useful. But it only really works out well, because of the hack in /etc/inittab that says c7:12345:respawn:/sbin/mingetty --autologin=eeedora tty7. Seeing that I disabled autologin for the eeedora user, I pretty much see a login screen, and startxfce4 manually. Sometimes, I can actually get away with working in just a shell… However, I can see this from a usability point of view, I guess. Of course, the other issue is that the eeedora user, does not actually have a password!
• Logging out of XFCE4 and going back to the console, immediately triggers a machine shutdown. This seemed counterintuitive.

Needless to say, I disabled all this, in my new user account, as well as disabled autologin in the inittab.

Why XCFE4? Its a 900MHz Mobile Celeron processor, deliberately underclocked to 600MHz, with 512MB of RAM. This is the kind of environment, that can run GNOME. And when we notice problems with GNOME, its time we fixed it. Heck, I have an almost similarly configured IBM laptop, without PAE, that runs GNOME just fine!

I must commend WiFi Radar – this is something that pretty much, just works. And its small. It looks unmaintained, sadly, so its probably time to take an interest in it. Fn+F2 enables and disables wireless very well. I do however notice that even with the wireless disabled, the WiFi light (in blue), remains turned on. I find this to be rather quaint, and must prod it further.

Brightness control (Fn+F3/Fn+F4) seem to work well in Xandros, but I can’t seem to replicate such joy in Fedora. There is absolutely no reason why this shouldn’t work. It does annoy me, because its sucking precious battery life, from my usage of the Eee, by keeping it nice and brightly lit, when I’m on the battery.

Suspend and resume, just work. And because we’re dealing with an SSD, it just works, really fast as well. Sure, when I open up the screen of my Eee, I actually need to press the power button to get it to resume, but this kind of behaviour is perfect for me.

Do I like recompiling all the magic in /root/eee-setup, everytime I get a new kernel? No. Speaking with Dave Jones though, there is an expectation that all the drivers will be available in F9. Kudos.

Webcam? Works fine when I run lucview. Doesn’t work with the Skype beta that I get from the Skype download site. No idea what is missing, I just feel a bit bummed that I’ve got to find the solution to this issue at some stage soon.

As an install report, I think this does, just fine. Its now just time to hack on the distribution, to get it to the way I want it to work.

This month has clearly been an interesting one. Maybe, I should reserve it to be just this week.

On the weekend, I went looking for a rack, and came back with an Asus Eee PC. Today, while there were about 70 OLPCs given away to lucky, random linux.conf.au 2008 attendees, I wasn’t one of them.

However, I got one of the reserved units. Apparently, 100 units were shipped. Excellent.

Pack comes with a power supply, laptop, manual of some sort, and Jim gave me an Australian adapter (it comes with a US based plug).

Like I said in a previous post, I see great reason for the OLPC to rock. Now, I’ve got plenty of work ahead of me, to ensure thing do. Tasks include ensuring Fedora 9 boots off the Eee (I spoke to DaveJ about this, and the ideal plan is to make it installable by default – now its all driver hacking).

Excited, I am, naturally. Working on Fedora again. Whee!

The Eee has enabled me to work on the train and tram rides to get from South Yarra to Melbourne University. The OLPC, just got on my network (ifconfig -a eth0, find MAC, and voila! it works) and I’m sure to be using and hacking on it soon.

Time to sleep, these 9am keynotes are a killer. I am so not a morning person.

Some notes from Jim’s OLPC talk, which was full. I’ve added thoughts about (rural) Malaysia, because its naturally close to my heart. I’ve not had a similar opportunity to go around rural Australia (does it even exist?), and things are (I think) a lot better in Australia, than they are in Malaysia… I’m sure Jim’s talk video will be up soon, and his slides are worth going through – some interesting pictures there too.

Everybody wants the best for their kids. Most kids in the world, have no electricity at home :( Parents, themselves, might be illiterate or aware of the world. Teaching is not a valued profession by society, sometimes, and worse, books sometimes may not exist. Classrooms? You might not be so lucky.

So, what is the OLPC trying to achieve? A few hours of access in the computer lab every week, is not enough. Children lack opportunity, not capability. We should focus on high-quality education for all.

If schools exist in some places, there are insufficient classrooms. True, take Malaysia for example – there are two sessions to ensure folk get an education (morning, and afternoon sessions). This makes one get a fewer hours per day in school. Schools are often very crowded, so cables don’t really work.

The OLPC has a sunlight readable screen. This means classrooms can be outside. This won’t really affect Malaysians, seeing that there are actually classrooms. Electricity, might be hard to come by in some rural areas, but there are at least classrooms (or cabins, even).

In e-book mode, the OLPC runs for about 13 hours in B&W mode, and 8 hours in colour mode.

Parents tend to not be computer literate. Girls tend to have lower literacy than boys. Do we have fonts, for said languages? Does a computer interface (or even Sugar) apply, because we base our UI on metaphors?

Teachers, often only have 5-6 years of schooling themselves. Luckily in Malaysia, there are teacher training colleges, and you don’t become a teacher, unless you’ve gone through teacher training. However, teaching in Malaysia, does seem like an undesirable job (low paying, for example, might be an answer). One laptop/teacher? I honestly don’t even think this has been achieved in Malaysia.

What does one do about drop outs? Logistics, sometimes make it not feasible, for one to get to school, every day. Even in Malaysia, we know of people spending 2 hours to walk, to get to school. These stories move you, and you wonder, if its possible that they don’t have to go to school daily. People in the Klang Valley (read: Kuala Lumpur and surrounds) don’t even have electricity (or can’t afford it).

Regular laptops take about 10-20 watts, but the OLPC goal is 2W! They’re not there yet, but they’re getting there. 3-5W isn’t good enough.

A small child can generate 5-10 watts. Its a 21 watt battery. So the idea of hand cranking, too much is not good. Jim passed around a hand crank, and a solar panel. Both cost about the same, about USD$12-15. The solar panel does 5W, and costs about USD$12.

Mesh network, demands that the wireless is always on, otherwise, packets will not be forwarded.

Whats the CPU, on average, doing? Usually just refreshing the screen, and forwarding packets. Incidentally, the OLPC screens use LEDs – this makes them easy to recycle. The OLPC is an environmentally friendly laptop!

The OLPC actually turns off the CPU and most of the system – it just keeps the wireless and mostly, the display on. They use a DCON chip, and Marvell’s wireless chipset allows this to work well. During e-book reading, you’re only using 1.5-2.5 watts, depending on the screen and wireless use (so the 21 watt battery will do wonders).

Generators at schools, means that there’s a great cost, for fuel. And fuel prices are going up. So power for regular laptops, suck.

Jim highly recommends us to read Three Cups of Tea. Its been on the New York Times bestseller list for quite a while.

I noticed someone using the hand crank in the talk. The hand crank, can be mounted to a table, by the side, and then cranked. Very smart design. It however, does make noise. Imagine a classroom filled with folk, cranking their OLPCs? I don’t think the teacher will be very pleased.

The OLPC’s are hardy. You want to ensure they survive a rainstorm or a duststorm. Accidents happen, so allow kids to repair these things. A repair station can be very far away. The machine itself, can be assembled and disassembled, with a screwdriver. This could be interesting if its sent around Malaysia – rural areas in Sabah/Sarawak, and probably on the Peninsular, will definitely benefit from this.

Mountains are great for the network. Speaking to David Woodhouse earlier, and he mentioned that dry climates (like what we get in Melbourne, and generally the rest of Australia) is great for the mesh network. However, humidity, is generally not so good – this can affect Malaysia, especially during the monsoon season, the humidity goes up pretty high.

5 year life is aimed at. 2,000 battery cycles is the aim. Now, imagine replacing books? In Malaysia, 5 years means 5 years of either secondary education, or 5 years of primary education (well, its 6, but with the OLPC, I can imagine kids get to skip grades…) Books are really expensive, and while going to the schools are generally free, its not free to get books. Students going without books, is just silly. It realistically seems that the OLPC might be ideal to kids in primary school, rather than secondary school. I don’t actually know if its expandable for learning biology, chemistry, physics, etc.

The “Show Source” key exists. Learn, learning, by debugging!

The aim is to create children that will be both consumers, and creators. This I see benefiting not only Malaysia, but also Australia (ICT deficit in Australia is the largest in the budget, at something like $21 billion, and growing yearly).

Sugar: GTK+/Pango/ATK/Cairo. Python is what is used to tie it all together. Presence, allows you to build collaborative applications with each other. Take pictures, and share with people all around you. That’s what the “Share” activity in the UI is for.

How do you deal with parents who are suspicious with the OLPC, due to illiteracy? Involve them in the process. Do it upfront, don’t wait till later.

I’m actually highly moved by the OLPC. My recent travels to Vietnam, Cambodia, Thailand, India, China, and of course Malaysia, has made me think hard about what we need to do for spreading education and FLOSS. I’m sure I’ll talk about this more, later…

I’ve been very excited seeing that we announced the Maria Engine Preview. Giuseppe and I were trying to setup a demo, for Maria, at the lightning talks happening later today, towards the end of the MySQL MiniConf at linux.conf.au 2008. It involved MySQL with Maria, and an Asus Eee PC. For the demo, we wanted to show pulling the plug, which can be done either via a kill -9 `pgrep mysqld` or pulling out the battery of the Eee.

However, we failed to get MyISAM to reliably crash! Yes, imagine that, we actually want it to crash – pity that it might have only happened about 1/3rd of the time we tested it. The magic we were looking for:

check table t1;
+---------+-------+----------+---------------------------------------------------------+
| Table   | Op    | Msg_type | Msg_text                                                |
+---------+-------+----------+---------------------------------------------------------+
| test.t1 | check | warning  | 1 client is using or hasn't closed the table properly   |
| test.t1 | check | warning  | Size of datafile is: 23998464       Should be: 16000256 |
| test.t1 | check | error    | Recordlink that points outside datafile at 23000368     |
| test.t1 | check | error    | Corrupt                                                 |
+---------+-------+----------+---------------------------------------------------------+
4 rows in set (0.09 sec)

What we got instead was just a warning mentioning a client is using or hasn’t closed the table properly. Clearly, not so good for a demonstration.

When the magic of set storage_engine=maria; was run, and you crashed (either via pulling out the battery or doing a kill operation), Maria would survive the crash. At worst, you’ll see:

check table t1;
+---------+-------+----------+------------------------------------------------------------+
| Table   | Op    | Msg_type | Msg_text                                                   |
+---------+-------+----------+------------------------------------------------------------+
| test.t1 | check | warning  | Auto-increment value: 0 is smaller than max used value: 32 |
| test.t1 | check | status   | OK                                                         |
+---------+-------+----------+------------------------------------------------------------+
2 rows in set (0.20 sec)

So, Maria is clearly tough, as Giuseppe puts it. Give it a twirl (binaries, sources), and hop on over to the Maria Forums if you have questions. And if you’re checking out/building from source, you might find the Building MySQL from source guide handy

Alas, only a demo in a VM today . It seems more reliable inside a VM, not quite on the Eee. Besides, the poor SSD has been through quite a number of fsck’s now. BTW, one other thing to note: Maria log files seem to be pretty huge. Running it on a SSD that has about 1.5GB of free space (total, 4GB), is harsh.

For reference, the test (generally, wait for count to reach 128, switch terminals, and kill mysqld/pop battery off):

drop table if exists t1;

create table t1 (id int not null auto_increment primary key, b longblob) ;
select table_schema,table_name,engine
     from information_schema.tables
     where table_schema=schema() and table_name='t1';
insert into t1 values (1, repeat('a',1000000));
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;
insert into t1 select null,b from t1; select count(*) from t1;

Technorati Tags: mysql, maria, myisam, demo, storage engine

While I ducked out of Giuseppe’s miniconf talk, on MySQL Proxy (a great session, might I add – it takes up 2 slots right up until lunch), I went over to the LinuxChix miniconf, to attend a talk about memcache, by Brenda Wallace. Brenda, works at Catalyst IT, in New Zealand – they use a lot of memcache, in the telco business.

Memcache: volatile cache for keeping data in. Its a daemon. The code, can connect to memcache, put values in, read values, delete values. An example of how to use memcache, is given in PHP5.

A killer feature, is the setting of expiry. You can tell it to cache for 30 seconds, and then forget about it, no worries there.

What do you store? Database, generated content (front page of a website, just like a blog even), web service lookups (useful in telco, or say, if you’re playing with the Flickr API), LDAP, things that are far away (from the other side of the pond, etc.).

Wikipedia made memcache famous, Twitter uses it a lot, and there are probably heaps more.

Many APIs, and there’s a postgres client too. There’s a memcache storage engine for MySQL as well.

Code should be written such that if its in memcache, use that, otherwise, get it from the database and put it in memcache.

Another nifty feature, is incrementing a value – increment functions $memc->inc(‘name’);. You can also read stats, to see a cache hit or miss.

Memcache doesn’t have locks. Memcache is not atomic. There are other libraries out there to do locking, there is a known Perl library for this.

What not to store? Remember, its completely volatile. Don’t store anything you’d be sad to lose, and make sure the real copy is safe elsewhere. There is no method to get list of keys in store. There is a 1MB limit per item, so if data is larger than that, you’re in trouble.

Where do you run it? Remember, it is memory hungry, but CPU lite. Running memcache on the webserver is the recommended method, so beware of the security.

There is no authentication. You just connect (no username, no password). So, when running on the web server, you probably want a firewall. In shared hosting, everyone on that host can read/write to your memcache instance.

No check for validity. No referential integrity, its not a database.

There is transparent failover! So if one fails, the client just automatically connects it to another.

Usage ideas? Communicate between layers (talk to a PHP app, from Java). Instead of squid, you can store stuff in memcache, if you want.

Some competing technologies: Tugela – same concept, but its saved to disk, so it will survive a reboot. This is the Wikipedia fork, of memcached. Couch DB is mention, but its not really a competitor, seeing that its a document database. Lucene is another competitor, but remember, its a fast indexer, and its non-volatile.

I haven’t looked into memcached much, but its quite clear, its a great technology to look at. Now the fact that you can use the MySQL storage engine, it might actually be really, interesting.

Technorati Tags: linux.conf.au, lca08, linuxchix, memcached

Upcoming MySQL Features – Stewart Smith

Stewart’s talk on Upcoming MySQL Features was sort of a roadmap of what one might expect to see in MySQL 5.1 and above – he touched on Falcon, online backup, batched key access, Maria, Proxy, Workbench, and some cluster changes. When he shares his slides, it might be great to link to Worklog items, and Forge pages about these new features and previews (because, believe me, the stuff thats coming in future, is clearly very exciting).

MySQL Indexing Methods – Jonathon Coombes

Sitting in Jonathon Coombes session on MySQL Indexing Methods now – he’s going through covering indexes, the B+-tree index, hash index, full text indexing.

Some select points, that aren’t in the slides (otherwise, the slides themselves are very verbose, and when they make it online, it will provide some great reading material):

• InnoDB uses a B+-tree, and a secondary hash index.
• MyISAM has R-Tree index support, so it can be used quite usefully for GIS applications. MyISAM is not the only engine that supports spatial indexing, but its the only one that uses R-Tree indexes.
• T-Tree index is used in MySQL Cluster
• Touched on the Lucene search engine, with at least about 8 hands going up, as to people using it in production. It does allow live indexing, does proximity searches
• Sphinx, has a high indexing speed, and by default it sits outside the database, however you can set it to be a storage engine as well. Sphinx is distributed, so its similar to the way memcached is architectured. Distributed indexing. Full text fields.
• Bitmap indexes, not available yet, but its something people are waiting for

Bit weird seeing Jonathon now showing off OpenQuery, as opposed to Cybersite!

MySQL Optimisation by Design – Arjen Lentz
Its a training course that Arjen is working on, a one day event. He has no slides, per se, so he’s going to use the whiteboard instead. So this is something you’ve got to hope to get a video of, at least.

If you use MySQL, quite often you stick a database abstraction layer in between. However, you never usually ever port to another database, and the abstraction layer always leaks.

Arjen now walks us thru engines available, by firing up the MySQL monitor, and going through the output of show storage engines;.

Arjen then goes through a demonstration, on how many rows per second you can INSERT. These are 60-byte rows, from data acquisition styled information (IP addresses from a router, web clicks, etc.). Using MyISAM. No indexes. Sustained inserts of about 150,000 rows per second (40 million rows, in about 5 minutes, till his disk got full), on his Mac OS X Leopard laptop (and he can’t max anything out). This demo, is of course, for Arjen’s new gizmo, that’s written using the MySQL C API, that currently is in development. The application makes use of a multi-row insert.

Using Blackhole, it holds at about 245,000 rows per second, for INSERTs. So, we’re seeing the slowdown, in really, disk I/O.

Codename: lossylogger.

Its called lossy, because its allowed to loose some data. If you’re tracking web clicks, does it really make a difference, if you’ve lost some clicks, in the grand scheme of things? Not really. It makes use of UDP, so you’re not opening connections either. So, occasionally, you get a buffer overflow, and occasionally you lose data, but its no big deal, in the grand scheme of things.

Most interestingly, at the tail-end of Arjen’s talk, I see the MySQL Australia account manager (sales), Ralph, walking in. Yes, we have sales in Australia/New Zealand now!

Technorati Tags: mysql, mysql miniconf, linux.conf.au, lca08