Calculating significance indicates insignificance

Calculating significance indicates insignificance -- Jasper Nuyens, 2011

Joan Halifax's "Compassion"

Equanimity is a state of mental or emotional stability or composure arising from a deep awareness and acceptance of the present moment.

Great TED talk: Joan Halifax: Compassion and the true meaning of empathy

How to replace a failed disk of a RAID 5 array with mdadm on Linux

This is easy, once you know how it's done :-) These instructions were made on Ubuntu but they apply to many Linux distributions.

First of all, physically install your new disk and partition it so that it has the same (or a similar) structure as the old one you are replacing.

Then, install mdadm if you haven't already:

sudo apt-get install mdadm

Optional but might be a good idea if you can: reboot to make sure the whole md (multi-disk) software stack is loaded.

Let's check the status of our disks - mdadm should have discovered at least *something* on boot:

evy@evy-server:~$ cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]

md0 : inactive sdb2[1](S) sdc2[2](S)

3858200832 blocks

So this looks like it created one RAID device (/dev/md0) that consists of /dev/sdb2 and /dev/sdc2. We are need at least 3 disks for our RAID 5 so let's add another one:

evy@evy-server:~$ sudo mdadm --manage /dev/md0 --add /dev/sda3

mdadm: cannot get array info for /dev/md0

Hmm, this doesn't work because the RAID array in /dev/md0 is not active.
Let's activate it then:

evy@evy-server:~$ sudo mdadm --assemble /dev/md0

mdadm: no devices found for /dev/md0

Hmm, mdadm is not aware of the physical disks that /dev/md0 consists of. That's strange, because that information is clearly visible in /proc/mdstat, remember ?

A look at the man page reveals the --scan option, that tells mdadm to scan /proc/mdstat (and others) for RAID device information.

Let's try again:

evy@evy-server:~$ sudo mdadm --assemble /dev/md0 --scan

mdadm: /dev/md0 assembled from 2 drives - not enough to start the array while not clean - consider --force.

Ok, so we knew that - the RAID 5 is missing a disk.
But we need to activate it anyhow, to be able to add the missing disk !

So we force it:

evy@evy-server:~$ sudo mdadm --assemble /dev/md0 --scan --force

mdadm: SET_ARRAY_INFO failed for /dev/md0: Device or resource busy

Hmm, something's keeping our RAID array busy...
Let's just restart that RAID volume to fix that:

evy@evy-server:~$ sudo mdadm --stop /dev/md0

mdadm: stopped /dev/md0

And try again:

evy@evy-server:~$ sudo mdadm --assemble /dev/md0 --scan --force

mdadm: /dev/md0 has been started with 2 drives (out of 3).

Success !
Let's see what happened just now: 

evy@evy-server:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sdb2[1] sdc2[2]
      3858200832 blocks level 5, 64k chunk, algorithm 2 [3/2] [_UU]

Aha, there is movement !
The RAID 5 array is online, and the [_UU] flags show us that one disk is missing.

Here's some more info that demonstrates this nicely:

evy@evy-server:~$ sudo mdadm --detail  /dev/md0
/dev/md0:
        Version : 00.90
  Creation Time : Mon Feb  7 18:27:14 2011
     Raid Level : raid5
     Array Size : 3858200832 (3679.47 GiB 3950.80 GB)
  Used Dev Size : 1929100416 (1839.73 GiB 1975.40 GB)
   Raid Devices : 3
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Apr 25 13:04:09 2011
          State : clean, degraded
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : 90b591c9:2fd9536f:86e10ba8:fe4e4a37
         Events : 0.172

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       18        1      active sync   /dev/sdb2
       2       8       34        2      active sync   /dev/sdc2

So let's try to add that missing disk, which had been our intent all along.

evy@evy-server:~$ sudo mdadm --manage /dev/md0 --add /dev/sda3
mdadm: added /dev/sda3

And this works.
Check it:

evy@evy-server:~$ sudo mdadm --detail /dev/md0
/dev/md0:
Version : 00.90
Creation Time : Mon Feb 7 18:27:14 2011
Raid Level : raid5
Array Size : 3858200832 (3679.47 GiB 3950.80 GB)
Used Dev Size : 1929100416 (1839.73 GiB 1975.40 GB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Wed Jun 29 23:39:54 2011
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Rebuild Status : 0% complete

UUID : 90b591c9:2fd9536f:86e10ba8:fe4e4a37
Events : 0.176

Number Major Minor RaidDevice State
3 8 3 0 spare rebuilding /dev/sda3
1 8 18 1 active sync /dev/sdb2
2 8 34 2 active sync /dev/sdc2

The RAID has already started rebuilding!

We can watch the progress like this:

evy@evy-server:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sda3[3] sdb2[1] sdc2[2]
3858200832 blocks level 5, 64k chunk, algorithm 2 [3/2] [_UU]
[>....................] recovery = 0.0% (673180/1929100416) finish=429.6min speed=74797K/sec

Aaaaaaah, there's no greater time to go to bed than when your RAID 5 is recovering :-)

Reverse engineered Logitech C110 color changing driver

I reverse engineered the windows driver of my Logitech C110 keyboard and remade it for Linux (with libusb) so that we can now change the color of the led lights in any shade between red and blue.

Attached is a proof of concept that I keep running all the time. It slowly, gradually makes the backlight color change from blue to red and back.

This open source Linux driver outperforms the proprietary windows driver because it enables continuously changing the backlight color in addition to setting just one, fixed color.

You can find it here: http://code.google.com/p/logitech-keyboard-change-color/downloads/list

The most interesting line of code (MILOC) is this one:

int written = usb_control_msg(handle, CONTROLFLAGS, 0x00000009, 0x00000307, 0x00000000, buffer, 0x00000005, 5000);

where:  

• handle is a pointer to the (properly initialized) USB device 046d:c22b
• #define CONTROLFLAGS 0x21 hold the RequestTypeReservedBits (as per the USB specs)
  
• unsigned char buffer [] = {0x07, 0x00, 0x00, 0x00, 0xFF}; is sent to the keyboard where:
  
• buffer[1] determines the led color (between 0x00 = blue and 0xFF = red)
• buffer[4] determines the led intensity  (between 0x00 = off and 0xFF = maximal) 

I hope you enjoy this keyboard as much as I do, and I hope Logitech starts shipping more of their products with Linux compatibility !

Or at least throwing some documentation online so we don't have to reverse engineer these things all the time :-)

Enjoy !

Git'Shit

What a great name for a revision controlled toilet: a GitShit !

A friend made an image with Acronis in an undocumented? proprietary format that looks like this:

00000000  b4 6e 68 44 ae 2e 9e f2  01 00 00 00 66 63 66 60  |.nhD........fcf`|
00000010  60 70 66 b0 62 88 61 60  04 b2 18 00 4f f6 0e eb  |`pf.b.a`....O...|
00000020  01 13 63 60 60 00 00 5f  d7 36 54 05 93 fe e6 c8  |..c``.._.6T.....|
00000030  a6 c7 c8 c0 b0 41 f0 91  ba 26 90 06 00 2c 3c 71  |.....A...&...,mmI...|
000000c0  f0 e8 0f a7 64 56 ad 15  c5 24 91 25 3e 5e 3e d7  |....dV...$.%>^>.|
000000d0  e1 a4 b1 2b ac 61 6f e6  bb 81 99 92 50 c6 16 e6  |...+.ao.....P...|
000000e0  86 f9 bf 4b f3 07 f9 ad  cc 0f a5 24 82 29 88 ab  |...K.......$.)..|
000000f0  ed 4b 53 9a df 1c bb 24  0d 8e c1 d6 ef 9c 1c 92  |.KS....$........|
00000100  3e 39 2e be f4 4a 4e ce  c9 59 e9 c7 bb 0f 83 bb  |>9...JN..Y......|
00000110  41 58 fb a4 4d 9a 64 29  79 2c f0 cc 24 45 95 d3  |AX..M.d)y,..$E..|
00000120  9e 24 da 3c 7d 2e c3 48  fc da 4c 54 be 00 e2 cb  |.$.<}..H..LT....|
00000130  f1 d9 59 0e 3e 9d 06 43  bd ee 8c 1a 16 d8 97 9f  |..Y.>..C........|
00000140  44 65 a7 3e ed 9c 5c 94  23 72 14 6c 36 4a 06 7f  |De.>..\.#r.l6J..|
00000150  7c b0 39 24 a7 f4 20 d3  d3 f0 1e 07 1f 7f d1 3f  ||.9$.. ........?|
00000160  4d c0 5c b3 df 2a 68 ce  fb 24 ef 85 ea cc bf 8f  |M.\..*h..$......|
00000170  56 f1 5d 9d 59 63 51 85  fd b5 3a 4b 13 26 ce d4  |V.].YcQ...:K.&..|
00000180  b6 45 57 b8 07 95 1c 46  4d 3f 60 9c 82 e6 47 6d  |.EW....FM?`...Gm|
00000190  8d ec 49 0e c3 74 60 71  fd ab 87 d7 17 75 05 ea  |..I..t`q.....u..|
000001a0  0a d4 15 a8 2b 50 57 e0  33 56 a0 6b 92 ff 0f e2  |....+PW.3V.k....|
000001b0  5b 21 54 ca f3 33 66 2a  ca af 24 4f 7a ce c6 25  |[!T..3f*..$Oz..%|
000001c0  a3 7d 22 fd cf 11 33 7d  35 1f 43 0c bf 1b 18 fb  |.}"...3}5.C.....|
000001d0  a3 2a e5 6f 60 b4 25 d2  27 58 de 5b 89 aa 20 c6  |.*.o`.%.'X.[.. .|
000001e0  48 00 77 2d fc 5d 8d 06  97 b1 45 f8 62 89 52 fe  |H.w-.]....E.b.R.|
000001f0  8e c5 1d c2 67 d6 04 d6  b4 1d c3 20 6e cb f2 0f  |....g...... n...|
00000200  71 99 1d 79 36 3f 57 62  4e d8 f3 fc 7f 0c d7 ef  |q..y6?WbN.......|
00000210  80 99 c1 9a 98 db 17 c9  4b c2 77 00 a3 cb e6 05  |........K.w.....|
00000220  6b 74 5c fa 9a 6b 5c f6  00 b7 68 71 9d 3f b7 ac  |kt\..k\...hq.?..|
00000230  e6 a7 06 e4 3a ee 79 f2  b7 3d f7 66 e0 7c e7 23  |....:.y..=.f.|.#|
00000240  e7 9e ac 27 d7 e0 6b 87  e6 e3 f3 9c 4f 8d 07 94  |...'..k.....O...|
00000250  27 f9 a6 1a 3e f3 62 1c  31 a3 67 16 df 79 e3 c0  |'...>.b.1.g..y..|
00000260  28 ff 14 97 d5 53 d4 07  3e db 27 e2 ba 3e 69 07  |(....S..>.'..>i.|
00000270  5e ec 13 f5 25 2e 9f 02  fb 7c 31 ae d7 ec 43 0f  |^...%....|1...C.|
00000280  fa 40 1f eb 18 01 e6 32  78 b8 d6 b1 21 73 66 11  |.@.....2x...!sf.|
00000290  39 9d 88 e3 f7 2e b9 14  60 e7 cc bb 51 c2 fa 0f  |9.......`...Q...|
000002a0  f6 d5 de a9 41 ac b2 6b  4c 1e e3 7f 85 8f f7 6c  |....A..kL......l|
000002b0  8c 5c 17 b8 53 9b 56 a6  4f b4 22 86 5c 87 c1 61  |.\..S.V.O.".\..a|
000002c0  c6 e2 11 fb e3 3b fa 00  be 21 c4 a4 a6 e7 c7 63  |.....;...!.....c|
000002d0  fc 66 e0 91 1f f1 22 d0  b9 d8 1f 97 0e c4 9f 41  |.f...."........A|

Now to convert this to something usable...

The first line is:

00000000  b4 6e 68 44 ae 2e 9e f2  01 00 00 00 66 63 66 60  |.nhD........fcf`|

Which looks like:
b4 6e 68 44 = signature
ae 2e 9e f2 = file size backwards (0xF29E2EAE = 4GB)
01 00 00 00 = file number for split-images or total number of files in archive
66 63 66 60 60 70 66 b0 62 88 61 60 04 = meta data ?
b2 18 00 4f f6 0e eb 01 13 = ?
63 60 60 00 = more meta data
00 5f d7 36 54 05 93 fe e6 c8 = start of raw image data ?