Tag Archives: programming

I love being a programmer

Posted on by

My ZFS RAID array is resilvering. It’s a long recovery process. A report on progress looks like this:

Every 10.0s: zpool status                                             love: Tue May  4 22:32:27 2021

  pool: data
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sun May  2 20:26:52 2021
        1.89T scanned out of 5.03T at 11.0M/s, 83h19m to go
        967G resilvered, 37.54% done
config:

        NAME                       STATE     READ WRITE CKSUM
        data                       DEGRADED     0     0     0
          mirror-0                 ONLINE       0     0     0
            sda                    ONLINE       0     0     0
            sdb                    ONLINE       0     0     0
          mirror-1                 DEGRADED     0     0     0
            replacing-0            DEGRADED     0     0     0
              4616223910663615641  UNAVAIL      0     0     0  was /dev/sdc1/old
              sdc                  ONLINE       0     0     0  (resilvering)
            sdd                    ONLINE       0     0     0
        cache
          nvme0n1p4                ONLINE       0     0     0

errors: No known data errors

So that 83h19m to go wasn’t in units I could easily grok, what I wanted to know was how many days. Lucky for me, I’m a computer programmer!

First I wrote watch-zpool-status.php:

#!/usr/bin/env php
<?php

main( $argv );

function main( $argv ) {

  $status = fread( STDIN, 999999 );

  if ( ! preg_match( '/, (\d+)h(\d+)m to go/', $status, $matches ) ) {

    die( "could not read zpool status.\n" );

  }

  $hours = intval( $matches[ 1 ] );
  $minutes = intval( $matches[ 2 ] );

  $minutes += $hours * 60;

  $days = $minutes / 60.0 / 24.0;

  $report = number_format( $days, 2 );

  echo "days remaining: $report\n";

}

And then I wrote watch-zpool-status.sh to run it:

#!/bin/bash

watch -n 10 'zpool status | ./watch-zpool-status.php'

So now it reports that there are 3.47 days remaining, good to know!

A great MySQL session

Posted on by

I found this great MySQL session over here. It has a clever approach for creating a big table, and shows how to invoke shell commands from the `mysql` client.

USE test;
SET GLOBAL innodb_file_per_table=1;
SET GLOBAL autocommit=0;

-- Create an uncompressed table with a million or two rows.
CREATE TABLE big_table AS SELECT * FROM information_schema.columns;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
INSERT INTO big_table SELECT * FROM big_table;
COMMIT;
ALTER TABLE big_table ADD id int unsigned NOT NULL PRIMARY KEY auto_increment;

SHOW CREATE TABLE big_table\G

select count(id) from big_table;

-- Check how much space is needed for the uncompressed table.
\! ls -l data/test/big_table.ibd

CREATE TABLE key_block_size_4 LIKE big_table;
ALTER TABLE key_block_size_4 key_block_size=4 row_format=compressed;

INSERT INTO key_block_size_4 SELECT * FROM big_table;
commit;

-- Check how much space is needed for a compressed table
-- with particular compression settings.
\! ls -l data/test/key_block_size_4.ibd

Programming principles from id software

Posted on by

These are great: John Romero on Programming principles from id software

  1. Just do it (and do it well)
  2. Keep your code always runnable
  3. Keep it simple
  4. Invest time in building great tools
  5. Test your code thoroughly
  6. Fix bugs as soon as possible
  7. Use a superior development system
  8. Write code for this version of the product
  9. Use good component abstractions
  10. Seek feedback from peers while coding
  11. Give coders creative freedom

Peter Norvig: What to demand from a Scientific Computing Language

Posted on by
Reply

Doing some research on Peter Norvig (I’m fascinated by the guy and want to know what he thinks) and I found a talk of his: Peter Norvig: What to demand from a Scientific Computing Language. In the talk Peter explains what he wants out of a programming language and why he feels that Python fits the bill.

I watched the whole thing but I think I’ll put it on my TODO list to watch this again one day.

Interviewing at Google

Posted on by
Reply

I’m sick and tired of seeing Google’s bullshit “this stuff matters” advertising as part of their latest privacy policy update (which I’ve been doing my best to ignore).

As I was complaining about it I thought I’d try and find some examples of their hypocrisy. I’m fairly sure I’ve seen articles before where people said their job for a Google interview was to process gigabytes of web logs. If you’re data-mining web logs to *spy* on people on the one hand and then telling people you’re protecting their privacy on the other hand then you’re a lying sack of shit really, aren’t you?

Anyway, I didn’t find what I was looking for in a big hurry and I don’t really have time for this, so I’m giving up on my little fact finding mission and just going back to ignoring the whole thing.

However, during my web search I found this article, My Job Interview at Google, which seems like a fairly content rich article. I’m particularly interested in the resources that the guy linked to in his post. So figured I’d swing by my blog and make a note so that when I have some free time (hey, could happen) I can go over that post and read the reference material.