This on HN today: Data Structure Visualizations.
Tag Archives: data
Data Visualization With Matplotlib and Seaborn
I was referred to Data Visualization With Matplotlib and Seaborn on #lobsters today.
A great MySQL session
I found this great MySQL session over here. It has a clever approach for creating a big table, and shows how to invoke shell commands from the `mysql` client.
USE test; SET GLOBAL innodb_file_per_table=1; SET GLOBAL autocommit=0; -- Create an uncompressed table with a million or two rows. CREATE TABLE big_table AS SELECT * FROM information_schema.columns; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; INSERT INTO big_table SELECT * FROM big_table; COMMIT; ALTER TABLE big_table ADD id int unsigned NOT NULL PRIMARY KEY auto_increment; SHOW CREATE TABLE big_table\G select count(id) from big_table; -- Check how much space is needed for the uncompressed table. \! ls -l data/test/big_table.ibd CREATE TABLE key_block_size_4 LIKE big_table; ALTER TABLE key_block_size_4 key_block_size=4 row_format=compressed; INSERT INTO key_block_size_4 SELECT * FROM big_table; commit; -- Check how much space is needed for a compressed table -- with particular compression settings. \! ls -l data/test/key_block_size_4.ibd
8GB data reservation
In the news today: Why All My Servers Have an 8GB Empty File. The idea is to created a /spacer.img file to take up space, so in a crisis you can delete it to free up space.
MySQL admin with Ian Gilfillan
I was doing some reading today and I came upon an old series of articles over on www.databasejournal.com by a dude called Ian Gilfillan:
- MySQL backups
- Database Replication in MySQL
- Restoring lost data from the Binary Update Log
- Optimizing MySQL: Queries and Indexes
- Optimizing MySQL: Hardware and the Mysqld Variables
- MySQL’s Query Cache
- MySQL’s Over-looked and Under-worked Slow Query Log
For the ‘type’ in SQL ‘EXPLAIN’ Ian says: from best to worst the types are: system, const, eq_ref, ref, range, index, all.
It seems this Ian Gilfillan fellow has been rather prolific.
Charts.css
Kickass: Charts.css. Also of interest: Which color scale to use when visualizing data.
Don’t Compare Averages
An article Don’t Compare Averages from Martin Fowler has lots of good info about data visualisation techniques.
Backup Thunderbird data
I wanted a backup of my Thunderbird data on my Debian desktop, and I created it like this:
jj5@tact:~$ tar -c -f jj5-.thunderbird.tgz --use-compress-program='pigz -p 7 --best' .thunderbird
Error in rsync protocol data stream
So I was running a backup with rsync and I saw this:
lib/mysql/ibdata1 437.40M 33% 4.53MB/s 0:03:10 inflate returned -3 (0 bytes) rsync error: error in rsync protocol data stream (code 12) at token.c(557) [receiver=3.1.2] rsync: connection unexpectedly closed (155602 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(235) [generator=3.1.2]
The issue seems to be that if you’re using rsync compression and the remote file gets changed while the rsync copy is in progress then shit gets corrupted. My solution was to handle error level ’12’ and retry without compression. If the file changes while the rsync is in progress the file will be corrupt, so you shouldn’t rely on the integrity of such files.
Loading MySQL timezone data
Turns out MySQL doesn’t load timezone info by default! As you can read about here.
To load MySQL timezone info:
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql