Wanted to know a little more about the pros and cons of different compression tools and strategies.
Found this article A Quick Benchmark: Gzip vs. Bzip2 vs. LZMA and looking at that gzip is a pretty clear winner I think wrt my concerns (i.e. fast compression over high compression). Also found gzip vs. bzip2 which concluded basically the same.
I want to post-process the output of an ‘ls’ command with ‘sed’ so that I can remove the ‘./’ prefixes that I can’t avoid going into the ls output (this is a result of using ‘find’ safely).
The thing is, if I pipe ls output to sed, then the default –color=auto setting applies and ls detects that it’s not talking to a terminal so doesn’t output colour codes. But I want colour codes, usually, so I need to change the ls command to use –colour=always, which I’ve done. This means I can have colour and also have sed format the ls output.
The problem is then what happens if I want to pipe my output to ‘less’? Then the colour code commands appear as garbage in the output stream. So, usually I want colour codes, and sometimes I don’t.
I found this article, Remove color codes (special characters) with sed, which helped me come up with the following bash alias:
alias noco='sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"'
So now that I have the ‘noco’ alias (short for “no colour”) I can pipe my output through that if I want the colour codes removed, which I can apply before piping output to less.
It’s a little bit annoying that I have to do things this way but I haven’t been able to think of a better way to make it all work and this all seems to get the job done.
I’ve been through this process before (for older versions of MediaWiki than the 1.18 version I’m using at the moment, and things have changed) but yesterday I found myself following instructions for enabling support for mathematical equations in MediaWiki. So that’s up-to-date on my Morpheus (private) and Sixsigma (public at jj5.net) wikis (which are based on my jjwiki project). At the moment I have four wiki projects: bkwiki and bkdevwiki for Blackbrick, pcwiki for ProgClub and jjwiki for jj5.net. I also have another wiki project that’s not presently under version control called BBS for Blackbrick. I think I’ll roll the bkwiki, bkdevwiki and BBS projects into a single project (bkwiki), and the pcwiki and jjwiki projects into a single project (jjwiki) so that I only have two MediaWiki projects to maintain (for now). But, as usual, making things simpler demands more work… not sure when I’ll get around to that.
Today I learned about the SVNIndexXSLT Apache directive that lets you configure an XSLT stylesheet for your Subversion repository. Also found a whole web-site ReposStyle which makes me laugh — seems like overkill having a whole web-site for a repository stylesheet…
The SQL History feature of my phpMyAdmin setup wasn’t working, the history table had been configured correctly as ‘pma_history’ but there was no data being written into that table when I ran queries and the SQL History tab of the query window just showed a blank.
Eventually I figured out that in order for the pma_history table to be populated I needed to set the QueryHistoryDB setting to true. So I did that and now SQL History is working!
I think I’ve probably done this before (the links in my browser were marked as visited), but today I wanted to create a desktop/toolbar shortcut icon (with shortcut key) to a saved PuTTY session called “peace tunnel”. The “peace tunnel” opens an SSH session to a development server called “peace” and automatically configures a tunnel from port 80 on the localhost to port 80 on the server, so I can check on the progress of a web application under development.
Anyway, I found the documentation for Starting a session from the command line and basically to load my saved session called “peace tunnel” I had to run the command:
putty.exe -load "peace tunnel"
Too easy.
I was configuring MediaWiki to allow uploads and was getting an error in the browser about the POST data being too large (“does not allow request data with POST requests, or the amount of data provided in the request exceeds the capacity limit.”). I had a look in the Apache error log and found:
[Thu Feb 23 16:12:45 2012] [error] [client 60.240.67.126] request body exceeds m aximum size (131072) for SSL buffer, referer: https://www.jj5.net/morpheus/Speci al:Upload [Thu Feb 23 16:12:45 2012] [error] [client 60.240.67.126] could not buffer messa ge body to allow SSL renegotiation to proceed, referer: https://www.jj5.net/morp heus/Special:Upload
So I did some research. I found this document, File upload size which suggested editing /etc/php5/apache2/php.ini which I did:
upload_max_filesize = 20M post_max_size = 80M
That didn’t fix the problem though. I found Request entity too large which suggested checking my setting for LimitRequestBody, but that wasn’t the problem either.
Eventually I found Bug 491763 – HTTPS+SSLVerifyClient require in <Directory>+big POST = Apache error which suggested I needed to apply the SSLRenegBufferSize directive which I did like this:
<Location /morpheus>
SSLVerifyClient require
SSLVerifyDepth 1
SSLRenegBufferSize 20971520
</Location>
And then after restarting Apache the problem was solved.
I found the documentation for Apache Module mod_rewrite. There’s also a section on RewriteRule Flags.
I had a problem with phpMyAdmin not using encrypted connections.
My server was correctly configured for SSL as indicated by:
SHOW VARIABLES LIKE '%ssl%'
Which returned:
| Variable_name | Value |
|---|---|
| have_openssl | YES |
| have_ssl | YES |
| ssl_ca | /etc/mysql/cacert.pem |
| ssl_capath | |
| ssl_cert | /etc/mysql/server-cert.pem |
| ssl_cipher | |
| ssl_key | /etc/mysql/server-key.pem |
However when I ran:
SHOW STATUS LIKE 'Ssl_cipher'
I got back a null result, indicating that the connection was not encrypted.
Eventually I figured out that the problem was caused by using the ‘mysqli’ provider for my connections in phpMyAdmin. When I switched my connections to use ‘mysql’ instead then encryption started working and an Ssl_cipher was reported.
I’d love to know what the actual problem is, but for now I’m just happy that my connections are actually encrypted. I spent a while hacking on the mysqli dbi interface to try and get it to play nice with SSL but I didn’t make any progress.
Found some discussion on dealing with “fail2ban.actions.action: ERROR” errors from fail2ban. Basically there’s a race condition and a few suggestions to deal with it. One is to modify /usr/bin/fail2ban-client like this:
def __processCmd(self, cmd, showRet = True): beautifier = Beautifier() for c in cmd: time.sleep(0.1) beautifier.setInputCmd(c)
But the other one, that I think I like better, is to edit /etc/fail2ban/actions.d/iptables-multiport.conf to include a call to sleep for a random time up to three seconds:
actionstart = sleep `perl -e 'print rand(3);'`
iptables -N fail2ban-<name>
iptables -A fail2ban-<name> -j RETURN
iptables -I INPUT -p <protocol> -m multiport --dports <port> -j fail2ban-<name>