Tag Archives: stream

Error in rsync protocol data stream

Posted on by

So I was running a backup with rsync and I saw this:

lib/mysql/ibdata1
        437.40M  33%    4.53MB/s    0:03:10  
inflate returned -3 (0 bytes)
rsync error: error in rsync protocol data stream (code 12) at token.c(557) [receiver=3.1.2]
rsync: connection unexpectedly closed (155602 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [generator=3.1.2]

The issue seems to be that if you’re using rsync compression and the remote file gets changed while the rsync copy is in progress then shit gets corrupted. My solution was to handle error level ’12’ and retry without compression. If the file changes while the rsync is in progress the file will be corrupt, so you shouldn’t rely on the integrity of such files.

PHP file_get_contents with HTTP Basic Auth

Posted on by
Reply

Today I needed to figure out how to read some data from a URL that required HTTP Basic Auth. The solution was pretty simple, use file_get_contents and pass in a configured stream context. I found the following code on the stream_context_create documentation:

$cred = sprintf( 'Authorization: Basic %s',
  base64_encode( 'username:password' )
);
$opts = array(
  'http' => array(
    'method' => 'GET',
    'header' => $cred
  )
);
$ctx = stream_context_create( $opts );
$data = file_get_contents( $url, false, $ctx );

Easy-peasy!

tr

Posted on by
32

I learned about the ‘tr’ Unix command today. It’s for translating text in streams. The particular example was:

  echo | tr '012' '001'

And I didn’t really understand what that did, but now I do. Basically the ‘echo’ part will echo a new line character, which is octal 012. Then tr will read its input stream and read that new line. It then has a rule to translate 012 (new line) to 001 (Ctrl+A), which it does. So basically it’s just a way of getting a Ctrl+A character in a stream. If you use Ctrl+A as your regular expression delimiter you’re unlikely to have a collision in the expression itself.