Author Archives: Jay Jay

About Jay Jay

Hi there. My name is John Elliot V. My friends call me Jay Jay. I talk about technology on my blog at blog.jj5.net and make videos about electronics on my YouTube channel @InTheLabWithJayJay.

Debugging JINJA in Salt Stack

Posted on by

So I was trying to figure out how to report the template source file within the template itself and I found myself wanting to know what was available in the JINJA globals for a Salt Stack JINJA template.

Turns out you use the show_full_context() function, like this:

Context is: {{ show_full_context() }}

You can parse the output to see what top-level items are available. In my case I found the ‘source’ setting for the JINJA template in use.

Hot tip: if you’re using Vim to inspect the show_full_context() output, try using ‘%’ or ‘]}’ to move between matching braces.

Bonus: while I was learning about reporting the JINJA context I discovered that you can call Salt functions from JINJA templates, like this:

# The following two function calls are equivalent.
{{ salt['cmd.run']('whoami') }}
{{ salt.cmd.run('whoami') }}

If Apache2 won’t serve your JavaScript file try…

Posted on by

I had an issue with Apache2 improperly serving a JavaScript file which I seem to have fixed by making sure the file was terminated with a new-line character… this was really hard to diagnose and resolve! The behaviour in Firefox was that the file just didn’t finish to download, whereas the Apache2 logs indicated a 200 result… I think it may have had something to do with automatic compression, which is a dark art that I do not understand (mumbles something about mod deflate…).

rsync errors

Posted on by

I was having an issue with my rsync command:

time rsync --progress --verbose --acls --xattrs --exclude="/lib/mysql/ibdata1" --exclude="*.tmp" --stats --human-readable --recursive --del --force --times --links --hard-links --executability --numeric-ids --owner --group --perms --sparse --compress-level=6 diligence-test:/usr/ /data/temp/2017-11-15-163925/usr/

getting stuck during “receiving incremental file list” then giving up with:

Timeout, server diligence-test not responding.
rsync: connection unexpectedly closed (16384 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [receiver=3.1.2]
rsync: connection unexpectedly closed (32569 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(235) [generator=3.1.2]
Command exited with non-zero status 255

I could see the server-side process hang in select() with:

root@diligence-test:/home/jj5# ps aux | grep rsync
root      5421  0.0  0.1  15636  2660 ?        Ss   18:55   0:00 rsync --server --sender -vlHogtpAXrSe.iLsfxC --numeric-ids . /usr/

root@diligence-test:/home/jj5# strace -p 5421
strace: Process 5421 attached
select(1, [0], [], [0], {42, 979828})   = 0 (Timeout)

Anyway I figured out how to dodge the problem by nominating –delete-before instead of –del, e.g.:

time rsync --progress --verbose --acls --xattrs --exclude="/lib/mysql/ibdata1" --exclude="*.tmp" --stats --human-readable --recursive --delete-before --force --times --links --hard-links --executability --numeric-ids --owner --group --perms --sparse --compress-level=6 diligence-test:/usr/ /data/temp/2017-11-15-163925/usr/

Waiting for VirtualBox guests to close before reboot or shutdown

Posted on by

See this and this. Basically:

vim /etc/init.d/virtualbox
chmod +x /etc/init.d/virtualbox
update-rc.d virtualbox defaults

The /etc/init.d/virtualbox script should look like this:

#!/bin/sh

# 2017-08-07 jj5 - SEE: How make Debian wait for all VirtualBox guests to
# stop during shutdown/reboot?: https://superuser.com/a/929292/615689

### BEGIN INIT INFO
# Provides:          virtualbox_start_and_stop
# Required-Start:    $local_fs $network
# Required-Stop:     $local_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: 
# Description:       Start virtualbox on boot, and shutdown safely on shutdown/reboot.
### END INIT INFO

case "$1" in
  start)
    echo "Starting Virtualbox "

    # Do whatever to start or resume your virtualbox instances.
    # Perhaps look for a txt file someplace with VMs that need to be
    # restarted or resumed... then start 'em.
    ;;
  stop)
    echo "Stopping Virtualbox"

    # Do something to either shutdown or savestate your virtualbox instances.
    # maybe also save the instances that should be resumed into a txt file
    # someplace for the start method above.

    # 2017-08-07 jj5 - SEE:
    # https://askubuntu.com/a/457564

    # 2017-08-07 jj5 - NOTE: You should make sure that ACPI Shutdown actually
    # shuts down the guest...
    # 2017-08-07 jj5 - SEE: Force Ubuntu Desktop to shutdown on power button:
    # https://www.progclub.org/blog/2017/08/07/force-ubuntu-desktop-to-shutdown-on-power-button-keypress-acpi-shutdown/

    while [ -n "$( sudo -u jj5 VBoxManage list runningvms )" ]; do

      sudo -u jj5 VBoxManage list runningvms \
        | sed -r 's/.*\{(.*)\}/\1/' \
        | xargs -L1 -I {} sudo -u jj5 VBoxManage controlvm {} acpipowerbutton;

      sleep 1;

    done;

    ;;
  *)
    echo "Usage: /etc/init.d/virtualbox {start|stop}"
    exit 1
    ;;
esac

exit 0

You may also need to Force Ubuntu Desktop to shutdown on power button keypress…

VirtualBox VBoxManage for autostart

Posted on by

I have a script tact:/etc/vbox/register-autostart.sh which will register a VirtualBox VM for auto-start. It required other config in /etc/vbox the details of which I don’t remember, but I think it’s done now anyway… there’s more information in Starting virtual machines during system boot.

  online "$machine" && {

    report "halting machine '$machine'...";

    ssh "$machine" sudo poweroff;

    report "waiting a moment...";

    sleep 8;

  };

  run VBoxManage modifyvm "$machine" --autostart-enabled on;

  run sudo service vboxautostart-service restart;