XFCE 4.8 panel : Fix missing Suspend and Hibernate icons

The “Action Buttons” panel plugin of XFCE 4.8 on Debian “wheezy” has a known bug regarding missing icons for the Suspend and Hibernate actions and users will only see a generic placeholder icon.

Ognyan Kulev who reported the bug says this is because most icon themes do not provide icons named “system-suspend” and “system-hibernate”.

As a work-around he suggests linking to the corresponding xfce4-power-manager icons. This fixed the issue on my Debian laptop:

sudo apt-get install xfce4-power-manager-data
cd /usr/share/icons/hicolor/scalable/actions
sudo ln -s xfpm-suspend.svg system-suspend.svg
sudo ln -s xfpm-hibernate.svg system-hibernate.svg
sudo gtk-update-icon-cache-3.0 -f ../..

I saved save the commands into a shell script so I can run it again if the symbolic links get deleted during an apt-get update or accidental apt-get remove:

echo "#! /bin/sh" > /usr/local/bin/fix-xfce-action-icons.sh
chmod ugo+x /usr/local/bin/fix-xfce-action-icons.sh
sudo vim /usr/local/bin/fix-xfce-action-icons.sh

Using libgdx for cross-platform app development

I am looking for a framework that allows me to develop modern apps (mobile, web, desktop) all from one Java codebase. I prefer Java because I know it very well, it is already cross-platform and a statically typed language that allows IntelliJ, Eclipse and Netbeans to be better than any dynamically typed scripting language editor could ever be.

Currently my favorite is libgdx. I am planning to use it with IntelliJ Community Edition and with Maven.

By using RoboVM, libgdx even supports iOS.

For user input (forms) libgdx provides the scene2d.ui widgets. I hope that will be sufficient for most of my UIs. Now I just have to get OpenGL to work on my Linux box …

Java map comparison with detailed error messages using Guava

When you compare two Java maps that are supposed to be equal, i.e. contain the same name/value pairs, you might want to give some details about potential mismatches, for example in your log output.

The Guava library from Google provides a convenient tool for that, namely the class com.google.common.collect.MapDifference.

In the sample code below I have implemented a simple utitily method that compares two maps and logs detailed error messages if they are not equal.

This code is also the first item in my guava-based github project.

Maven dependencies

Put this into pom.xml :

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>16.0</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.5</version>
</dependency>

You will also need an slf4j implementation, like logback. For testing, we can use this :

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>1.7.5</version>
</dependency>

And for the JUnit tests further below you will need this :

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.10</version>
    <scope>test</scope>
</dependency>

Java class

Save this as src/main/java/net/doepner/util/MapDiffUtil.java :

package net.doepner.util;

import java.util.Map;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.google.common.collect.MapDifference;
import com.google.common.collect.Maps;

import static com.google.common.collect.MapDifference.ValueDifference;

/**
 * Map comparison with detailed log messages
 */
public class MapDiffUtil {

    private static final Logger log =
        LoggerFactory.getLogger(MapDiffUtil.class);

    public static <K, V> boolean validateEqual(
               Map<K, V> map1, Map<K, V> map2,
               String map1Name, String map2Name) {

        final MapDifference<K, V> diff = Maps.difference(map1, map2);

        if (diff.areEqual()) {
            log.info("Maps '{}' and '{}' contain exactly the same "
                   + "name/value pairs", map1Name, map2Name);
            return true;

        } else {
            logKeys(diff.entriesOnlyOnLeft(), map1Name, map2Name);
            logKeys(diff.entriesOnlyOnRight(), map2Name, map1Name);
            logEntries(diff.entriesDiffering(), map1Name, map2Name);
            return false;
        }
    }

    private static <K, V> void logKeys(
                Map<K, V> mapSubset, String n1, String n2) {
        if (not(mapSubset.isEmpty())) {
            log.error("Keys found in {} but not in {}: {}",
                n1, n2, mapSubset.keySet());
        }
    }

    private static <K, V> void logEntries(
                Map<K, ValueDifference<V>> differing, 
                String n1, String n2) {
        if (not(differing.isEmpty())) {
            log.error("Differing values found {key={}-value,{}-value}: {}",
                        n1, n2, differing);
        }
    }

    private static boolean not(boolean b) {
        return !b;
    }
}

Unit tests

Save this as src/main/java/net/doepner/util/MapDiffUtilTest.java :

package net.doepner.util;

import java.util.HashMap;
import java.util.Map;

import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;

/**
 * Tests MapDiffUtil
 */
public class MapDiffUtilTest {

    private final Logger log = LoggerFactory.getLogger(getClass());

    @Rule
    public TestName testName = new TestName();

    @Before
    public void logTestName() {
        log.info("Executing {}", testName.getMethodName());
    }

    @Test
    public void testEqual() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);
        map1.put("B", 2);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 2);
        map2.put("A", 1);

        assertTrue("Maps should be equal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));
    }

    @Test
    public void testSubset() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 2);
        map2.put("A", 1);

        assertFalse("Maps should be unequal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));

    }

    @Test
    public void testSeparate() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 2);

        assertFalse("Maps should be unequal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));
    }

    @Test
    public void testMismatches() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);
        map1.put("B", 2);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 20);
        map2.put("C", 3);

        assertFalse("Maps should be unequal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));

    }
}

Append entries to Windows user PATH from script or command line

If you have to work in a Windows environment where you do not want to or cannot change the system level PATH, you can usually still use the “setx” command to change the user level PATH variable.

But what if you want to add (append) a directory to the user level PATH? The following would set your user PATH variable to the full (system + user) PATH plus the appended directory.

setx PATH "%PATH%;c:\whatever\else"

But this would duplicate the system PATH into your user PATH variable, which would ultimately cause duplicate entries in the full PATH.

Instead we want to append only to the user PATH variable. Unfortunately, to get its value from the Windows command-line you must query the “HKCU\Environment” registry entry and then parse the relevant part from the output string.

I wrote the add-to-user-path.bat script below to fix the problem. It has been tested on Windows 7, where “req query” uses 4 spaces to delimit its output, and it works even when the user PATH value already contains spaces:

@echo off 
setlocal EnableDelayedExpansion 

if %1.==. ( 
  echo Usage: %0 directory 
  goto End 
) 

for /f "skip=2 tokens=*" %%A in ('reg query "HKCU\Environment" /v PATH') do ( 
   set regstr=%%A
    
   for /f "tokens=3 delims=|" %%X in ("!regstr:    =^|!") do ( 
     setx PATH "%%X;%1" 
     echo PATH changes will only take effect for newly started processes. 
   ) 
) 

:End 

Organize image and video files by creation date with exiftool

I use a simple bash script /usr/local/bin/move-cam-files.sh to let exiftool move photos and videos from our camera to directories on our file server, based on creation date as per media file metadata:

#! /bin/bash -x

move() {
  exiftool -r -v -ext $2 '-directory<DateTimeOriginal' \
           -d "/home/storage/$3/%Y/%m" "$1"
}

# directory where to read media files from is $1
# if unspecified use current directory
dir=${1:-.}

move $dir AVI videos
move $dir JPG photos

Install and maintain Cygwin without Windows admin rights

Sometimes you work on Windows as a restricted user, without admin rights. Like many other good software packages, Cygwin can be installed anyway (see the respective FAQ entry).

These are the steps I used:

  1. Download setup-x86.exe (32bit) or setup-x86_64.exe (64bit).
  2. Run it with the “--no-admin” option
  3. During installation select the “wget” package
  4. Create /usr/local/bin/cygwin-setup.sh (for 32bit omit the “_64”):
    #! /bin/sh
    rm setup-x86_64.exe
    wget http://cygwin.com/setup-x86_64.exe
    chmod u+x setup-x86_64.exe
    run ./setup-x86_64.exe --no-admin
    
  5. Make the script executable:
    chmod ugo+x /usr/local/bin/cygwin-setup.sh
  6. Create a copy of the Cygwin terminal shortcut, rename it “Cygwin setup”
  7. Edit the shortcut target, replace
    mintty.exe -i /Cygwin-Terminal.ico -

    with

    mintty.exe -i /Cygwin-Terminal.ico /bin/bash -l -c 'cygwin-setup.sh'

Whenever you want to run the Cygwin installer to install or remove packages, you can just execute the shortcut or run cygwin-setup.sh from the Cygwin command prompt.

Alternatively, you could also use the pure command-line tool apt-cyg.

Use standard bash for ssh login on shared hosts

Please note: The following instructions are for ssh logins on a remote host. The approach is not suitable for executing remote commands via ssh.

Problem

When you work on a remote Linux or Unix server (via ssh) you sometimes cannot control your login shell and/or its default config file. For example, you might be sharing the same user account on the server with other people or the use of the chsh tool might be locked down.

Maybe the default shell is something like ksh, or if bash is used maybe the .bashrc sets vi key bindings. This can be annoying if you are used to standard Linux bash with its default Emacs style bindings.

Suggested solution

In these cases you can do the following, assuming bash is installed and in the path on the host:

1) Create /usr/local/bin/sbash.sh (on Windows, use Cygwin).:

#! /bin/bash

 set -x

 if [ "$#" -gt 0 ]; then
   user_at_host="$1"
   shift
   ssh_options="$@"
 else
   set +x
   echo Usage: $(basename "$0") user@host [ssh-options]
   exit 1
 fi

 ssh -t $ssh_options \
        $user_at_host \
     "bash --rcfile .bashrc.for-remote-user-${USER}"

Make the file executable using something like chmod ugo+x /usr/local/bin/sbash.sh. You can then use it for remote logins like the ssh command, for example:

 
oliver@basement:~$ sbash.sh user@host

The $USER variable in the sbash.sh script will be substituted by the local shell with your local user name, which in this example is “oliver”.

2) On the host create ~/.bashrc.for_remote_user_USERNAME where USERNAME is the user name from the ssh client as mentioned above:

user@host$ vim $HOME/.bashrc.for-remote-user-oliver

Make sure this file name matches the –rcfile option in your ssh command.

You can then edit and use this file like a normal .bashrc file, i.e. for setting your favorite environment variables, bash options, aliases, etc.

Minimal Debian VM upgraded to wheezy / Jenkins with OpenJDK 7

I have upgraded my minimal Debian VM to current Debian stable (“wheezy”). It comes in OVA format which is deployable to Virtualbox or VMware.

The Debian system JDK, i.e. the location of java, javac, etc. commands in the PATH, is still OpenJDK 6 because that is the default-jdk on wheezy and the Jenkins deb packages from jenkins-ci.org depend on it. In particular this means that the Tomcat / Jenkins process itself is executed by OpenJDK 6.

But my Jenkins installer now also installs OpenJDK 7 and pre-configures it as the default JDK for Jenkins jobs. That means you can now use this Jenkins instance to build your Java 7 projects, as well as older Java projects.

Please follow the step-by-step installation instructions if you want to use the VM. It consists completely of Free / Open Source software. I provide it for download “as is” without any warranty of any kind.

Lazy logging of Java collections content

Sometimes you might want to include Java collections content in a log message. But standard Java collections do not provide a useful toString() implementation. So developers often use a helper method like this:

    /** 
     * @param collection A collection 
     * @return A string listing all collection items 
     */ 
    public static String toContentString(Collection<?> collection) { 
        return Arrays.toString(collection.toArray()); 
    } 

Then in typical slf4j log output code you might have something like this:

log.debug("Processing list: {}", toContentString(list));

But if list is long, then this code will cause unnecessary overhead in your application whenever it runs at a log level that does not actually log debug messages.

A simple solution for this is to use a wrapper object instead:

log.debug("Processing list: {}", toStringWrapper(list));

A simple toStringWrapper(..) implementation could be this:

/** 
     * Provides a wrapper object that can be used for logging of 
     * collection content 
     * 
     * @param collection A collection 
     * @return A wrapper toString() implementation listing all 
     *         collection items 
     */ 
    public static Object toStringWrapper(Collection<?> collection) { 
        return new Object() { 
            @Override 
            public String toString() { 
                return toContentString(collection); 
            } 
        }; 
    } 

This postpones the String generation until slf4j actually calls toString() on the log message parameter to substitute it for the ‘{}’ placeholder. And this substitution only happens if the log level actually requires it, i.e. less cpu cycles wasted on unused log message parameters.

Nightly file server backups to external harddrive

I use a small headless Debian system as file server for all family photos, videos, documents, etc. Its hostname is “bubba”. I have recently set it up to run backups to an external harddrive, using cron and rsync.

The external disk is a 500G laptop SATA disk in an USB/eSATA enclosure. It requires no separate power supply. So far I have only got it to work over USB. Somehow the eSATA does not work for me on Debian 6 (aka “squeeze”), even though the file server has an eSATA port.

Prerequisites


sudo mkdir /mnt/backup
sudo apt-get install ntfs-3g

NTFS mount/unmount with sudo

I use NTFS as the filesystem on the backup disk because we wanted it to be compatible with MS Windows. The Debian Linux on the file server uses ntfs-3g for mounting the disk read-write. Unfortunately that only works well with root rights, so I configured sudo to permit myself password-less mounting and unmounting of the device.

/etc/sudoers entry
oliver ALL = NOPASSWD: /bin/mount /mnt/backup, \
                       /bin/umount /mnt/backup

Nightly rsync

The nightly backup process itself is a simple non-destructive local rsync command, wrapped by mount and unmount commands, to make sure that we can unplug the external disk anytime we want (just not around midnight).

My crontab:


oliver@bubba:~$ crontab -l
0 0 * * * /home/oliver/shared/scripts/backup.sh

The backup.sh script
#! /bin/sh

if mountpoint /mnt/backup; then
  sudo umount /mnt/backup
fi

sudo mount /mnt/backup

if [ $? -eq 0 ]; then

  rsync -avvih --progress \
    --exclude /downloads \
    --exclude /movies \
  /home/storage/ /home/oliver/backup \
  > /tmp/cron_output.log 2>&1

fi

sudo umount /mnt/backup

Symlinks and fstab

Symlink in my home for convenience:

oliver@bubba:~$ ls -l /home/oliver/backup
lrwxrwxrwx 1 oliver users 11 Aug 7 21:38 /home/oliver/backup -> /mnt/backup/

Entry in /etc/fstab:

oliver@bubba:~$ grep "/mnt/backup" /etc/fstab
/usr/local/share/backup /mnt/backup ntfs-3g defaults 0 0

Device symlink /usr/local/share/backup:

oliver@bubba:~$ ls -l /usr/local/share/backup
lrwxrwxrwx 1 root staff 84 Jun 23 01:51 /usr/local/share/backup -> /dev/disk/by-id/usb-WDC_WD50_00BPVT-00HXZT3_FDC0FD500000000FD0FF61A6103926-0:0-part1

Try it manually


/home/oliver/shared/scripts/backup.sh &
less /tmp/cron_output.log

Room for improvement

The symlink to the device file is the ugliest part of the whole solution. Currently I have to plug the disk directly into a USB slot on the file server because if I connect it via a USB hub, it will appear under a different name in /dev/disk/by-id and my symlink won’t work. I would like to use a udev rule instead that automatically creates an identical symlink no matter how the the disk is plugged in.

I would also like to implement a 2-way backup so that files we put on the external disk, for example photos from a trip to relatives, will be mirrored to the file server. It should be just another rsync command going in the opposite reaction.

Maybe I would also like the backup process to start right away when the disk is plugged in, in addition to the nightly cron job. This would probably require another udev rule.