Month: December 2013

Minimal Debian VM upgraded to wheezy / Jenkins with OpenJDK 7

I have upgraded my minimal Debian VM to current Debian stable (“wheezy”). It comes in OVA format which is deployable to Virtualbox or VMware.

The Debian system JDK, i.e. the location of java, javac, etc. commands in the PATH, is still OpenJDK 6 because that is the default-jdk on wheezy and the Jenkins deb packages from jenkins-ci.org depend on it. In particular this means that the Tomcat / Jenkins process itself is executed by OpenJDK 6.

But my Jenkins installer now also installs OpenJDK 7 and pre-configures it as the default JDK for Jenkins jobs. That means you can now use this Jenkins instance to build your Java 7 projects, as well as older Java projects.

Please follow the step-by-step installation instructions if you want to use the VM. It consists completely of Free / Open Source software. I provide it for download “as is” without any warranty of any kind.

How I manage my website logins (using Firefox)

This blog post describes how I manage my many website logins (usernames, passwords). I am very interested in how I can improve my current approach. I want it to be highly secure and highly convenient. Please feel free to add your comments and suggestions!

I try to use secure passwords for all of my web accounts, i.e. long random combinations of “special characters”, mixed case letters and numbers. They are usually so secure that I definitely cannot remember them.

So I let my web browser remember, manage and auto-fill my website logins. I use

The approach works well and I never have to memorize my passwords. But my worries are:

  • Is Firefox Sync data as secure as Mozilla claims it is?
  • What can happen if malicious hackers gain access to the Firefox Sync servers?
  • Is the Triple-DES encryption with cipher block chaining that is allegedly used for local password storage in the Firefox profile secure enough (especially given this long-standing bug)?
  • Is it a bad idea to let Firefox even remember my online banking, Paypal and other sensitive passwords?

Lazy logging of Java collections content

Sometimes you might want to include Java collections content in a log message. But standard Java collections do not provide a useful toString() implementation. So developers often use a helper method like this:

    /** 
     * @param collection A collection 
     * @return A string listing all collection items 
     */ 
    public static String toContentString(Collection<?> collection) { 
        return Arrays.toString(collection.toArray()); 
    }

Then in typical slf4j log output code you might have something like this:

log.debug("Processing list: {}", toContentString(list));

But if list is long, then this code will cause unnecessary overhead in your application whenever it runs at a log level that does not actually log debug messages.

A simple solution for this is to use a wrapper object instead:

log.debug("Processing list: {}", toStringWrapper(list));

A simple toStringWrapper(..) implementation could be this:

/** 
     * Provides a wrapper object that can be used for logging of 
     * collection content 
     * 
     * @param collection A collection 
     * @return A wrapper toString() implementation listing all 
     *         collection items 
     */ 
    public static Object toStringWrapper(Collection<?> collection) { 
        return new Object() { 
            @Override 
            public String toString() { 
                return toContentString(collection); 
            } 
        }; 
    }

This postpones the String generation until slf4j actually calls toString() on the log message parameter to substitute it for the ‘{}’ placeholder. And this substitution only happens if the log level actually requires it, i.e. less cpu cycles wasted on unused log message parameters.

Policies for Open Source use at the workplace

Things I like about some companies’ approach to software installation on employee’s machines:

  • No mandatory and restrictive centralized software distribution system where all employees have to hope and pray that certain tools are available and up-to-date.
  • No URL blocking or similar measures of distrust that would prevent employees from downloading and installing software.

This leaves room for a focus on education, knowledge sharing and trust that employees act responsibly.

Based on my experience, these are some useful policies / objectives for Open Source use at the workplace (especially for IT companies):

  • All employees who work with a computer learn the basics about Open Source licenses (OSI definition, copyleft vs permissive, see link below).
  • All employees who work with a computer learn about the differences of Open Source vs Freeware vs Shareware.
  • The company establishes simple and employee-friendly policies for the use of Open Source at work:
    • Declare the major permissive (non-copyleft) licenses (Apache, MIT, BSD, LGPL, EPL, …) as pre-approved for all software library and tool use.
    • Declare all OSI approved licenses as pre-approved for stand-alone tools, i.e. where copyleft cannot affect any derived or bundled code developed at the workplace.
    • Software that satisfies the pre-approval criteria above should not require any further request or approval process.
  • Additionally, the company could maintain a “blacklist” of software (versions) that are known to have security flaws or other aspects that make them unsuitable for use at the workplace.
  • Recommendations can be given to watch out for and uncheck unwanted add-ons during software installations.
  • Software development companies, should train their team leads in basic legal aspects like software copyright and license terms.

Nightly file server backups to external harddrive

I use a small headless Debian system as file server for all family photos, videos, documents, etc. Its hostname is “bubba”. I have recently set it up to run backups to an external harddrive, using cron and rsync.

The external disk is a 500G laptop SATA disk in an USB/eSATA enclosure. It requires no separate power supply. So far I have only got it to work over USB. Somehow the eSATA does not work for me on Debian 6 (aka “squeeze”), even though the file server has an eSATA port.

Prerequisites


sudo mkdir /mnt/backup
sudo apt-get install ntfs-3g

NTFS mount/unmount with sudo

I use NTFS as the filesystem on the backup disk because we wanted it to be compatible with MS Windows. The Debian Linux on the file server uses ntfs-3g for mounting the disk read-write. Unfortunately that only works well with root rights, so I configured sudo to permit myself password-less mounting and unmounting of the device.

/etc/sudoers entry
oliver ALL = NOPASSWD: /bin/mount /mnt/backup, \
                       /bin/umount /mnt/backup

Nightly rsync

The nightly backup process itself is a simple non-destructive local rsync command, wrapped by mount and unmount commands, to make sure that we can unplug the external disk anytime we want (just not around midnight).

My crontab:


oliver@bubba:~$ crontab -l
0 0 * * * /home/oliver/shared/scripts/backup.sh

The backup.sh script
#! /bin/sh

if mountpoint /mnt/backup; then
  sudo umount /mnt/backup
fi

sudo mount /mnt/backup

if [ $? -eq 0 ]; then

  rsync -avvih --progress \
    --exclude /downloads \
    --exclude /movies \
  /home/storage/ /home/oliver/backup \
  > /tmp/cron_output.log 2>&1

fi

sudo umount /mnt/backup

Symlinks and fstab

Symlink in my home for convenience:

oliver@bubba:~$ ls -l /home/oliver/backup
lrwxrwxrwx 1 oliver users 11 Aug 7 21:38 /home/oliver/backup -> /mnt/backup/

Entry in /etc/fstab:

oliver@bubba:~$ grep "/mnt/backup" /etc/fstab
/usr/local/share/backup /mnt/backup ntfs-3g defaults 0 0

Device symlink /usr/local/share/backup:

oliver@bubba:~$ ls -l /usr/local/share/backup
lrwxrwxrwx 1 root staff 84 Jun 23 01:51 /usr/local/share/backup -> /dev/disk/by-id/usb-WDC_WD50_00BPVT-00HXZT3_FDC0FD500000000FD0FF61A6103926-0:0-part1

Try it manually


/home/oliver/shared/scripts/backup.sh &
less /tmp/cron_output.log

Room for improvement

The symlink to the device file is the ugliest part of the whole solution. Currently I have to plug the disk directly into a USB slot on the file server because if I connect it via a USB hub, it will appear under a different name in /dev/disk/by-id and my symlink won’t work. I would like to use a udev rule instead that automatically creates an identical symlink no matter how the the disk is plugged in.

I would also like to implement a 2-way backup so that files we put on the external disk, for example photos from a trip to relatives, will be mirrored to the file server. It should be just another rsync command going in the opposite reaction.

Maybe I would also like the backup process to start right away when the disk is plugged in, in addition to the nightly cron job. This would probably require another udev rule.