Undo/Redo in Java using Protostuff serialization and binary diffs

Many applications need Undo/Redo functionality. Commonly used implementation patterns are:

  • Command Pattern
  • Memento Pattern (state snapshots)
  • State diffs

When using the Command Pattern one would encapsulate both the change logic and its reversal in command objects. Undo/Redo is implemented by managing stacks of those objects. This approach has its limitations, for example for changes that are unidirectional in nature, like anything involving randomness, encryption, etc.

State snapshots save the full state of the edited data as object graphs or some representation thereof. This is also called the Memento Pattern. It often uses serialization and typically compression of the object graph to reduce memory use and ensure immutable snapshots that can also be stored out-of-process, if desired.

State diffs are based on the idea of State snapshots, but only store the difference between states. This can vastly reduce memory consumption of your Undo/Redo history. It is based on diffing algorithms that compute the delta between two states (or their memento) and allow Undo/Redo by applying the deltas as patches against a given state. A disadvantage is that jumping to a state involves a whole chain of patch applications. But it is a good approach when the user mainly navigates the Undo/Redo history sequentially.

A highly reusable implementation of Undo/Redo using State Diffs is available at my github account: https://github.com/old-o/diffing-history

It uses the following Open Source libraries:

  • Protostuff for object graph serialization using runtime schema
  • JavaxDelta for binary diffing and patching

It provides the following features:

  • Unlimited Undo and Redo
  • Can handle any type of Java objects
  • Low memory footprint
  • Straightforward type-safe API
  • Supports stack size listeners
  • Gzip compression for the serialized current state

It is Open Source under the Unlicense.

Usage

The main API is the History interface.
Create an instance of DiffingHistory to get started.
The DiffingHistoryTest calls all History methods and illustrates the API.

Continuous delivery using github, travis-ci and bintray

Continuous-Delivery-schema

Let’s say you work on a Java application and want to frequently make it available for download so that user’s can easily try the latest version.

Let’s say you work primarily on your laptop or personal computer using a Java IDE and commit code changes, but you don’t want to spend time manually building jars, packaging war or zip files, testing your application or uploading files to a website, etc.

Instead you want to have a fully automated process that compiles your source code, runs automated tests and other quality control mechanisms, builds your application and uploads the result to a public website.

But you don’t want to install any infrastructure for this and not run anything besides Java and your IDE on your own machine(s).

Basically you want to use developer-friendly reliable cloud services but you don’t want to pay a single cent.

All of this is possible, as long your code is Open Source:

  • Host your source code on github
  • Let travis-ci run vour build process
  • Let travis-ci upload the build result to bintray

For details, you can take a look at one of my github projects.

Relevant config files:

JBoss Undertow is pulling me in … :o)

I am very impressed as I am trying out the various code examples for Undertow, a kick-ass, light-weight yet powerful, ultra-easy-to-embed HTTP and Java Servlet engine.

One of my side projects requires an embeddable yet feature-complete Java HTTP engine with low memory footprint and a simple straightforward API. I dismissed Tomcat, briefly considered Jetty, found Winstone too old and unmaintained, simpleframework not well-enough documented, vert.x and netty a little too much for my purposes and/or too complicated, so that a few weeks ago I had actually started to clone and refactor NanoHttpd.

The NanoHttpd refactoring was a great learning experience, but it certainly felt like I was reinventing the wheel in the form of a cute and mobile but slightly rusty foldable unicycle. ;o) – no offense please, nanohttpd developers

Then I found out about Undertow. The author Stuart Douglas is now officially my hero. What an awesome job he is doing! The server meets all of the above mentioned requirements and is apparently also comparatively fast. No wonder it is the HTTP engine used by Wildfly, the new JBoss AS.

Anyway, if you want to try yourself, I’d go with version 1.1 final at this time, i.e. this in your Maven pom.xml:

<dependency>
    <groupId>io.undertow</groupId>
    <artifactId>undertow-core</artifactId>
    <version>1.1.0.Final</version>
</dependency>

I decided to pretty much ignore the documentation section of the undertow.io website for now, as it is still for version 1.0 and the API has changed – improved, I guess – since then. It seems to me, that at this point the core code itself and the usage examples are the best documentation for version 1.1. Both are Maven modules of the undertow github project.

By the way, if you are wondering why the project has no issues section on github: The issue tracking is done in the JBoss Jira.

Java software engineering – reference resources

Official Java and JEE

Java Technology Reference

Java Standard Edition (JSE)

Java Enterprise Edition (JEE)

The official Java tutorials

The official JEE 7 tutorial

JEE 7 Technologies index

Java language spec and JVM spec

Java community

Oracle Java community

OpenJDK

Java Community Process (JCP)

Apache Commons

Apache.org Java projects

JBoss.org

Spring

Google Guava

Trending Java projects on github

JEE and Java web servers

Apache Tomcat

JBoss Wildfly

Glassfish

Build and test automation

Sonatype Maven books

Jenkins documentation (wiki)

JUnit reference documentation

Source and version control

The SVN reference book

Git reference documentation

Java IDEs

Intellij IDEA documentation

Eclipse documentation

Netbeans knowledge base

Vim configuration for Java coding

Using libgdx for cross-platform app development

I am looking for a framework that allows me to develop modern apps (mobile, web, desktop) all from one Java codebase. I prefer Java because I know it very well, it is already cross-platform and a statically typed language that allows IntelliJ, Eclipse and Netbeans to be better than any dynamically typed scripting language editor could ever be.

Currently my favorite is libgdx. I am planning to use it with IntelliJ Community Edition and with Maven.

By using RoboVM, libgdx even supports iOS.

For user input (forms) libgdx provides the scene2d.ui widgets. I hope that will be sufficient for most of my UIs. Now I just have to get OpenGL to work on my Linux box …

Java map comparison with detailed error messages using Guava

When you compare two Java maps that are supposed to be equal, i.e. contain the same name/value pairs, you might want to give some details about potential mismatches, for example in your log output.

The Guava library from Google provides a convenient tool for that, namely the class com.google.common.collect.MapDifference.

In the sample code below I have implemented a simple utitily method that compares two maps and logs detailed error messages if they are not equal.

This code is also the first item in my guava-based github project.

Maven dependencies

Put this into pom.xml :

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>16.0</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.5</version>
</dependency>

You will also need an slf4j implementation, like logback. For testing, we can use this :

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>1.7.5</version>
</dependency>

And for the JUnit tests further below you will need this :

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.10</version>
    <scope>test</scope>
</dependency>

Java class

Save this as src/main/java/net/doepner/util/MapDiffUtil.java :

package net.doepner.util;

import java.util.Map;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.google.common.collect.MapDifference;
import com.google.common.collect.Maps;

import static com.google.common.collect.MapDifference.ValueDifference;

/**
 * Map comparison with detailed log messages
 */
public class MapDiffUtil {

    private static final Logger log =
        LoggerFactory.getLogger(MapDiffUtil.class);

    public static <K, V> boolean validateEqual(
               Map<K, V> map1, Map<K, V> map2,
               String map1Name, String map2Name) {

        final MapDifference<K, V> diff = Maps.difference(map1, map2);

        if (diff.areEqual()) {
            log.info("Maps '{}' and '{}' contain exactly the same "
                   + "name/value pairs", map1Name, map2Name);
            return true;

        } else {
            logKeys(diff.entriesOnlyOnLeft(), map1Name, map2Name);
            logKeys(diff.entriesOnlyOnRight(), map2Name, map1Name);
            logEntries(diff.entriesDiffering(), map1Name, map2Name);
            return false;
        }
    }

    private static <K, V> void logKeys(
                Map<K, V> mapSubset, String n1, String n2) {
        if (not(mapSubset.isEmpty())) {
            log.error("Keys found in {} but not in {}: {}",
                n1, n2, mapSubset.keySet());
        }
    }

    private static <K, V> void logEntries(
                Map<K, ValueDifference<V>> differing, 
                String n1, String n2) {
        if (not(differing.isEmpty())) {
            log.error("Differing values found {key={}-value,{}-value}: {}",
                        n1, n2, differing);
        }
    }

    private static boolean not(boolean b) {
        return !b;
    }
}

Unit tests

Save this as src/main/java/net/doepner/util/MapDiffUtilTest.java :

package net.doepner.util;

import java.util.HashMap;
import java.util.Map;

import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;

/**
 * Tests MapDiffUtil
 */
public class MapDiffUtilTest {

    private final Logger log = LoggerFactory.getLogger(getClass());

    @Rule
    public TestName testName = new TestName();

    @Before
    public void logTestName() {
        log.info("Executing {}", testName.getMethodName());
    }

    @Test
    public void testEqual() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);
        map1.put("B", 2);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 2);
        map2.put("A", 1);

        assertTrue("Maps should be equal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));
    }

    @Test
    public void testSubset() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 2);
        map2.put("A", 1);

        assertFalse("Maps should be unequal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));

    }

    @Test
    public void testSeparate() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 2);

        assertFalse("Maps should be unequal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));
    }

    @Test
    public void testMismatches() {
        final Map<String, Integer> map1 = new HashMap<String, Integer>();
        map1.put("A", 1);
        map1.put("B", 2);

        final Map<String, Integer> map2 = new HashMap<String, Integer>();
        map2.put("B", 20);
        map2.put("C", 3);

        assertFalse("Maps should be unequal", MapDiffUtil.validateEqual(
            map1, map2, "map1", "map2"));

    }
}

Transparently improve Java 7 mime-type recognition with Apache Tika

Java 7 comes with the method java.nio.file.Files#probeContentType(path) to determine the content type of a file at the given path. It returns a mime type identifier. The implementation actually looks at the file content and inspects so-called “magic” byte sequences, which is more reliable than just trusting filename extensions.

However, the default implementation included in Java 7 seems to be platform dependent and not very complete. For example, for me it did not even recognize an mp3 file as audio/mpeg. Fortunately, the Open Source library Apache Tika provides more comprehensive mime type detection and seems to be platform independent.

As shown below, you can register a simple Tika based FileTypeDetector implementation with the Java Service Provider Interface (SPI) to transparently enhance the behaviour of java.nio.file.Files#probeContentType(path). As soon as the resulting jar is in your classpath, the SPI mechanism wil pick up our implementation class and Files.probeContentType(..) will automatically use it behind the scenes.

Maven dependency

        <dependency>
            <groupId>org.apache.tika</groupId>
            <artifactId>tika-core</artifactId>
            <version>1.4</version>
        </dependency>

FileTypeDetector.java

package net.doepner.file;

import java.io.IOException;
import java.nio.file.Path;

import org.apache.tika.Tika;

/**
 * Detects the mime type of files (ideally based on marker in file content)
 */
public class FileTypeDetector extends java.nio.file.spi.FileTypeDetector {

    private final Tika tika = new Tika();

    @Override
    public String probeContentType(Path path) throws IOException {
        return tika.detect(path.toFile());
    }
}

Service Provider registration

To register the implementation with the Java Service Provider Interface (SPI), you need to have a plaintext file /META-INF/services/java.nio.file.spi.FileTypeDetector in the same jar that contains the class net.doepner.file.FileTypeDetector. The text file contains just one line with the fully qualified name of the implementing class:

net.doepner.file.FileTypeDetector

With Maven, you simply create the file src/main/resources/META-INF/services/java.nio.file.spi.FileTypeDetector containing the line shown above.

See the ServiceLoader documentation for details about Java SPI.

Play MP3 or OGG using javax.sound.sampled, mp3spi, vorbisspi

I tried to come up with the simplest possible way of writing a Java class that can play mp3 and ogg files, using standard Java Sound APIs, with purely Open Source libraries from the public Maven Central repositories.

The LGPL-licensed mp3spi and vorbisspi libraries from javazoom.net satisfy these requirements and worked for me right away. As service provider implementations (SPI), they transparently add support for the mp3 and ogg audio formats to javax.sound.sampled, simply by being in the classpath.

For my AudioFilePlayer class below I basically took the example code from javazoom and simplified it as much as possible. Please note that it requires Java 7 as it uses try-with-resources.

Maven dependencies

  <!-- 
    We have to explicitly instruct Maven to use tritonus-share 0.3.7-2 
    and NOT 0.3.7-1, otherwise vorbisspi won't work.
   -->
<dependency>
  <groupId>com.googlecode.soundlibs</groupId>
  <artifactId>tritonus-share</artifactId>
  <version>0.3.7-2</version>
</dependency>
<dependency>
  <groupId>com.googlecode.soundlibs</groupId>
  <artifactId>mp3spi</artifactId>
  <version>1.9.5-1</version>
</dependency>
<dependency>
  <groupId>com.googlecode.soundlibs</groupId>
  <artifactId>vorbisspi</artifactId>
  <version>1.0.3-1</version>
</dependency>

AudioFilePlayer.java

package net.doepner.audio;

import java.io.File;
import java.io.IOException;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine.Info;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.SourceDataLine;
import javax.sound.sampled.UnsupportedAudioFileException;

import static javax.sound.sampled.AudioSystem.getAudioInputStream;
import static javax.sound.sampled.AudioFormat.Encoding.PCM_SIGNED;

public class AudioFilePlayer {

    public static void main(String[] args) {
        final AudioFilePlayer player = new AudioFilePlayer ();
        player.play("something.mp3");
        player.play("something.ogg");
    }

    public void play(String filePath) {
        final File file = new File(filePath);

        try (final AudioInputStream in = getAudioInputStream(file)) {
            
            final AudioFormat outFormat = getOutFormat(in.getFormat());
            final Info info = new Info(SourceDataLine.class, outFormat);

            try (final SourceDataLine line =
                     (SourceDataLine) AudioSystem.getLine(info)) {

                if (line != null) {
                    line.open(outFormat);
                    line.start();
                    stream(getAudioInputStream(outFormat, in), line);
                    line.drain();
                    line.stop();
                }
            }

        } catch (UnsupportedAudioFileException 
               | LineUnavailableException 
               | IOException e) {
            throw new IllegalStateException(e);
        }
    }

    private AudioFormat getOutFormat(AudioFormat inFormat) {
        final int ch = inFormat.getChannels();
        final float rate = inFormat.getSampleRate();
        return new AudioFormat(PCM_SIGNED, rate, 16, ch, ch * 2, rate, false);
    }

    private void stream(AudioInputStream in, SourceDataLine line) 
        throws IOException {
        final byte[] buffer = new byte[65536];
        for (int n = 0; n != -1; n = in.read(buffer, 0, buffer.length)) {
            line.write(buffer, 0, n);
        }
    }
}

Minimal pom.xml for executable jar

When you develop a stand-alone Java application, Maven can create the executable jar for you, with main-class and classpath manifest entries. You just need the configuration of the maven-jar-plugin shown below.

The sample pom.xml also specifies that we use Java 7 and UTF-8. You can take out the sourceEncoding and maven-compiler-plugin configurations, if you want to go with defaults instead.

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>net.doepner</groupId>
    <artifactId>executable-jar-sample</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.0</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.4</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <mainClass>net.doepner.sample.Main</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Here is a matching minimal main class, src/main/java/net/doepner/sample/Main.java:

package net.doepner.sample;

import javax.swing.JOptionPane;

/**
 * The class that has the main method
 */
public class Main {

    public static void main(String[] args) {
        JOptionPane.showMessageDialog(null, "It works!");
    }
}