“Why do I have to pay for Redhat if it is ‘Free Software’?”

Unfortunately but quite naturally, there are many many people who are surprised when they first learn that “Free Software” is not necessarily available as a free-of-charge download in immediately usable (i.e. compiled binary) form.

“Free” is an ambiguous word in the English language: Free like “free beer” (= gratis, free of charge) versus free like “Free Speech” (= libre, based on guaranteed freedoms, liberties).

This ambiguity is an old problem of the term “Free Software” – first coined by the “Free Software Foundation” (FSF) in the 1980s – and was actually one factor that motivated the foundation of the “Open Source Initiative” (OSI) and its official definition of “Open Source”.

Both definitions use the same criteria and are essentially different names for the same category of software. To acknowledge and peacefully combine both of these naming conventions some people also speak of “Free/Libre Open Source Software” (FLOSS).

The Redhat Linux distribution is Free/Libre Open Source Software. The source code is licensed under the GPL and similar Open Source licenses and can be downloaded from Redhat’s ftp server. The binaries are not available as gratis download, which is perfectly in line with FLOSS rules.

For almost every IT professional these days, it is very beneficial to understand what “Free/Libre Open Source Software” (FLOSS) is. It might seem like a complex and dry subject at first, especially when some business folks confuse things further by using the vague term “Intellectual Property” for everything from copyright, trademarks, patents to license agreements, etc.

Minimal Debian Wheezy netinstall – no network after reboot

I just installed Debian Wheezy on a Toshiba laptop using the netinstall CD.

During installation everything worked fine, ethernet and wireless interfaces were both detected and functional. I actually did the installation using the wireless connection. On the “Task selection” screen I only selected “Laptop” and “Standard system utilities”.

However, after the reboot at the end of the installation, I had no network anymore. Here is what I did to fix it:

Edit /etc/network/interfaces and add this block:

auto eth0
allow-hotplug eth0
iface eth0 inet dhcp

Then run as root:

/etc/init.d/networking start

If the machine is connected via ethernet cable to a router, you should see messages about your network being configured via DHCP, i.e. receiving its local IP address.

If you get error messages, then you might have to install firmware for your network card:

On another computer, download firmware.tar.gz and save it to a USB stick.

Then do the following as root on the new Debian system:

Plug in the USB stick and mount it. You might have to run “dmesg” to see which device file to mount. In my case it was /dev/sdb

mount /dev/sdb /mnt

Unzip the firmware tarball:

cd /mnt
tar xvzf firmware.tar.gz

Install the core firmware packages:

dpkg -i firmware-linux-*.deb

Install additional firmware packages as necessary. Use the output of the lspci command to identify your network cards. Consult the “Firmware” Debian wiki page to find out what package you have to install.

If you are done installing firmware, restart the system:

shutdown -r now

Hopefully you will have a network connection after the reboot.

Sqoop daily Oracle data into Hive table partition

The following bash script can be used to import Oracle records into a Hive table, partitioned by date. It uses Sqoop. Both Hive and Sqoop are part of typical Hadoop distributions, like the Hortonworks Sandbox, for example.

#!/bin/sh

function upper() {
  echo "$1" | tr [a-z] [A-Z]
}

if [ $# -ge 5 ]; then
  schema=$(upper $1)
  table=$(upper $2)
  column_to_split_by=$(upper $3)
  date_column=$(upper $4)
  date_value="$5"
else 
  echo
  echo "Usage: $(basename $0) schema table column-to-split-by date-column YYYY-MM-DD"
  echo
  echo "Imports all records where value of date-column is \$date_value from"
  echo "Oracle table \$schema.\$table as a Hive table partition."
  echo "Hadoop will split the import job based on the column-to-split-by."
  echo "* The table must have the columns specified as column-to-split-by and date-column."
  echo "* The column-to-split-by must be finer granularity than date-column, ideally unique."
  echo "* The date_value must be in YYYY-MM-DD format."
  echo "* If date_value is unspecified, the current date will be used."
  exit 1
fi

echo "schema = $schema"
echo "table = $table"
echo "column_to_split_by = $column_to_split_by"
echo "date_column = $date_column"
echo "date_value = $date_value"

# we have to drop the partition, because --hive-overwrite does not seem to do it
hive -e "use $schema; alter table $table drop if exists partition($date_column='$date_value');"

columns=$( \
sqoop eval \
--options-file /usr/local/etc/sqoop-options.txt \
--query "select column_name from all_tab_columns where table_name = '$table'" \
| tr -d " |" \
| grep -Ev "\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-|COLUMN_NAME|$date_column" \
| tr '\n' ',' \
| sed -e 's/\,$//'
)

query="select $columns from $schema.$table \
       where $date_column = to_date('$date_value', 'YYYY-MM-DD') \
       and \$CONDITIONS"

echo "query = $query"

sqoop import \
--options-file "/usr/local/etc/sqoop-options.txt" \
--query "$query" \
--split-by "$column_to_split_by" \
--target-dir "$schema.$table" \
--hive-import \
--hive-overwrite \
--hive-table "$schema.$table" \
--hive-partition-key "$date_column" \
--hive-partition-value "$date_value" \
--outdir $HOME/java

JDBC connection details

Put them into /usr/local/etc/sqoop-options.txt, in a format like this:

--connect
jdbc:oracle:thin:@hostname:port:hostname
--username
oracle_username
--password
oracle_password

Save video stream as file using rtmpdump (even on Windows)

The rtmpdump tool can help you save video streams as local files.

It comes with most Linux distributions, e.g. on Ubuntu or Debian (with sudo):
sudo apt-get install rtmpdump

On Windows, the most convenient way is probably via Cygwin and Cygwin Ports:

  1. Install Cygwin
  2. Install Cygwin Ports
  3. On the package selection screen, select “rtmpdump”

Read the manual page and study the options:
man rtmpdump

Record and share shell activity via shelr.tv

Terminal

Shelr.tv allows Unix/Linux command line users to record something interesting from their terminal and share it to followers.

It is a bit like YouTube for plain text shellcasts. A great feature is that you can copy and paste everything you see.

A nice intro with interesting comments from one of the core developers can be found on linuxaria.com.

A Debian package has been proposed through the Debian “package mentor” system.

Backup/restore XFCE desktop icons

Sometimes the XFCE desktop icons get messed up, for example by games that temporarily change the screen resolution to 800×600.

A solution to this problem has been mentioned here. It suggests using “sudo chattr +i” to lock the config file where XFCE stores the icon positions.

Alternatively (and without the repeated need for sudo and chattr) you can also backup and restore the ~/.config/xfce4/desktop/icons* file(s) like this:

Create a script /usr/local/bin/save-xfce-desktop-icons.sh like this:

#! /bin/sh
mkdir -p ~/.config/xfce4/desktop.bak
cp -f ~/.config/xfce4/desktop/icons* ~/.config/xfce4/desktop.bak

Create another script /usr/local/bin/load-xfce-desktop-icons.sh like this:

#! /bin/sh
cp -f ~/.config/xfce4/desktop.bak/icons* ~/.config/xfce4/desktop

Make the scripts executable like this

sudo chmod ugo+x /usr/local/bin/save-xfce-desktop-icons.sh
sudo chmod ugo+x /usr/local/bin/load-xfce-desktop-icons.sh

Then in the XFCE start menu, go to “Settings” – “Keyboard” – “Application Shortcuts” and configure 2 keyboard shortcuts:

Command Shortcut
save-xfce-desktop-icons.sh <Control><ALT>S
load-xfce-desktop-icons.sh <Control><ALT>L

Then you will be able to backup and restore your icons like this:

  • Backup: Press F5 then <Control><ALT>S then F5
  • Restore: Press F5 then <Control><ALT>L then F5

The F5 is necessary to synchronize what you see on the screen with the content of ~/.config/xfce4/desktop/icons*.