The Daily Build

The task: take a bunch of photos from my digital camera and burn them to a DVD that would play the photos as a slideshow in any DVD player. (Actually, the task was to figure out how to do this in the simplest possible way, and make a set of instructions for an unsophisticated user to follow.)

The result: several hours of fighting with a dozen different applications before I actually had photos displayed on my TV. And this post.

I don’t understand why what seems like a fairly simple task is so complicated.

This is what I found to be the easiest recipe on a stock Ubuntu 11.10 installation. Fair warning: this is quick & dirty, you don’t get special effects, you don’t get music, etc.

1. Required software (load through the software center, or via apt-get): imagination, devede, brasero.
2. Open Imagination. Click the import photos icon (it is a terrible choice of icon — there’s no clue that you should want to click on the black square to import photos) and choose your photos.
3. Export video. Choose DVD for output; to keep it organized, drop the video file in the Videos directory in your home directory. This can take a while if you have a slow computer and/or a lot of photos.
4. Open DeVeDe. Configure the title. Import the video you just generated.
5. Generate the DVD ISO image; again save the image to your Videos folder.
6. Insert a DVD in the drive. Open your Videos folder. Right click on the ISO image and choose Write to Disc.

Some notes, in no particular order:

• If you want to tweak the slideshow, imagination will let you play around with transitions, slide duration, etc.
• If you want to import video clips, use Openshot instead of Imagination. It’s harder to use (and overkill) compared to Imagination if all you want to do is produce a static photo slideshow. But if you want to do any video editing, it’s a good combination of power and ease of use.
• If you want music, Openshot might also be your best bet. I didn’t look at adding music via Imagination or DeVeDe.
• Don’t try to use the Brasero application directly. I experienced crashes/lockups while using it, but it worked well when all I did was right click the ISO and choose Write to Disc. (It’s also an ugly, hard to use piece of junk.)

Things I tried and rejected:

• K3b: After Brasero failed miserably twice, I tried K3b. Rejected it in favor of not pulling in a bunch of KDE infrastructure that I wasn’t using anyway, and also in favor of the simplicity of just directly burning the file without having to explicitly open a separate app.
• dvd-slideshow: This command line app does appeal to my inner geek, but I can’t recommend it for users who want a GUI. (Even the inner geek appeal isn’t enough to make me want to use it when the combination outlined above works well.)
• Mistelix: As good as Imagination, and I would have selected it if I had quickly found a way to rotate a photo from within Mistelix. Photos that showed up auto-rotated in other apps showed up sideways in Mistelix and I didn’t see an obvious rotate button. Imagination did the right thing.
• Bombono: Junk. Don’t bother with it. I opened it, saw that the tabs were overlapping the menus, couldn’t figure out how to import any photos, and purged it from my system.
• Videoporama: Development is dead. It wasn’t immediately obvious that it was going to be easy to use for the simple task of creating a DVD from a set of photos.

The software stack above 10x better if Imagination implemented a feature to export a DVD ISO. It would be 100x better if it had a “Burn DVD” button that spit out a shiny disc that I could drop into a DVD player.

(To be fair, the whole video situation is 1000x easier than it was 5 or 6 years ago when I tried to do something similar, realized how much work it was going to be, and walked away in frustration. The tools have come a long way.)

Like this post? Get updates from my feed.

The auth.log on one of my servers (really, on all of the servers I have access to) is full of stuff like this:

Dec  8 03:19:33 localhost sshd[4718]: User root from 10.1.2.3 not allowed [...]
Dec  8 03:19:35 localhost sshd[4721]: Invalid user db2inst1 from 10.1.2.3
Dec  8 03:19:38 localhost sshd[4723]: User root from 10.1.2.3 not allowed [...]

fail2ban is configured to (temporarily) block these after a certain number of attempts, but they keep coming back. One particular IP address was hitting ssh constantly (except for the ban periods) for several days, so I added a rule to drop everything from that address — but this strategy isn’t scalable.

The simple, obvious solution is to only allow access from known hosts. It would be very rare that I need to access this server from anywhere except a small number of addresses. I started with a ufw (Ubuntu’s “uncomplicated firewall”) ruleset like this:

% sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
Anywhere                   DENY        10.1.2.3
Anywhere                   DENY        172.16.99.88
22                         ALLOW       Anywhere

To avoid getting locked out, I added (temporarily) a crontab entry for root:

*/15 * * * * /usr/sbin/ufw allow ssh

This will allow access to ssh from any un-banned IP address (the current policy) every 15 minutes. Then I inserted a rule that allows access to ssh from my home ISPs netblock. (You can figure out the netblock by doing a whois lookup on your external IP address; you may need to add multiple netblocks if your ISP has several allocated. Be careful: it’s no fun to get locked out!) This action is safe because I have not yet removed global ssh access. Note that I’m using “insert 3″ to add this rule at a specific position in the list.

sudo ufw insert 3 allow proto tcp from 10.9.8.0/18 to any port 22

And I added a rule to permit access from another server with a fixed IP I have access to (for the rare case where I need to access this server when I’m not at home):

sudo ufw insert 3 allow proto tcp from 172.17.101.102 to any port 22

Now my ruleset looks like:

Status: active

To                         Action      From
--                         ------      ----
Anywhere                   DENY        10.1.2.3
Anywhere                   DENY        172.16.99.88
22/tcp                     ALLOW       172.17.101.102
22/tcp                     ALLOW       10.9.8.0/18
22                         ALLOW       Anywhere

So far, so good, but ssh access is still permitted from anywhere — because of that last rule. This is the dangerous part… you could get locked out if you haven’t set the rules correctly. (You set up that crontab entry, right?)

sudo ufw delete allow ssh

Wait for the prompt… hooray, I’m still connected! After checking that I can access ssh from home and the other server, I know it’s safe to remove the crontab job. (If the cron job has already fired, you’ll need to rerun the ufw delete allow ssh command.)

At this point I can delete the first two rules that ban specific IPs, since they’re outside my netblock and won’t be allowed anyway.

Now I can enjoy quieter logs without all those access attempts from China and Croatia!

My ISP has apparently made no progress whatsoever with IPv6, but I’ve got an IPv6 enabled VPS.

SSH makes it trivial to use that VPS as a SOCKS5 proxy. Just do:

ssh -D 8080 myvps.example.com

Then set your browser’s SOCKS proxy to localhost:8080. In Firefox on Linux, this is Edit > Preferences > Advanced > Network tab > (Connection) Settings > Manual Proxy Configuration. Leave all fields blank except for SOCKS Host and Port — localhost and 8080, respectively. Choose SOCKS5. Then browse to about:config and change change network.proxy.socks_remote_dns to true. This tells Firefox to ask the proxy to resolve names instead of trying to resolve them using your ISP. Chrome worked for me without hassle.

Go to test-ipv6.com to test that it works. If your results from test-ipv6.com indicate that IPv6 name lookups are failing, make sure you’ve got that about:config setting mentioned above changed.

“AnyConnect cannot confirm it is connected to your secure gateway. The local network may not be trustworthy. Please try another network.”

There may be several reasons for this error, which you’ll find on other pages that hit for a search on this string.

The reason that I encountered seems to be unique. What I found by digging into a wireshark capture is that AnyConnect sends a TLS alert to the server, disconnecting the session. The alert message says “Unknown CA”.

It could have something to do with installing the firefox plugin “Certificate Patrol” recently. AnyConnect apparently uses firefox’s certificate store. Perhaps Certificate Patrol does something to the store that makes it so that AnyConnect can no longer use it?

In case it matters, I’m on Ubuntu 10.04.

Whatever the cause, you can fix it by doing the following:

1. Figure out the CA that signs your VPN server’s certificate. (Hat tip to Didier Stevens for the easy way to do this.)
   1. Fill in your server, I’m going to use www.google.com.
   2. openssl s_client -connect www.google.com:443 >! /tmp/google
   3. Hit ctrl-C.
   4. You saw “Thawte Consulting (Pty) Ltd.”.
2. Now search for “thawte root certificate”. (Obviously this will vary depending on who signs your server’s certificate.)
3. Go to the page where they list certificates. Download them all. (I tried downloading the one that looked like it matched the signature on my server’s certificate, but I think you need everything down to the root in order to fully verify it. It’s easier to just download everything than try to figure out exactly which ones are needed.)
4. Convert to PEM format. (Hat tip to Mozekoze for this recipe.)
   1. openssl x509 -in input.crt -out input.der -outform DER ## (if the certs are in .crt format)
   2. openssl x509 -in input.der -inform DER -out output.pem -outform PEM
5. Copy all the PEM files to /opt/.cisco/certificates/ca. (Or to ~/.cisco/certificates/ca.)

Alternatively, you could copy the certs out of the keystore on your machine, convert to PEM, and then copy the PEMs into the directory mentioned above. (On ubuntu, you can see the certificates in the package ca-certificates. You could copy just the thawte certs by doing something like “cp $(dpkg -L ca-certificates | grep -i thawte) /tmp/certificate-conversion/”.)

Three cheers for wireshark, strace, openssl, and google for help figuring out what was going on. Cisco could make things a little bit easier to figure out — a decent diagnostic message would have been great — just tell the user “Unknown CA”, it’s already buried in the wireshark trace.

I have not yet played with python 3, which sounds like it makes working with Unicode easier.

In python 2.x, working with Unicode can be annoying, but when you remember this rule of thumb, it’s much easier and allows you to keep your sanity:

Always use unicode strings internally. Decode whatever you read/receive. Encode whatever you write/send.

I’ve recently done some work on a project to generate ReStructuredText (as an intermediate form on the way to generating HTML). The input data has Unicode sprinkled throughout (in UTF-8). I kept getting UnicodeDecodeError: 'ascii' codec can't decode byte exceptions in various places, until I applied that rule everywhere:

• When reading, decode from UTF-8.
• Use Unicode strings u'E.g. this formatted %s string' % (decoded_string1, decoded_string2) internally — everywhere.
• When writing, encode to UTF-8.

Hat tip to nosklo on StackOverflow for mentioning this simple rule.

Here are my notes about getting the Android SDK up and running using only the command line tools that come in the SDK.

The following worked for me on Ubuntu 10.04 LTS as of September 2011. I find Eclipse to be an excruciating form of torture, so I’m not following the Eclipse-based instructions. Everything below can be done from vi or emacs and the command line.

Installation

1. Download the SDK from http://developer.android.com/sdk/index.html.
2. Unpack the SDK.
3. Run android-sdk-linux_x86/tools/android to bring up the AVD Manager. (AVD is “Android Virtual Device” — it will run in the emulator.)
4. Select “Available Packages”, and expand “Android Repository”. It will fetch a list of packages.
5. Select “Android SDK Platform-Tools, Version 7″. (You need this.)
6. Select “SDK Platform Android 2.2, API 8, revision 3″. (You need at least one platform. My development phone has Android 2.2, so that’s what I’m initially targeting. You can add more platforms later.)
7. I also selected Documentation and Samples, for convenience.
8. Install the packages you selected. Restart ADB when prompted.
9. Then create a new virtual device. In the AVD Manager, click “Virtual Devices”, then “New…”. Give your device a name, choose the platform, click Create.
10. Install the ubuntu package “ant1.8″. Don’t install “ant” — this is version 1.7, which won’t work with the android SDK.

Create a New Project

I unpacked the tools under ~/projects/android.

From that directory, I can give this command:

./android-sdk-linux_x86/tools/android create project \
   -p ./hello_android -t android-8 -a HelloAndroid \
   -k "com.blakitasoftware.hello_android"

This creates ~/projects/android/hello_android/ containing the new project.

Build the Project

Change into the hello_android directory. Run ant debug.

(Info on package signing and building in debug mode: http://developer.android.com/guide/developing/building/building-cmdline.html#DebugMode.)

Run the Application

Since we built in debug mode, we don’t need to worry about signing — we can run it in the emulator right away.

First start the virtual device: run android with no arguments, find your device, click “Start…”, “Launch”.

Install the application by running (assuming you’re still in the hello_android directory):

../android-sdk-linux_x86/platform-tools/adb \
   install ./bin/HelloAndroid-debug.apk

Then go to the emulator, click the Launcher. (This is the grid of little gray squares in the bottom of the screen. Yeah, it took me a few seconds to figure this out — I’ve never even used an Android phone, this really is an adventure…) Find your HelloAndroid app, click it, and you’ll see the greeting (“Hello World, HelloAndroid”).

Make a Change

To change that greeting, edit res/layout/main.xml.

Assuming you only have one emulator running, the build and install sequence given above can be abbreviated to ant install. This reinstalls your app with the new greeting, find your app in the launcher again and rerun it to see the change.

Something I didn’t know until recently:

zsh does not require cd to change directories.

Using the directory as a command implies “cd”. For example, instead of doing:

me@server:/home/me $ cd /tmp
me@server:/tmp $

You can just do:

me@server:/home/me $ /tmp
me@server:/tmp $

That’s three whole keystrokes (nearly half the command shown above), all day long.

A python-based project I’m working on has a win32 user that I need to support. Until yesterday I’ve been moving to a win32 laptop in order to run python setup.py bdist_wininst so I can produce a self-installing executable. Then I discovered how trivial it is to use wine to do the job:

1. Install wine. (sudo aptitude install wine)
2. Install python into the wine environment. (Download an msi from python.org and run msiexec /i python-x.x.x.msi.)
3. Install whatever prerequisite packages you need (e.g. wxPython) using wine or msiexec.
4. When you’ve got everything ready to build, just do wine c:/Python27/python.exe setup.py bdist_wininst and look in ./dist/ for your exe!

It’s a hassle to have to switch from emacs to the shell, run unit tests, pick out the failure, switch back to emacs, navigate to the line, forget where you were supposed to go, switch back, etc etc.

The whole process is much easier with compilation-perl.el. This handy bit of code sets up patterns for compilation-mode so that test failures (among other things) are recognized. Just add compilation-perl.el to your emacs lisp directory and follow the directions at the top of the file for installation instructions.

So my test process is now:

1. Hit F4 (which I have mapped to compile).
2. If necessary, change the command to either make test or to run a specific test I’m working on (e.g. perl t/MyModule.t).
3. If there’s a failure, press C-F4 (which I have mapped to next-error) and it brings me right to the failure.
4. Repeat from step 1 (but the second time around I don’t need to do step 2 since the command is already set properly).

Since my company’s product is based on ubuntu, we use apt to distribute upgrades. We don’t normally want customers to pull upgrades from the upstream distribution, so we’ve pruned back the main /etc/apt/sources.list to include just our server. We rename the original default sources.list so that it doesn’t get picked up.

But sometimes I want to be able to pull from upstream, so I want to reactivate the original sources.list. Up to now, I’ve been having to do an awkward dance that involves renaming our file, copying the old default to the active, and then doing the reverse when I’m done.

(As an aside, yum-based systems like Centos or Fedora make this easy. You can say enable=0 in the list file, and then use --enablerepo=myrepo on the command line. There doesn’t seem to be a way to do this with apt, but the command below is almost as good.)

To avoid this renaming dance, here’s a better solution.

First, create a directory to put the original sources.list: /etc/apt/sources.list.ubuntu.

Then copy the original default sources.list into that directory. I called it /etc/apt/sources.list.ubuntu/jaunty.list.

When you want to pull from upstream, run:

apt-get -o Dir::Etc::SourceParts=/etc/apt/sources.list.ubuntu update
apt-get -o Dir::Etc::SourceParts=/etc/apt/sources.list.ubuntu install some-new-package

Then you’ll want to run apt-get update to reset your repository list back to the trimmed-down set.