The following worked for me on Ubuntu 10.04 LTS as of September 2011. I find Eclipse to be an excruciating form of torture, so I’m not following the Eclipse-based instructions. Everything below can be done from vi or emacs and the command line.

Installation

1. Download the SDK from http://developer.android.com/sdk/index.html.
2. Unpack the SDK.
3. Run android-sdk-linux_x86/tools/android to bring up the AVD Manager. (AVD is “Android Virtual Device” — it will run in the emulator.)
4. Select “Available Packages”, and expand “Android Repository”. It will fetch a list of packages.
5. Select “Android SDK Platform-Tools, Version 7″. (You need this.)
6. Select “SDK Platform Android 2.2, API 8, revision 3″. (You need at least one platform. My development phone has Android 2.2, so that’s what I’m initially targeting. You can add more platforms later.)
7. I also selected Documentation and Samples, for convenience.
8. Install the packages you selected. Restart ADB when prompted.
9. Then create a new virtual device. In the AVD Manager, click “Virtual Devices”, then “New…”. Give your device a name, choose the platform, click Create.
10. Install the ubuntu package “ant1.8″. Don’t install “ant” — this is version 1.7, which won’t work with the android SDK.

Create a New Project

I unpacked the tools under ~/projects/android.

From that directory, I can give this command:

./android-sdk-linux_x86/tools/android create project \
   -p ./hello_android -t android-8 -a HelloAndroid \
   -k "com.blakitasoftware.hello_android"

This creates ~/projects/android/hello_android/ containing the new project.

Build the Project

Change into the hello_android directory. Run ant debug.

(Info on package signing and building in debug mode: http://developer.android.com/guide/developing/building/building-cmdline.html#DebugMode.)

Run the Application

Since we built in debug mode, we don’t need to worry about signing — we can run it in the emulator right away.

First start the virtual device: run android with no arguments, find your device, click “Start…”, “Launch”.

Install the application by running (assuming you’re still in the hello_android directory):

../android-sdk-linux_x86/platform-tools/adb \
   install ./bin/HelloAndroid-debug.apk

Then go to the emulator, click the Launcher. (This is the grid of little gray squares in the bottom of the screen. Yeah, it took me a few seconds to figure this out — I’ve never even used an Android phone, this really is an adventure…) Find your HelloAndroid app, click it, and you’ll see the greeting (“Hello World, HelloAndroid”).

Make a Change

To change that greeting, edit res/layout/main.xml.

Assuming you only have one emulator running, the build and install sequence given above can be abbreviated to ant install. This reinstalls your app with the new greeting, find your app in the launcher again and rerun it to see the change.

1. Install wine. (sudo aptitude install wine)
2. Install python into the wine environment. (Download an msi from python.org and run msiexec /i python-x.x.x.msi.)
3. Install whatever prerequisite packages you need (e.g. wxPython) using wine or msiexec.
4. When you’ve got everything ready to build, just do wine c:/Python27/python.exe setup.py bdist_wininst and look in ./dist/ for your exe!

Let’s say you have a half-dozen machines at work you want to log into. Instead of setting up a remote forwarding connection from each of those machines, you can have the connection from your main machine perform multiple forwardings instead of just one. This even works if some of the machines don’t support ssh.

It shouldn’t surprise you at this point that you can do this with your config file. On your work machine, you might have something like:

Host tunnel
  HostName cloud.example.com
  User mycloudusername
  IdentityFile ~/.ssh/id_dsa
  Port 22
  RSAAuthentication yes
  PubkeyAuthentication yes
  ExitOnForwardFailure yes
  # tunnel ssh to myworkmachine
  RemoteForward 4022 localhost:22
  # tunnel remote desktop to mywindowsbox via myworkmachine
  RemoteForward 5389 192.168.4.10:3389
  # tunnel http to mywindowsbox via myworkmachine
  RemoteForward 5080 192.168.4.10:80
  # tunnel remote desktop to otherwindowsbox via myworkmachine
  RemoteForward 6389 192.168.4.11:3389
  # tunnel ssh to workserver via myworkmachine
  RemoteForward 7022 192.168.4.2:22

You can add a bunch of forwardings as shown above. Each entry will open the given port on cloud and forward it to the specified port on the specified machine. Now when you run “ssh tunnel” on your work machine, it will connect to cloud and set up the five port forwardings specified in your config file.

Then when logged in to cloud.example.com, you can do, for example, “ssh -p 7022 myserverlogin@localhost” to log into the machine called workserver.

If you mirror the remote forwardings in your home config file as local forwardings, then when you “ssh work” from home you can remote desktop to a windows machine from your home pc by doing “rdesktop -u myworkwinuser localhost:5389″ and it will use the tunnel. (The connection will go from your home pc to cloud, to myworkmachine, to mywindowsbox.) The windows machine does not need to know anything about ssh.

As I mentioned in a previous post on ssh configuration, your config file can specify a variety settings for each server.

In fact, the Hosts you use don’t even have to exist! (The HostName is the important part.) Consider the following snippet in your ~/.ssh/config.

#
Host work
  HostName localhost
  User myworklogin
  IdentityFile ~/.ssh/id_dsa
  Port 4022
  RSAAuthentication yes
  PubkeyAuthentication yes
  LocalForward 4022 localhost:4022

I’m going to assume remote forwarding is set up and the connection is open from work to cloud as described in this post on remote forwarding; and you’ve got local forwarding set up from home to cloud as described in this post on local port forwarding.

Now you can do “ssh work” from your home pc, and it will automatically log you into your work pc with the right credentials using the tunnel on cloud.example.com. And the scp example above simplifies to “scp work:/tmp/foo.txt ~/foo.txt” — you don’t have to remember the forwarded port numbers.

Typing “ssh work” is nine keystrokes (eight letters plus enter). If you can type 40 wpm, that’s 200 keystrokes per minute, or 3.33 keystrokes per second, which means you can open the tunnel in four seconds!

If you add “alias ssw=’ssh work’” to your ~/.bashrc, you’re down to four keystrokes.

Local port forwarding is the same as remote port forwarding but works in the opposite direction. An example is the clearest way to explain.

Assuming you’ve done the steps in the previous posts, then at home you can run “ssh -L 4022:localhost:4022 me@cloud.example.com”. This listens on TCP port 4022 on your home machine. Any connections there will be forwarded through the ssh connection to port 4022 on cloud… which, as we recall, gets forwarded to port 22 (ssh) at work. If you leave this connection open, you can run “ssh -p 4022 localhost” on your home machine and it will connect to work in just one hop. This means that you can use scp to copy files from home to work or vice versa. For example, “scp -P 4022 localhost:/tmp/foo.txt ~/foo.txt” will copy a file from work to home. (Note: scp needs capital “-P” to give the port. I got it wrong the first time.)

Ssh tunneling can be a bit mind bending at first, but it’s simple when you get used to it. Assume that you’re trying to ssh between two sites that do not allow incoming ssh. Maybe your IT at work is unenlightened and doesn’t have an ssh gateway. And your ISP has braindead configuration rules that don’t allow incoming ssh or they make it difficult.

What you need to get around this is a server “in the cloud” that permits ssh logins. This could be a hosting server that you pay for, or even a friend with an enlightened ISP who will give you a login account.

On your work PC, use ssh to login to the “cloud” server. Using the “-R” argument, you tell ssh to listen on a TCP port on the cloud server. Any connection coming in to this server will be forwarded back through the ssh connection to the TCP port you specify. For example, on mymachine.work.com, “ssh -R 4022:localhost:22 me@cloud.example.com” tells ssh to listen on cloud’s port 4022. Incoming connections to that port on cloud will be forwarded to port 22 (ssh) on mymachine.

By default, ssh will only listen to port 4022 on cloud’s localhost interface. So to log in to work, you will first need to log into cloud, and then use “ssh -p 4022 myworklogin@localhost” to log into work.

We’ll work around this limitation in the next post in this series.

The capability that you probably aren’t using, and the easiest to use, is customizing your config file (in ~/.ssh/config) for the various servers you log into.

For example, I frequently log into about ten different servers using at least four different usernames. By default, if I type “ssh server” the client will use my login name on the client machine to try to log into the server — which is usually wrong. Instead you can tell your ssh client which username to use on each server. (Thanks to Rich Adams for the tip.)

You can customize a variety of settings — not just the username. For example, I specify a different identity file for a couple of servers.

This saves a bunch of typing and occasional confusion. (By avoiding login errrors as I try to log into a server using the wrong username and can’t figure out why my password isn’t working…)

(This is the first post in series of posts about how to get the most out of ssh. Make sure you don’t miss the rest of the series: subscribe to my feed or follow me on twitter.)

>>> from ctypes import CDLL
>>> libc = CDLL('libc.so.6')
>>> print libc.strlen('abcde')
5

As with everything else in python, it gets even better when you scratch the surface. In the example above, CDLL returns an object that represents the dynamic library. You can access the functions in that library by attribute access (“libc.strlen”) or item access (“libc['strlen']“). Both access mechanisms return a callable object.

This callable object has an “errcheck” attribute that can be assigned a callable. We can use this for error-checking our calls into the library. Let’s write a simple version of the “kill” command that uses the kill(2) system call.

import sys
from ctypes import *

# Load the library.
libc = CDLL('libc.so.6')

# Our error checking function. This will receive the
# return value of the library function, the function that
# was called, and the arguments passed to the function as a
# tuple.
def kill_errcheck(retval, func, funcargs):
    '''Check for error -- retval == -1.'''
    if retval < 0:
        raise Exception('kill%s failed' % (funcargs, ))
    return True

# Get the kill function from the standard library.
kill = libc.kill

# Set the error checker for kill().
kill.errcheck = kill_errcheck

# Pass the command line argument as a pid to kill, with
# SIGSEGV (11).
pid = int(sys.argv[1])
kill(pid, 11)

Save this as kill.py. Then, in your shell, try something like this:

# Notice that the 3401 is the pid of the process
# we're putting into the background. Yours will
# be different.
bash$ sleep 120&
[1] 3401
bash$ python kill.py 3401
[1]+ Segmentation Fault         sleep 120
bash$ python kill.py 3401
Traceback (most recent call last):
  File "kill.py", line 17, in
    kill(pid, 11)
  File "kill.py", line 10, in kill_errcheck
    raise Exception('kill%s failed' % (funcargs, ))
Exception: kill(3401, 11) failed

At line 4 of the output we run sleep in the background. At line 5 we learn the pid of this process. At line 6 we run our kill program, giving it the pid we just spawned, and we see the notification from bash that the process was killed (with signal 11, segmentation fault). At line 8 we run our kill program again on pid 3401, but it doesn’t exist, the kill system call returns -1, and our error checker raises an exception when it detects the system call failure.

But wait, there’s more… I’m working on a follow up post that combines ctypes.Structure with calls into a linux system call.

You can run the file to see the behavior. Simply provide an integer 1-5 as a command line argument and you’ll run the selected scenario and see the output. The goal in this case is for cleanup to occur and an exception to be reported as having occurred at line 10.

The code in reraiser1 is wrong because this behaves as if a brand new exception were thrown at line 27. That may not seem so bad, but this code is pretty simple. If this happens and the stack trace is deep, it will be almost impossible to diagnose what went wrong.

The code in reraiser2 shows what happens when a second exception occurs in the except block. A bare raise statement here might be an attempt to re-raise the original exception, but python’s rules about re-raising specify that the most recent exception in the scope is what is reraised. In this case, that’s the exception thrown from the cleanup function. Again, this makes troubleshooting difficult.

In reraiser3 I worked around the problem in reraiser2 by moving the cleanup function’s exception into a separate scope by defining a local cleanup function and calling it from within the except block. This prevents the cleanup function’s exception from polluting the scope with an irrelevant exception and we can re-raise the original exception. This results in a stack trace rooted at line 10.

Reraiser4 takes a different approach. Instead of moving the cleanup function’s exception into a separate scope, it captures the traceback information from the original exception and then passes it back to the raise statement so that the reported traceback is accurate.

The cleanest way to handle this is to use a finally block as shown in reraiser5. This situation is what “finally” is meant for: it does not trap the exception, it just gives you a chance to clean up before control moves back up the stack to the caller. The presence of finally clues readers in to the fact that you aren’t messing with the exception, and that the point of the block is to perform cleanup.

Kindly drop me a note if I’ve got something wrong above, or if I’m missing a technique (or a common anti-pattern!). Thanks.

In case you missed any of the previous posts in this series, here they are:

1. Day One, Get Ready (Concept and prep)
2. Day Two, Mockups (Creating a design)
3. Day Three, Coding (Coding tests, views, templates, and models)

Pre-Deployment

First, we need to make a couple of decisions:

• How are we going to push updates to the live site: FTP, git, svn?
• How are we going to handle backups?

Version Control as Distribution System

In my case, I’m using svn+ssh to push updates. Notice that this does not require special setup on your server — you do not need to install the svn stuff that DreamHost or your host may provide. Just do svn init to create a repository on the server (outside the DocumentRoot!). Then point your development pc to svn+ssh://USERNAME@host.example.com/home/USERNAME/svn/PROJECT. (Git works similarly, no support from your host required except the binaries.)

In the directory on the host where you’re going to store your project files, point to the same URL. (You could use the file:///…/ url, but I prefer to avoid accessing the repo directly. Superstition?)

Now, whenever you make a change on your development system, just "svn ci" and then on the host “svn up” and restart your fcgi to pick up the new code. Presto! The live site is updated with your change.

Backups

You must have a backup strategy: Your app will have users. Your host’s disk will burp. Your users will hate you when the disk burps and you don’t have a good backup.

There are probably 374 different ways of backing up your Django app. The two major things you need to capture are the database and your code. If you are storing objects (e.g. uploaded files) outside the database, you’ll need to back these up too. The option I’m using is django-backup.

Pull the code from subversion into your project. Rename the directory to "django_backup". Add django_backup to your INSTALLED_APPS. Verify it works by running ./manage.py backup -c. Sanity check the backup by doing zless backups/*.gz. We’ll set up a cron job on the host to run this regularly when we deploy.

Deploy

I’ve previously written about deploying Django apps on DreamHost, so I’m not going to duplicate that here. Keep in mind that you want to use the version control strategy outlined above. Read that article, deploy your app and come back here when you’re done.

Post-Deployment

Backup

Let’s add that cron job we previously mentioned. On the host, run crontab -e. If you’re on DreamHost and this if the first time you’ve used cron, it will prompt you for an email address to send output to. Then it will dump you into an editor. (Side note: "joe" is the default. If you want something different, like vim, be sure that EDITOR=/usr/bin/vim is set in your environment.)

Set up a job something similar to the following:

MAILTO="YOU@EXAMPLE.com"

# m h  dom mon dow   command
4 2 * * * (cd /home/PATH/TO/PROJECT; ./manage.py backup -c --email=YOU@EXAMPLE.com)

This will run a backup every day at 02:04 (AM). You will get two emails: one with the output from the job, and one with the compressed backup file. (When you get to the point where your database backups are too big for email, you’ll need to find another strategy.)

Now we’ve got another problem to solve: we’re going to accumulate a bunch of backups on the disk. Let’s get rid of the old backups. This is pretty safe, since we’re receiving backup files via email. Add a cron job like this:

14 2 * * * (cd /home/PATH/TO/PROJECT;
    touch --date=`date --iso --date='10 days ago'` .backup.oldest;
    find ./backups/ -mindepth 1 \! -newer .backup.oldest -execdir rm '{}' +)

(Formatted here for readability — you need to put that all on one line.)

This will remove backup files older than 10 days. You could do this more concisely with a tool like tmpwatch or tmpreaper, but neither is installed on my host and this incantation should work on pretty much any flavor and installation of linux.

At this point we’re deployed and the majority of the work is done. Tomorrow we’ll take a look at some maintenance issues.