The Daily Build

Icon

Software Development, version 3.0

Open an SSH Tunnel in Four Seconds or Less

(This is part four in a series of posts on ssh.)

As I mentioned in a previous post on ssh configuration, your config file can specify a variety settings for each server.

In fact, the Hosts you use don’t even have to exist! (The HostName is the important part.) Consider the following snippet in your ~/.ssh/config.

#
Host work
  HostName localhost
  User myworklogin
  IdentityFile ~/.ssh/id_dsa
  Port 4022
  RSAAuthentication yes
  PubkeyAuthentication yes
  LocalForward 4022 localhost:4022

I’m going to assume remote forwarding is set up and the connection is open from work to cloud as described in this post on remote forwarding; and you’ve got local forwarding set up from home to cloud as described in this post on local port forwarding.

Now you can do “ssh work” from your home pc, and it will automatically log you into your work pc with the right credentials using the tunnel on cloud.example.com. And the scp example above simplifies to “scp work:/tmp/foo.txt ~/foo.txt” — you don’t have to remember the forwarded port numbers.

Typing “ssh work” is nine keystrokes (eight letters plus enter). If you can type 40 wpm, that’s 200 keystrokes per minute, or 3.33 keystrokes per second, which means you can open the tunnel in four seconds!

If you add “alias ssw=’ssh work’” to your ~/.bashrc, you’re down to four keystrokes.

Use Local SSH Forwarding to Reduce the Number of Manual Hops

(This is part three in a series of posts on ssh.)

Local port forwarding is the same as remote port forwarding but works in the opposite direction. An example is the clearest way to explain.

Assuming you’ve done the steps in the previous posts, then at home you can run “ssh -L 4022:localhost:4022 me@cloud.example.com”. This listens on TCP port 4022 on your home machine. Any connections there will be forwarded through the ssh connection to port 4022 on cloud… which, as we recall, gets forwarded to port 22 (ssh) at work. If you leave this connection open, you can run “ssh -p 4022 localhost” on your home machine and it will connect to work in just one hop. This means that you can use scp to copy files from home to work or vice versa. For example, “scp -P 4022 localhost:/tmp/foo.txt ~/foo.txt” will copy a file from work to home. (Note: scp needs capital “-P” to give the port. I got it wrong the first time.)

How to Use SSH Remote Port Forwarding to Set Up Secure Tunnels

(This is part two in a series of posts on ssh.)

Ssh tunneling can be a bit mind bending at first, but it’s simple when you get used to it. Assume that you’re trying to ssh between two sites that do not allow incoming ssh. Maybe your IT at work is unenlightened and doesn’t have an ssh gateway. And your ISP has braindead configuration rules that don’t allow incoming ssh or they make it difficult.

What you need to get around this is a server “in the cloud” that permits ssh logins. This could be a hosting server that you pay for, or even a friend with an enlightened ISP who will give you a login account.

On your work PC, use ssh to login to the “cloud” server. Using the “-R” argument, you tell ssh to listen on a TCP port on the cloud server. Any connection coming in to this server will be forwarded back through the ssh connection to the TCP port you specify. For example, on mymachine.work.com, “ssh -R 4022:localhost:22 me@cloud.example.com” tells ssh to listen on cloud’s port 4022. Incoming connections to that port on cloud will be forwarded to port 22 (ssh) on mymachine.

By default, ssh will only listen to port 4022 on cloud’s localhost interface. So to log in to work, you will first need to log into cloud, and then use “ssh -p 4022 myworklogin@localhost” to log into work.

We’ll work around this limitation in the next post in this series.

How to Tell SSH Who You Are

Ssh has amazing capabilities that you probably aren’t using on a daily basis.

The capability that you probably aren’t using, and the easiest to use, is customizing your config file (in ~/.ssh/config) for the various servers you log into.

For example, I frequently log into about ten different servers using at least four different usernames. By default, if I type “ssh server” the client will use my login name on the client machine to try to log into the server — which is usually wrong. Instead you can tell your ssh client which username to use on each server. (Thanks to Rich Adams for the tip.)

You can customize a variety of settings — not just the username. For example, I specify a different identity file for a couple of servers.

This saves a bunch of typing and occasional confusion. (By avoiding login errrors as I try to log into a server using the wrong username and can’t figure out why my password isn’t working…)

(This is the first post in series of posts about how to get the most out of ssh. Make sure you don’t miss the rest of the series: subscribe to my feed or follow me on twitter.)

One Simple Step for Avoiding Shallow Reviews

We’ve all been guilty of giving a shallow review: “Looks ok.”

Given typical defect densities, any non-trivial design or code is going to contain some errors. Even seemingly trivial maintenance fixes are likely to be defective.

It’s your job as a reviewer to find as many of these defects as possible. If you’re not finding defects, you’re wasting your time on reviews.

That “one simple step”? Remind yourself that there are almost certainly defects in the work product you’re reviewing, and then find them. It’s all about attitude.

If the comments are ugly, the code is ugly

If the comments are ugly, the code is ugly (via slashdot). Amen! I get uncomfortable whenever I have to leave a long comment, but it’s usually to document some deficiency in a lower layer that the code is working around. Typically broken hardware. (So that someone coming behind me doesn’t say, “This is overly complicated, I can simplify it” and then proceed to blow everything up.)

“Good programs do not contain spelling errors or have grammatical mistakes.”

Who Else Wants Better Short Term Memory?

In “Talent is Overrated”, Geoff Colvin at one point discusses how superstars in many fields use the memory technique of “chunking” to boost their short term memory.

His simple example is the 13-letter word “lexicographer”. To you and I (assuming you speak English and have a decent vocabulary), it is easy to remember. We don’t have to remember 13 letters, we just remember the whole chunk. But when presented with “trgdpxhdewfwm” for 3 seconds, you probably can’t remember more than half a dozen letters.

Another example is that chess masters can recall board positions after being shown a chess board with pieces on it for just a few seconds. They do this by chunking sets of pieces together — almost like “words” — whereas novices will try to remember individual pieces (“letters”).

It struck me that programmers do the same thing when reading and writing code.

The coding standard helps us, by telling us where the chunks are and how to draw the boundaries between chunks.

If you have a coding standard.

And apply it consistently.

When you choose names at random, you destroy your short term memory. You become a crappy programmer, and you drag everyone around you down too.

Quick example in C. Assume you’re writing a small module for checking environmental status.

BOOL isOverTemp();
BOOL isUnderTemp();

int GetCurTemperature();

BOOL env_isOverVoltage();
BOOL env_isUnderVoltage();

Anyone writing this pile of gibberish should be fired excommunicated.

Why?

For starters, I just wrote it, and I can’t now remember which was camel case and which was underscored. Also: which one was abbreviated?

Before you object that this is a contrived example, two points: (1) yes, it is contrived, that’s the point of an example, and (2) I have worked with far worse on an almost daily basis — the example above is fairly tame.

On the other hand, if you know the coding standard has some simple rules — and they are followed uniformly — you can easily remember the function names.

  1. Initial 2-3 letter module prefix (“env” for this example).
  2. All functions are lowercase with underscores.
  3. No abbreviations.

Then we have:

BOOL env_is_over_temperature();
BOOL env_is_under_temperature();

int env_get_cur_temperature();

BOOL env_is_over_voltage();
BOOL env_is_under_voltage();

Now, when you’re writing, reviewing, or maintaining code, you don’t need to constantly refer to the header or documentation to get it right. A simple three-line coding standard just boosted your memory capacity by over 643%.

(In the interest of full disclosure: I made that number up.)

I realize that implementing even this simple three line standard is controversial because all the camel case folks are pulling out big sticks and the underscore people are grabbing rocks. To which I say: just flip a coin and enjoy the extra brain power. Adopt a three-line standard, follow it, and save the curly-brace debate for the next major coding standard revision.

3 Easy Ways to Stick to a Coding Standard

When you’re writing python, you don’t need a lot of debate over the minutiae of most coding standards. PEP 8 does that for you. Even better, there are some tools that make it really easy to stick to the standard.

Why do this? Well, for one thing it makes code reviews easier when everyone follows the same conventions. It also makes maintenance easier.

  1. pep8.py is a style checker that enforces the rules of PEP 8. The “official home” (?) at browsershots.org was dead as I was writing this. (Thanks GitHub!) Run pep8.py on your code and it will tell you where you’ve drifted from the standard.
  2. pylint is lint for python. It provides more functionality than pep8.py, but is not a strict superset. (pep8.py is pickier about whitespace issues.) Pylint is also configurable to enforce various naming rules. Like most lints, the SNR is pretty low, but you can turn off most of the noise and get a reasonable signal for the things you want to check.
  3. Subversion (as well as most other tools) can be configured to run a script every time you check in code. Run one or both of the above tools in the pre-commit hook and bad code will be rejected. (I’d be wary of doing this with pylint unless you’ve got the categories of “noisy” warnings turned off.) I don’t have anything cookbook for this yet, but I’m working on it…

Hassle Free Way to Kill Sudo’d Jobs

Every now and then I have to run a foreground job under sudo that doesn’t want to die when I hit ^C. Then it’s a hassle to ^Z, get the pid of the sudo job, and sudo kill that pid.

So I wrote a little script (or a template for scripts) that runs the sudo job in the background (but preserves stdout/stderr) and relies on bash to clean up the job when you ^C the script. Only gotcha with this is that you may have to retype your sudo password when you ^C if your authentication has timed out by the time you get around to killing it.

#!/bin/bash

function cleanup()
{
    sudo kill $job_pid
    wait $job_pid
    exit 0
}

trap cleanup SIGTERM
trap cleanup SIGINT

sudo long_running_foreground_process &
job_pid=$!
wait

Using Python’s ctypes to Call Into C Libraries

The ctypes module makes loading and calling into a dynamic library incredibly easy:

>>> from ctypes import CDLL
>>> libc = CDLL('libc.so.6')
>>> print libc.strlen('abcde')
5

As with everything else in python, it gets even better when you scratch the surface. In the example above, CDLL returns an object that represents the dynamic library. You can access the functions in that library by attribute access (“libc.strlen”) or item access (“libc['strlen']“). Both access mechanisms return a callable object.

This callable object has an “errcheck” attribute that can be assigned a callable. We can use this for error-checking our calls into the library. Let’s write a simple version of the “kill” command that uses the kill(2) system call.

import sys
from ctypes import *

# Load the library.
libc = CDLL('libc.so.6')

# Our error checking function. This will receive the
# return value of the library function, the function that
# was called, and the arguments passed to the function as a
# tuple.
def kill_errcheck(retval, func, funcargs):
    '''Check for error -- retval == -1.'''
    if retval < 0:
        raise Exception('kill%s failed' % (funcargs, ))
    return True

# Get the kill function from the standard library.
kill = libc.kill

# Set the error checker for kill().
kill.errcheck = kill_errcheck

# Pass the command line argument as a pid to kill, with
# SIGSEGV (11).
pid = int(sys.argv[1])
kill(pid, 11)

Save this as kill.py. Then, in your shell, try something like this:

# Notice that the 3401 is the pid of the process
# we're putting into the background. Yours will
# be different.
bash$ sleep 120&
[1] 3401
bash$ python kill.py 3401
[1]+ Segmentation Fault         sleep 120
bash$ python kill.py 3401
Traceback (most recent call last):
  File "kill.py", line 17, in
    kill(pid, 11)
  File "kill.py", line 10, in kill_errcheck
    raise Exception('kill%s failed' % (funcargs, ))
Exception: kill(3401, 11) failed

At line 4 of the output we run sleep in the background. At line 5 we learn the pid of this process. At line 6 we run our kill program, giving it the pid we just spawned, and we see the notification from bash that the process was killed (with signal 11, segmentation fault). At line 8 we run our kill program again on pid 3401, but it doesn’t exist, the kill system call returns -1, and our error checker raises an exception when it detects the system call failure.

But wait, there’s more… I’m working on a follow up post that combines ctypes.Structure with calls into a linux system call.