Rusty's Blog

A couple of notes, This is to do tethering using an Android G1 phone. This should work with all Android based devices that have Debugging over USB enabled, and a network stack that the Tetherbot app can talk to. These instructions are focused on the Ubuntu release of Linux.

First, I’m not talking about full tethering here. To do that requires root access. To get root access you have to install a kernel that is not distributed by T-Mobile. If you are using a device that does give you root access, there are other solutions available.

This Tethering solution basically puts together a socks proxy that you can use to surf the web with your G1 phone anywhere you can get a good cell signal. It also works perfectly well for a local wifi connection (not between the computer and the phone, but say between the phone and your home wifi AP, or at a coffee shop.) For the most part that’s great for testing, but I suspect that if you have a modern laptop with good USB 2.0 ports, that you either have a wifi adapter already you can plug in, or your laptop came with a WiFi connection you can probably use. If you are having problems with some proprietary drivers, this may be a temporary workaround, but you’ve just added a $200 wifi adapter to your kit. Seems a bit extravagant. Doesn’t it?

Start by following the instructions over at http://graha.ms/androidproxy/

If the instructions work, you’re done. I had to do a bit more.

In my case I ended up with the following problem:

./adb devices
List of devices attached
???????????? no permissions


First you need to have a file named adb_usb.ini in your ~/.android/ folder. You can create it with the command

echo "0x0e79" >> ~/.android/adb_usb.ini

As much as I would like to say that the next step is a drop kick, it’s a little bit more detailed. You may not need this. If after doing the above command you see a device identified by ‘./adb devices’ you’re done. However experience suggests that you will need to create a /etc/udev/rules.d/51-android.rules file with the usb maunfacturer ID for your phone included.

To get the manufacturer ID, run lsusb twice, once with your phone not attached to your computer, the other with it connected. The line that is added with the phone there should look something like:

Bus 002 Device 003: ID 0bb4:0c02 High Tech Computer Corp.


Obviously Bus and Device will depend upon which port it’s plugged into. The ID is in two parts, the part to the left of the colon is the manufacturer ID, the other is a per manufacturer device ID, but for what we are doing is not critical. So for the Android G1 phone, the part of the ID that we are interested in here is ‘0bb4′

The rules file needs to contain the following:

SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666"


Once the file has been created, I’ll leave it to you how, but

sudo echo SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666" > /etc/udev/rules.d/51-android.rules

should do the trick, you need to set up the permissions properly, and restart the udev process:

sudo chmod a+rx /etc/udev/rules.d/50-android.rules
sudo /etc/init.d/udev restart


Personally I recommend doing the above with the phone detached (except when testing with adb) and once udev has been restarted plugging your phone into your computer again.

This time when you run ./adb devices you should see something like:

./adb devices
List of devices attached
HT???GZ????? device


The specific numbers have been replaced with ‘?’ but if you see something beginning with HT, you’re probably well on your way.

I ran into one last issue with FireFox. It may have been the fault of the platform, but if you run into it this is the cure.

Basically once I had set all the above up, when I disconnected my lan connection, I would get a ’server not found’ error from Firefox when attempting to navigate anywhere. The problem is that by default firefox will continue to do dns lookups through the OS DNS lookup resolver. Since that requires a network connection and on it’s own doesn’t recognize a Socks 5 connection as one, the connection doesn’t work.

So the fix? Install the FoxyProxy add on, and create an entry to browse using the proxy for all traffic. One of the check boxes is to do dns lookups through the proxy as well. Make sure it is checked.

Now go back to the instructions at http://graha.ms/androidproxy/ and you should be on your way.

Hope this is useful to someone besides me. Most of this page was written with a local lan connection. However final edits, and posting to the public occurred over a functioning foxyproxy tethered connection. It will work, it’s just a bit of a step by step process to get it going.

So you’re out and about. Having fun with your G1 (or other android phone) you’ve taken some pictures and you decide that right now you want to take one of thoe pictures and use it as the wallpaper on your laptop.

Well, you can e-mail it to yourself. Or you could post it on your picasaweb account, or well “neither of those really interest me.” you say? Well, if you had been thinking this morning when you left the house you probably would have packed along a usb cable to like up your computer to your phone, but if you are like most other people, you’ve decided that that really isn’t something you want to carry today, and you can always take care of that tonight. Or perhaps you have one of those micro sd-card to usb adapters on your keychain? No. Bummer.

Wouldn’t it be great if you could just fire up a small web server that points at your photo album? Point your web browser on your laptop at it and download the picture you want?

Actually if you have installed the Android Scripting Engine (ASE) you can.

ASE includes interpreters for Bean Shell, (a Java based shell scripting language) Lau, (found in some Linux distributions) Python, (pretty much everywhere these days) bash, (also pretty common) and more to come. Now I’ll be one of the first to admit that all of these are probably anything but feature complete. If you are thinking ‘Hey I’m going to write a push button app with custom lables using tk!’ sorry, tk is not included. However if you are interested in talking to the network, get information from features of the phone such as the location via gps, or the orientation of the phone (face up/down, on the side, etc) you can do some pretty interesting things. In time the screen may be dot addressable, and you may be able to do gui stuff, but it’s not currently being planned.

I’ve not done anything with Bean Shell, and not all that much with Lau. Both have fans and from what I have seen are likely to be used fairly heavily. However I do like to dabble with Python, so you’ll find most of my blog postings here relating to ASE are going to focus on Python.

But we’ve got that photo to get to the laptop. Right? Lets work on that.

Perhaps you haven’t installed ASE but that might work for you? Poing your browser at http://code.google.com/p/android-scripting/ and you should see a QRcode bar code on the page. If you have the BarcodeScanner application istalled fire it up on your phone, point the camera at the bar code on the screen, and you’ll end up in the application installer looking at the install helper for ASE.There are additional instructions that you will want to follow to add in the actual python interpreter as well as some sample scripts.

Now since the 0.10 alpha release of ASE, not only can you download the application itself via qrcodes, but you can enter small scripts that way as well. Launch ASE, hit ‘Menu’, pick ‘Add Script’ and select ‘Scan Barcode.’ The barcode reader will come up again, point it at the following qrcode based on the script posted by Mrflakey to the ASE-Scripting mail list:

httpd.py

You’ll actually be able to run it immediately, however it’s always a good idea to take a look at the code. It should look something like this:

httpd.py

import SimpleHTTPServer

from os import chdir

chdir('/sdcard/')
SimpleHTTPServer.test()

If all you wan to do is browse your pictures, then you will probably want to change the chdir line to read something a bit more like “chdir(‘/sdcard/dcim/Camera’)”

Now there is only one ‘issue’ left. “How do I get to the server?” Well, to tell the truth, that has been the real reason for this posting.

<hr>

The first time I used the script and downloaded content from my phone, the way I found the IP address for my phone was to open a terminal session on my dhcp server and looked for the message indicating that my phone had just been assigned an address via dhcp. That’s all well and good for when I am at home, but if you are out on the road, you probably don’t have that ‘advantage.’ Last I checked T-Mobile wasn’t exactly giving me access to their system at a level where I could get the IP address that my phone was using from their servers. Not really expecting that any time soon either.

If you dig around, you can find the ip address, but wouldn’t it be nice if you could present it as part of that httpd.py script? Actually, you can. But it’s a bit more work.

<hr>

The Android platform allows you to see the current IP addresses on your phone by using the command netcfg at the command prompt. You already know about 127.0.0.1. If you don’t a brief note, don’t bother to try to download anything from your phone by pointing your web browser on your laptop at 127.0.0.1:8000, it won’t work.

There are 6 different ‘interfaces’ that may have an IP address on your phone.:

$ $netcfg

lo      UP      127.0.0.1       255.0.0.0       0x00000049

dummy0  DOWN    0.0.0.0         0.0.0.0         0x00000082

rmnet0  DOWN    25.96.178.24    255.255.255.252 0x00001002

rmnet1  DOWN    0.0.0.0         0.0.0.0         0x00001002

rmnet2  DOWN    0.0.0.0         0.0.0.0         0x00001002

tiwlan0 UP      192.168.0.2     255.255.255.0   0x00001043

The above is something ‘like’ what is deisplayed on my phone. Presumption being that I’m at an internet cafe with wifi access and I’ve gotten both the cell phone and my laptop on the wifi, I would simply point my web browser at http://192.169.0.2:8000/ after launching httpd.py on my phone and I would be up and running. (caveat the internet cafe or wifi provider has to have enabled the ability of wirelessly connected customers to communicate directly with each other.)

If rmnet0 was UP rather than DOWN, and unless you have a rooted phone or a more recent phone than the G1 you will find tiwlan0 will thus be DOWN, and the internet cafe allows internet browsing, I could point my browser at http://25.96.178.24:8000/ and I would be able to browse the content as well.

On the other hand that’s a couple of extra steps. First you get the IP address of your phone, then you launch the server, then you hope you remember what the IP address was so you can point your laptop at the address. Annoying right?

Well we’re going to add two bits of content to the httpd.py script in a bit, one imports the library we are going to write, the other displays what our IP address is. But in order todo that we need to add a script to the phone. Let’s call the script ‘netip.py’.It’s going to have to do a few things.

• Get the list of ip addresses and interfaces from the phone. We’ll call the program netcfg to get that.
• ignore the ‘lo’ interface.
• return the interface and ip address of the UP interface(s) or
• return the ip address of the UP interface(s)
• be able to show that the library functions.

There are some contitions that over the long term you will want to address as well, but I’m not going to for this script. What do you do when you are in a place without cell service or wifi? Currently the script does not address that.

Ok, we are going to import the following libraries.

import os, sys, android, string, time

os and sys allow us to talk to the operating system, android because the script will want to be able to show us things like what IP address we got using the makeToast() android call. The time librarry we are using to allow us to display two or more results with a delay between them. We are going to be using the string library to manipulate the output of the system call to netcfg. but we better get that first:

def colint():
    status = []
    a = os.popen('netcfg','r')
    while 1:
      line = a.readlines()
      if line:
        status.append(line)
      else:
        return status
        break
    return status

Ok, status is set to an empty list. the os.popen() call tells the phone to run the netcfg program and we are going to ‘read’ the results from the system call. Start a while loop. ‘1′ is always ‘true’ so we’ll use that for the test. In the loop we assign ‘line’ to the result of doing a readline on a. Now if the result of that is true, i.e. we got something other then an end of file mark we append the content of line as a new list entry to status. if we don’t get anything from the readline we return the status and break out of the while loop. The final ‘return status’ is not necessarily required, but I would rather have the function return a blank status if the while fails for some reason. Up to you.

Of cours we’re really only interesetd in entries that have a status fo UP right? so let’s get rid of the rest.

def upips(work):

    status = []

    a = 0

    while a < len(work[0]):

      if "UP" in work[0][a]:

        status.append(work[0][a])

      a += 1

    return status

This time we call the function with a list and presumably in that list there are some entries that have the string ‘UP’ in them.

And of course we are not interested in the ‘lo’ interface so we’ll clean that out:

def nolo(work):

# given a list of strings (with embeded '\n') break as needed, return

# list of strings w/o 'lo' in them)

  status = []

  a = 0

  while a < len(work):

    if "lo" in work[a]:

      a += 1

    else:

      status.append(work[a])

      a += 1

  if len(status) == 1:

    rets = status[0]

    statusl = rets.split()

    status = statusl[2]

  return status

If there is only one interface with an up status (other than lo) we also break up the string and only return the IP address. That’s done by calling the ’split’ string function and assigning the resulting list to ’status1′ then status will get the third entry (first is at 0 and is the interface name, ‘lo’, ‘tiwlan0′, etc. the second at 1 will be the up/down state.) Otherwise if it looks like there are multiple interfaces up besides the lo interface we will return a list of the entire strings.

The rest of this can either be considered ‘test’ code, or you can call the next to functions directly. The labels are a bit misleading. At an early stage in development these functions actually displayed the IP addresses. I decided later on that I really just wanted them to return them to the calling function. At some point you may be interested in binding to the up interface. The two functions are displayUp() and displayNoLo() which returns all up interfaces, or all but the lo interface. You will probably be most interested in the displayNoLo() function, but who knows.

def displayUp():

  nets = colint()

  nets = upips(nets)

  return nets

def displayNoLo():

  nets = colint()

  nets = upips(nets)

  nets = nolo(nets)

  return nets

Now then it’s always a good idea to include in a script some code to run to make sure that it all works. Since each of the two functions above will call the earlier functions we really only need to call those two functions. However we are going to make use of the makeToast() android call so we need ot initialize android. Also makeToast() displays a popup for about 2 seconds so let’s give it a chance to complete between calls to it, and exiting the program.

if __name__ == '__main__':

  droid = android.Android()

  droid.makeToast(displayUp())

  time.sleep(5)

  droid.makeToast(displayNoLo())

  time.sleep(5)

  droid.exit()

The ‘if’ conditional at the top is the first line that will be run if you execute the program directly. I personally recommend including all of this. At the very least you will be able to look at how various functions can be called later on.

Ok, if you have a clean camera lens, you may be able to load the entire program via qrcode. Again launch ASE, ‘Add Script’ from the Menu button, and ‘From Barcode’ Once that is up, point the camer at the screen and see if it will read the following bar code:

netip.py

And a modified httpd that uses the code is at:

httpd2.py

If you would rather, the full text of the scripts are at http://www.beresourceful.net/~rusty/android/netip.py and http://www.beresourceful.net/~rusty/android/httpd2.py

Now if you have an available network connection and access to the internet, you should never be unable to get documents off your sdcard for your Android based cell phone.

Once you run a program that imports a python library you will end up with both a .py and a .pyc of that file in your list of scripts. The .pyc edition can be run the same as the .py, but is a ‘compiled’ version of the program that speeds up execution of library system calls in the future.

You may wish to create a ’shared’ folder on your sdcard that you point httpd2.py at instead of pointing at the entire card. Or as noted earlier just point at your /sdcard/dcim/Camera/ folder for images only. The reason for that is that there are no restrictions on who can download content from your phone while httpd2.py is running, except for the limitation that it is in a subdirectory of the current directory. You may also want to change the print out to read more like ‘print “connect to http://”, netip.displayNoLo(), “:8000/”‘ to give you a reminder that it is on port 8000 and that you use a web browser.  If 8000 is entirely too likely for someone to connect to, you can point SimpleHTTPServer.test() at a different port. I’ll leave learning how to do that to you. If you don’t have a rooted phone, you can’t point at anything below 1023, but you could point it at 58080 if that works better for your mental image.

Enjoy, and play around.

Ok, first of all DroidTrack probably won’t be the final name for this collection of scripts. It would not surprise mi if someone els already had the name. That’s OK, we’ll work around that as the issue crops up.

This article will discuss a couple of methods for capturing location information and making it available.

If you went through college before Java became the popular Comp Sci language for writing programs in, you probably encountered Modula-II or earlier Pascal. Both are highly structured languages used to help teach structured programming. Lau has some similarities with Pascal. There are some good references out there, some of which have worked for me in the past.

I tend to write more in Python these days, and what I will present in a bit is more along those lines, but to show that most of this can be done in either Python or Lau, lets start with a sample of Lau.

require “android”
android.startLocating(“course”) –Obtain location from network.

android.sleep(5)  –Give the location sensor a moment to come online.

OK, starting with the first line. This is needed on every application that is going to interface with the Android phone.

The second line starts the locating intent. In othe words it tells the Android phone to start asking the network or the GPS receiver where the phone is and be ready to report it.

It takes a little bit of time for the service to kick in. For ‘network’ location that is because we are asking the cell phone towers to approximate where we are, and they simply take some time to get back to us. For the GPS receiver it is a bit more complicated. First the GPS receiver needs to determine what time it is. Oh, not in minutes and seconds, but at a much finer resolution. So it listens for GPS satelites and based on the time stamps that they are providing, determines approximately where all of them are relative to where the receiver is. That initial estimate gets the phone to within a certain distance of itself. Let’s estimate that to be the better part of a mile. Not really much help when trying to put us on a map, but it’s a start. Now that we have that estimate, the system improves the accuracy of the clock and gets a much tighter resolution of where it is on the globe (or above or below it.) This goes back and forth a few times. Ultimately using just this technique the GPS can estimate where it is to within about 80 feet. Well, that could put me on either side of this freeway. A major reason for this level of inaccuracy is that satelites do not have perfectly maintained orbits. So while we know that a sat is in orbit, it has a limited amount of accuracy involved as well. It may be hundreds of feet away from it’s predicted location.

Since we do like a little bit better resolution than that, they had to come up with a better way of getting an idea of where the satelites were so that we can get a better idea of where we are. The standard method of doing this is to use two GPS receivers, one at a fixed known location, that tells the mobile receiver what the inaccuracies are that the satellites are giving us. The problem is that this requires either a lot of extra hardware to get the two GPS receivers to talk to each other, it also requires that both GPS receivers work their location information out from the same satelites. That is not a given. Likewise this really only works when both receivers are seeing roughly the same sky. If you are over the horizon or around the world, you will probably see something different, in addition to having a bit of difficulty getting the current variation information.

The next best thing is to use a variation of this built into the GPS system. There are several satelites in orbit that provide ‘enhanced’ information to receivers that are designed to receive them. The enhanced information is part of a feature called WAAS – Wide Area Augmentation System. There are a collection of GPS receivers located on known locations that collect drift calculations for the satelites they ’see’. This includes clock drift, and the like. That information is collected and sent to stations on both east and west coasts of the US, where the information is crunched and new calculations are made regarding where each satelite is in it’s orbit, as well as how far off from the standard time each clock actually is. A package of that information is sent to the enhanced capability satelites that then include it in their data stream. A WAAS enabled GPS receiver then uses that information to make improved calculations regarding it’s location. At best this gives a GPS receiver the ability to get their location to within a 2 foot circle, though on average 16 feet seems to be more common. (I’m at 45 degrees north, My experience is that the further south I get the better the accuracy is, and may get down to 5 feet or so near the equator, at which point I would suspect the resolution gets worse again.)

The big problem with GPS is that it requires a reasonably clear view of the sky. If you are in a building or in a metro area with lots of tall buildings. (Hey being in a canyon has an effect as well.) you can expect that GPS will not get you a usable location. As a result the Android phone, and most other 2nd generation and beyond phones also collect location information based on cell towers. Here the situation is a little bit different. Since the cell towers are not in orbit, and the phone is in contact with at least one, it can get an estimate of where it is by knowing what tower it is talking to, and the tower telling everyone where it is located. With 3G phones the phones are talking to multiple cell phone towers and can get an improved estimate of where it is by recognizin gthat it is probably somewhere between the cell phone towers that it can talk with. Also the phone can get some information based on how fast it gets a response from a tower. That said the accuracy is significantly lower than you see with good visibility of GPS satelites. However if I can figure out what block you are on, or what building you may be in, it makes getting emergency services to you that much easier.

So we’ve told the phone to start gathering location information, and we’ve put the application to sleep for 5 seconds for the phone to work out where it is. Let’s start a loop of what we want to do. That begins with ‘While True Do” and ends with”‘End”. (Purists will say that’s not like Pascal, where’s the ‘Begin’? I didn’t say it was pascal, just similar.)

Within the loop we need to do four things. Get the current location information, format what we want to present to user, present it, and go to sleep for some period of time. “l = android.readLocation()” does the first thing, putting it into a variable ‘l’. ‘l’ is a data structure with several blocks of information. We’re primarily interested in our longitude and latitude, which places us on the globe. In some cases we may be interested in our Altitude as well. So we’re going to construct a string: {lat ” .. l.result.latitude .. “\nlon ” .. l.result.longitude .. “\nalt ”  .. l.result.altitude} and wrap that up with a command to present it to the user, android.makeToast(). Finally we are going to let the script sleep again for 5 seconds.

Thank you to Shanjaq for the Lau script above. Lau also will alow you to ‘print’ the output. So if you want the output within the shell window, rather than as a pop-up that may obstruct whatever else you may be doing, you could replace ‘android.makeToast()” with “print()”.

As I say, I don’t know all that much Lau, and while it very likely is simple to edit and append files, I hadn’t originally started down that line. I tend to work in Pascal. ASE also support Ruby and BeanShell (not to be confused with BournShell or BournAgainShell) and both may also have similar capabilities.

So my script looks a bit different:

import android, string, time

droid = android.Android()

droid.startLocating()

time.sleep(10)

while 1:

  l = droid.readLocation()

  outstr = str(l['result']['longitude']) + ',' + strl['result']['latitude'])  + ',' + str(l['result']['altitude'])

  if l['result']['provider'] == 'network':

    droidfile = '/sdcard/droidtrack.cell'

  else:

    droidfile = '/sdcard/droidtrack.gps'

  fh = open(droidfile,'a')

  res= fh.write(outstr +'\n' )

  res = fh.close()

  time.sleep(10)

Note that you can see what the labels of each component of the data structure that droid.readLocation() provides by simply printing the result. so if ‘l = droid.readLocation()’ then you will get all the labels by doing ‘print l’ You may wish to modify the output based on what is in each portion. So for example I decided that it was important to distinguish between gps provided output and cell phone tower results. Cell phone provided results are taged with provider having a value of ‘network’. Since there are really only two ways that location information will be provided, that means that the alternative is ‘gps’.

Ok, there is also the possibility that all location information will be ‘wrong’ or ‘missing’. For the purposes of this script I’m not going to worry about those possibilities.

What I am looking for however is output in a csv format. Thats because I will be wrapping it up in a KML wrapper.

If you have been reading my earlier blogs, you might notice that you can also use resources on the internet to track this information. I can very likely make this script significantly more complex if I want to use those resources. As an example I can use xmpp to report my position to a jabber server where the information can update a “I’m located ‘here’” page if I wanted to do that. I could update the information with whether I was walking, riding the motorcycle, bicycle, do I have a radio with me, etc based on other parameters.

In APRS which is Amature Radio Positionin Reporting System, there is also a ’smart beacon’ mode available. Currently this script tracks my position every 10 seconds, whether I’m traveling or not. That’s not quite enough data for OpenMap, but probably far more information than I need for APRS. One of the things to remember with APRS is that it is shared radio spectrum. Try not to monopolize it. Since there is more information I can work with here we can use some of the same rules that APRS uses. If I am stationary or moving at less than 5 miles per hour, a beaon every 30 minutes is probably sufficient. If I am moving from 5 miles per hour to 40 miles per hour, let’s give a position report every 10 min. Over 40 miles per hour once every 2 minutes. Since we also can see when those rates change, let’s grab a report any time we come to a full stop. Finally we probably want to know when our course changes. So if we see our bearing change by more than 15 degrees since the last time we had a bearing, we will send a report as well. If we want to be pedantic, we can also supress new bearing updates until we see a change in bearin gof less than 14 degres. That way we won’t send a report every second while we are navigating three right turns on the cloverleaf because we missed the right turn we needed to take earlier. It’s a feature we can add later.

So we start with the ’speed’ option. This will give us a frequency of data collection of 30 min, 10 min or 2 min. :

getRate(speed):

if speed < 5:

rate=1800

else:

if speed < 40:

rate = 600

else:
rate = 120

return rate

Now we need a couple of functions to deal with major changes in speed, direction, etc. Conditions where we want to report new position information If speed crosses one of the thresholds for changing reporting, changes by more than 20 mph since the last report, or we come to a complete stop.

change_in_speed(old, new):

beacon = false

if (old = 0 and new <> 0) or (new = 0 and old <> 0) or (abs(old – new) > 20) or (old < 5 < new) or (old < 40 < new) or (new < 5 < old) or (new < 40 < old):

beacon = true

return beacon

How about a function to deal with change in bearing. This is a bit more interesting. First of all bearing is a number from 0 to 359. To do this we will set a ‘greater’ and a ‘lesser’ to compare. We then subtract the lesser from the greater. If the result is greater than 15, or less than 345 it’s time to beacon.

change_in_bearing(old,new)

if old < new:

lesser = old

greater = new

else:

lesser = new

greater = old

difference = greater – lesser

beacon = false

if (15 < difference < 345):

beacon = true

return beacon

Ok, as part of the loop we will set a counter, such that every second it is decremented by 1, we’ll start with 0 and any time we have a value of 0 we will send a beacon. So let’s use the ‘beacon_now’ variable to store this value in.

beacon_now = 0

while true:

l = droid.readLocation()

if beacon_now == 0:

Send_position_report(l['result']['longitude'], l['result']['latitude'] , l['result']['altitude'], l['result']['speed'], l['result']['bearing'])

beacon_now = getRate(l['result']['speed'])

oldPosition = l;  # Note that we only set this if we have had to beacon.

if change_in_speed(oldPosition['result']['speed'], l['result']['speed']):

beacon_now = 1

if change_in_bearing(oldPosition['result']['bearing'], l['result']['bearing']):

beacon_now = 1

beacon_now -= 1

time.sleep(1)

Now all we have to do is write the ‘Send_Position_Report()’ function. For that we can simply format a string and dump it to a text file. send an xmpp message to a tracking account, or update a twitter page.

I have taken a few liberties with the syntax for test in the change_in_speed() function. I will post an updated script as a comment. One thing I haven’t checked is how long it will take to run through the functions or comparisons. I suspect that if ASE converts this to bytecode that the response time should not be bad. Additionally I would suspect that some people will be looking for different threshods and rates. If you are a runner, you probably would not be all that pleased to get your rate of speed and location every 10 minutes. So perhaps you would rather set the rate to 120 for any speed over 5 mph. If you are a pilot, perhaps you want to set a threshold of 30 mph for ground speed, 200 for takeoff/landings and above that for cruising, with changes in speed of 50 mph to trigger track collection. You also may not be interested in getting samples as often at cruising speed as during takeoff and landings.

Ok, I’ve gone a bit affield of just a simple demonstration of how to get your location information to display on your phone. You may be looking to take this in an entirely different direction. If so, have a good time.

Minor setup stuff. I use a G1 phone from T-Mobile and have very much enjoyed the experience. One of the things that I’ve been looing for since I started using it has been a way to post microblogs to http://identi.ca. There are a couple of apps out there for twitter users, and while I do use twitter, it’s had a few problems that make it unsutible in my view for ongoing use.

There really are several different ways of getting content out of identi.ca onto my phone. sms messages, e-mail, web browser, gtalk/xmpp, etc. However I really didn’t want to be reading messages from identi.ca, I’m a bit more interested in sending messages to there.

For me a critical point is that I have a jabber/xmpp server running on one of my own servers. As a matter of fact I had already linked my jabber account there. If you have a gmail account, you might prefer to use that. However a pointer there is that any time you receive an IM at your gmail gtalk account, it very well may be using an sms message to talk to your phone. That can get expensive. I recommend using a server of your own, or getting an account on jabber.org or a public server and use that resource for sending and receiving jabber to identi.ca messages.

Now getting a jabber message from a phone to a jabber server is not really difficult. I suspect that I could install a development package on my desktop, write a bit of code for the android environment, compile a package, put it on a server I have access to, install the package to my phone, test, and repeat the steps from write a bit of code on, until testing gives me the results I expect.

An alternative that has just recently shown up is the Android Scripting Environment. http://code.google.com/p/android-scripting/

With the Android Scripting Environment (ASE) you can write scripts in Bean Shell, Lua, or Python. By default Bean Shell is installed, but you will need to install the interpreters for Lua and Python. The Environment is a .8 code base, which means it is very functional, but may not be complete from the perspective of a general user.

I’ve personally never done well in Java, and Bean Shell really is a scripting environment for Java. If you happen to like Java, it may be all you need. I’ve also never worked with Lua, but from what I see in the collection of scripts that are available as examples, it appears to be a reasonable language as well. I have played around a bit with Python, and while the environment is hardly complete, it seems to be sufficient for what I need. For android, it does include xmpp support, as well as support for pulling down URLs, interpreting html, and a few other elements as well. All I needed this time was the Android support and the xmpp support.

I’ve liberally copied code from a couple of different resources. For an example of how to send a message to a jabber server, (which was not written for the android platform, but it’s python and all the referenced libraries are included in ASE) I used the tutorial at http://snippets.dzone.com/posts/show/618

Here’s the sample:

import android
import xmpp

droid = android.Android()

# User interaction stuff
message = droid.getInput('identi.ca update', "What's Up?")['result']
tag = droid.getInput('tags?',"What tags? (Prepend w/#)")['result']

# Varios Variables we will use
_SERER = 'jabber.server.you.use', 5223   # put in your own jabber server name and appropriate port
username = 'your.user.name'              # give it the user name you want to use
password = 'your.password'               # until I figure out how to store these credentials securely
fullmessage = tag + message              # build the message we're going to send. (could use improvement)
destid = 'somename@jabber.server'        # for identi.ca use 'update@identi.ca'

# send the message
cnx = xmpp.Client('jabber.server.you.use') # don't need the port so don't use when creating a connection instance to use
cnx.connect( server=(_SERVER) )          # establish a connection
cnx.auth(username,password, 'G1')        # authenticate yourself, and give a 'presence' (G1 in this case)
cnx.send( xmpp.Message( destid, fullmessage ) ) # Send the message

# exit 'cleanly'
quit = droid.exit()                      # we're done, time to go

And that pretty much summs it up. There are a few things I will be investigating over time. I suspect that using a dictionary, I can create a set of ‘tags’ that I want to use regularly (#cvg2009, #c25k, etc.) then feed that to some sort of a pick list, and hopefully I could enter the update and select the appropriate tag all in one dialog. Worst case, 2 dialogs sort of like what I’m doing now. This currently has a bit more flexibility, but in the end I would also like to be able to identify if this is to be a message to update, or to some other specific user.

What am I doing with this? Well, as a department co-head for the 2009 CONvergence convention, I wanted a way to update my facebook, twitter, identi.ca and planet pages with what I’m doing, and what’s happening now. I’m pretty sure that I can update twitter directly, and possibly through that facebook, and it looks like I could import from there to identi.ca, which also would propogate to my world page, but after seeing the problems that twitter has been going through with it’s growing pains, I didn’t think that was the best route. If it works for you, ok. The facebook app for Android has been having problems since facebook changed something recently, so that was out. And it’s been a while since I wrote anything I considered interesting like this, so here it is.

How things work from here. My jabber server accepts a connection and forwards the message to update@identi.ca, which then places the update in my stream. Through the facebook interface it posts an update to facebook. I also have my account configured so that all events are posted to twitter as well. My planet page pulls any updates I’ve made in the past couple of hourse every 2 hours from twitter and identi.ca. Hey it pulls an update that I’ve posted a blog as well. Handy little page at times.

I always seem to discover ‘issues’ that need to be corrected that have almost nothing to do with the specific project at hand, except that they got in the way. Today I discovered I had not punched a hole for tcp through my firewall for jabber clients. I had punched one for UDP, but since nothing is listening for that…

Have fun. If you find the app useful, and want to let me know about updates, I would appreciate hearing about them.

For purposes of this entry, social media will be any resource that is suceptible to meme’s. The obvious ones are sites like Facebook, MySpace, Twitter, identi.ca,  and the tools that connect them with other systems, such as iPhone apps, etc. However we should not discard ‘mem’ related systems either. Where’s George for example, as they can provide independent corroberating evidence.

At one level or another the CDC already tracks communicable deseases such as HIV, Ebola, and variations on the Rhinovirus and Influencza. The standard means of tracking is to get reports from medical professionals regarding when the first symptoms of a communicable disease enters a community, and what the experience within that community is for propegation within the community. As best they can they try to determine the vector, but that is not always as simple as it seems. An example of a disese that we know is communicable, but do not know well how it enters into a ommunity is the Ebola virus. The last I read we had a strong suspicion that it exists and is innocuous within some animal, though we do not know well which one, and we only have theories as to how it first transitions to humans where it has a malevolent response.

As a result of the reporting that the CDC is already getting, we can see patterns in propegation, and hopefully get doctors and the medical communities prepared for the influx of paitents they will see, though accurately predicting when ior even if it will hit a community are still rather dicy.

The  big problem a tthe moment is that asside from self reporting of social contacts within a medical report, which is rarely going to give you useful information, knowing where a bug is only peripherally tells you where it is going. Will it arrive by plane tomorrow? On the train later this week? Or will it suddenly appera within one ethnic community and completely miss everyone else for a couple of weeks?

The Holy grail of prevention is being able to identify accurately the vectors. If you tried to explain how a disese got from one place to another based on medical reports, you would very likely see a pattern of early reports showing up first around clinics and hospitals, so those are ‘likely’ to be the vectors. Right? Maybe. What you would really like however is a way of tracking how a disese is propegating without it being bound to a single reporting mechnism. Ideally it would have a way of identifying relationships between people within the reporting system that already is in place. And that is where social media could come in.

You may have seen reports about, or even participated in ‘memes’ such as the ‘25 things about me’ or ‘1 word’ memes that have hit several social media platforms over the past few months. If you look at what happens, someone somewhere inserted the meme from one system into another. Perhaps they saw a blog entry on a blog agregator, and decided that it would be fun to try that on Facebook where they could tag their friends. And their friends tag others, and after a few itterations you litterally have millions of people posting inane facts about themselves, and a the meme dies away. Or perhaps someone from Facebook takes the meme over to MySpace, and it flares up again.

If you watch the pattern you see that the first one is extreamly virulent, and the ones following are less so. 25 things hits millions, one word hits hundreds of thousands, ‘alphabet’ may have gotten to tens of thousands. Which is also of interest in contemplation of how the various propegation mechanisms can adjust over time. There was no imunity to the first meme, but the second time around people were less interested, and by the third or fourth time around people were actively opposing the meme. A primary vector becomes identified and people either block that vector or filter it against similar events.

Chain letters are another example, as are other e-mail messages with images, etc. In fact it’s been used to actively propegate computer viruses, which are another example.

Those examples however are essentially models. And they provide useful information about how vectors can work, but don’t necesarily give us useful information about how to map that information onto medical situations. Our imune systems, both internal and external have gone through several very deadly propegations, such as a couple variations on Plague, the Spanish Flu, etc. We are not fully immune to those, as we have learned that each of these do change over time.

Given that we can model diseases, can we use that information to perhaps identify vectors for new disease outbreaks? This is where Social media sites get to be very interesting, and potentially very controversial. Interesting because if you can track a real disease across a social network, see who is reporting what sort of symptoms, and see how those symptoms then show up across various relationships, you can potentially identify the sorts of vectors those diseases use. This can be useful information in helping to prevent an outbreak in the first place. As an example, if you spot that the first people affected by salmanela food poisoning all are close to supermarkets, and it seems to take a few days before people around regular grocery stores are affected, you can look at what might sell first at supermarkets, then sells later at grocery stores. If you see that people on vacation to the Camen Islands are comming down sick a week later, but people who are there on business only trips there are mostly fine, you might consider looking at resort related activities.

Social media can be broken down into thre or four ‘types’ of relationships. First of all you have direct relationships. This might be work related social sites, company/vendor type sites, or sites that cater to families. While these are handy for looking at relationships that probably involve physical contact between the members, it’s less likely that someone like the CDC is going to be able to have access to the information, and also somewhat unlikely that such information will be available as they tend to discuss specific non health related information. If you do have access, you may see patterns of propegation in how people interact however. Bob usually responds immediately, but didn’t respond to anything on Monday or Tuesday this week. Turns out he called in sick. And so on.

Another variation is the ‘completely open’ social site. For some people these are the ‘best’ types. It gives someone like Robert Scoble, or Jason Calicanus the opportunity to market to litterally millions of people at a time. Friend the person and you immediately have a way to distribute information about your business, and you are more likely to end up with customers than by just advertising on a web site.

Between the two are ‘restricted’ social sites. In some way relationships are restricted here. You may have to identify how you are ‘related,’ (Family, work, hobby, neighbors, school chum, etc.) or you may only be allowed to have a certain number of friends (Friendster, Facebook)

For the moment we won’t get into other variations on the ‘restricted’ sites. There really are too many to mention. However another type of site to consider are the ‘dating’ sites. In most cases you are going to be looking here for either budding, or transient relationships, and in some cases there will be various levels of intimacy involved that may very well give you some very useful vector information.

As noted earlier the ‘closed’ relationship sites are of limited utility to someone like the CDC. I have to admit that I think the ‘completely open’ varieties are also of very limited utility. This because while they may provide a lot of relations, it is very likely that most of those relations do not provide a mechanism of transmitting communicable diseases. One could spend thousands of hours trying to figure out how to use a collection of MySpace or Twitter relationships, only to discover that the person you are ‘tracking’ with all those relationships only ever personally interacts with a very small subset of the collection.

Limited social media then are where you really would like to focus. Here however you run into other problems. Most of the limited sites do not have a means of looking at everyone’s updates. You very well may be albe to see that Bill is married to Marry, and works fro Walt, but if you can’t view their communications, you may never see how Walt was sick on Monday, Bill ended up taking a sick day Thursday, and Mary missed going to church on Sunday.

One thing you can ddo with such a site thour is develop an application where people ’share’ information with each other. Perhaps you build an ‘I’m feeling…’ app that allows you set a ‘health’ level with detailed information. Headache, blury vision, lethargic, sneezing, coughing, racing heart, feaver (potentially with temp) muscle ache, shivering, pins and needles in feet/fingers. and so on. You set that level, and you invite your friends to use the same app. Additionally you get a related app for your cell phone, so you can set this information wherever you hapen to be, and it tags the update with where you are. Perhaps a ‘nudge’ feature to get people to tell each other ‘Hey I feel great!’ and so on, again tracking where you are when you post the update.

Since we have gone ‘cross platform’ the information has to be maintained outside of the primary social media site. This can cause some very significant privacy concerns. Especially since we are dealing with health information. In fact the tracking of such information is very tightly regulated under law. I’ll leave HIPPA concerns out there, but they are very real, and need to be designed in from the get-go.

The big problem with the privacy concerns is that unless you have some way of distributing relationship information amoungst users, you simply have another means of tracking where a disease may have been. It’s important information, but in a way it puts you in no better of a position than you started out being.

In computer science there is a concept called a graph. This is not the plotting of stocks over time, or the charting of population growth, though both are variations or a graph. A graph consists of ‘nodes’ and ‘vectors’ Vectors connect nodes, and have various properties, such as distance, direction, flow, Nodes usually have location information, but may have other characeristics. Using that concept, you can show how money flows from one person to another, Or you could build a family tree and track how genetic problems progress.

The thing to consider for this situation is that if you can watch how vectors change, for relationships, and can find some way of gathering helpful information about the people involved in the relationship without tieing identifiable information to the node. you could gather a lot of useful information for watching how an illness propegates, and provide public service information specific to that illness.

The next problem is who should track that, and can they be trusted to not gather un-necessary information? Some would suggest that this is a ‘Homeland Security’ issue. We need to be albe to prevent and protect against diseases that may affect the safety of the country and our military forces. Perhaps, but history of late suggests that this might not be a very trusted situation. USAMRID and the CDC are much more likely to be able to properly handle the information, and already have means of distributing advisories to the medical community. I would like to think that at least the CDC is more trustworthy, but that is largely dependent upon who is in charge, and can also be variable based on internal policies. USAMRID has the perception problem of being the military, with all the legacy of Military Intelegence, and the fact that over the years it has come to light that they did non-consentual comunicble disease testing on large populations in several major cities.

That means that the CDC would have to be a customer of ’scrubbed’ information. They may get to know about what types of relationships are involved in the graph, and may get demographic information about the people, but someone else has to be trusted to convert the information from the raw social site data to something that the CDC can make use of without it being personally identifieable. Some people I would strongly oppose having this power would be any medical site related to insurance. I would recommend that insurance companies should not even be allowed to sponsor, much less advertise here. Yes being ahead of the game could help reduce medical expenses, however there is entirely too great a likelyhood that people will be rejected for coverage as a result. (or worse.)

I don’t know who would make the best resource for this. Commentary welcome.

Back a few years ago, I started buying my music on CD. I happen to be old enough to remember my folks buying 8-tracks, I had a 45 player. Actually I had a 78 player, but those were not in record stores any more by the time I was born, aincient I may be, that old I am not. The 78 player was one of my grandparents opening space in their home and I was currious enough about the stuff that I got to take it home with me.

I don’t know if there is a 45 record anyplace in my collection of ’stuff’. I’m kind of hoping not. Mostly because the conditions they would have to be in are not exactly good for the records. I do know I have some LPs (33 1/3 rpm vinyl) that I’m hoping are not in truely horrid condition, though I’m not going to suggest that they have been treated the best either.

I think I bought my first CD in 1984. I was stationed in Germany, and had bought a ‘midi’ system. (Nothing to do with the Musical Instrument Digital Interface platform.) I bought a Techniques CD player at the same time, and started building a small collection of CDs. Along the way I’ve lost a few due to various incidents, and a few have just gone missing. It’s not like I haven’t had kids or something right?

In any case I’ve been amassing a bit of a collection. Nothing like some people build, and I don’t expect to be hosting a radio station really worth listening to any time in the future, but I wouldn’t want to have to try to dig through the collection to find a specific CD any time soon either.

Back in the late 90’s I picked up my first MP3 player. A 6 gig Archos hard disk based player. I had already converted a few of my CDs to a small MP3 collection as I wanted to listen to a randomized playback without having to pay a couple hundred bucks for a cd jukbox, and having to wait for disks to change, and mechanical parts to fail, etc. I was using BeOS, and the ripped collection easily fit on the Archos drive at the time.

I looked at the various file sharing applications over time. Napster included, and decided for my own reasons that I really wasn’t interested in having copies of music I wasn’t willing to buy the CD for. I know that it was not a decision everyone made, but that was mine. And I continued to buy CDs from time to time.

I bought a lot of used CDs over the years. For the better part of a decade I would visit music stores and look for a couple of CDs that had disappeared, or been broken. and replaced most of them. At one point I had ripped my collection to over 13 gigabytes of 384kbps MP3’s. I got to thinking about my listening environment, and decided that I really didn’t need my collection to be at that high a bit-rate. So I re-ripped the entire collection back down to 128k, and left it at that. Along the way I discovered that some of my CDs had been revised on the various cd databases that the rippers were using, and some which had not been there before happened to match newer albums that didn’t happen to be in the same genera. So a bit of manual updates being used from time to time.

Over the years I’ve used a variety of MP3 players. Anything from devices that look like a USB key drive that a pair of headphones plug into the side, on up to my Nokia N800 with 2 16 gig sdhc cards in it. I have had a cell phone that had a built in MP3 player (SK-3), a Samsung Nexus XM player that worked as an mp3 player as well, and more than enough software MP3 players. There are a couple of other players here and there, but for the most part I’ll be ‘done’ with the hardware players fairly soon now.

The only significant problem I have with MP3, has nothing to do with the players themselves. Though carrying around special purpose devices that only play mp3 files seems to me to be a bit of a waste of resources at the moment. Perhaps not as bad as devices that need a special headset adapter perhaps, but that’s a different matter.

Back to what my biggest problem is, it’s that the encoder’s are pattent encumbered. Well, players are as well, but I’m buying the player from a retailer or manufacturer who is paying the licence fees for that player out of the sales price. Under Linux, I’m not buying the encoders. So any MP3 encoder I end up using also happens to be in violation of the patent that is applicable. I don’t expect that I’m likely to be taken to court over it any time soon over the matter, but it is a thorn in the side. A ‘legal’ way for me to circumvent the issue would be to play the audio through a sound card, into one of my Haupauge PVR-250 cards and capture to MPEG, then strip the MP3 audio out of the MPEG file. That encoder I presume is paid for in the cost of the hardware itself. Personally I think that’s a lot of excess work that I’m not interested in doing, especially as there are other ways of accomplishing the same goal.

Remember that the Goal is to have the opportunity to listen to the music I own the CD’s for. I’m not trying to resell any of them. I’m not trying to hand out copies to all my friends, or my family. Just me, listening when and where I would like, preferably without hauling around 20 pounds of CDs.

Obviously I could install a CD Jukebox in my car, duplicate the collection of CDs I have by buying another copy of each one, and so on. You know, I don’t think that’s a reasonable solution. It doesn’t help me when I ride the bus to and from work. It doesn’t help when I’m at work, etc.

A few years back, someone else who wasn’t happy with the situation regarding the pattents related to MP3 developed a new encoder, called Ogg Vobis. (or Vorbis, not sure at the moment) There’s another solution for video called Ogg Theora. (again, I think that’s the title.) They both do a very good job of encoding and playing back music and video, however there hasn’t been a lot of hardware on the market that has support built in for it. One odity related to that is the N800. This device is built around a Linux kernel and platform. Ogg has been available for almost all varients of Linux, but Nokia decided that there was insufficient demand for Ogg to include it in the software included in the N800. Now there are some reasonable explanations for this, in that the device does have a limited amount of space, but since the device supports full motion video and more, there are people who are not particularily happy about the situation. Yes, some people have ported the necessary libraries, and it would be reasonably easy to get them to work, but for most people it’s not really worth the effort.

Last week I bought a new cell phone. I like a lot of what the iPhone has to offer, but really don’t like the idea of having to do almost all update processes through iTunes. (If for no other reason than I don’t want to fight with trying to get that to work under Linux.) However Google anounced the Android cell phone platform specification some time back, and about 2 weeks ago T-Moble started selling the HTC Dream G1 in broad distribution. It’s no more for ‘everyone’ than the iPhone is. And I don’t expect the G1 to be an iPhone killer. However it does provide significantly more service than my previous combination of cell phones, N800, and ‘disposable digital’ camera provided. Mostly because with 1 device I have taken care of all three of those resources, as well as my GPS, and a few other things that I am hoping it will work out for. It won’t be a remote head for my ham radio gear, but…

One of my co-worker’s is a musician. He just released a new CD, of which I bought a copy. I went to rip it so I could listen (again without having to cary the CD itself around) and the CD-ripping software on the computer I used didn’t support ripping to MP3. Flaac and Ogg though are both supported. Again since I’m not too concerned about compression artifacts, I chose the ogg format and once the tracks had been ripped I copied them over to my phone figuring that at worst it would work as a way of getting the tracks home.

I disconnected the phone from the computer, and pulled up the music player. Now if you guessed that with all that I wrote above I was disapointed to find that the song’s were not in the collection the player would handle, you need to re-look at the title of the posting again. Yes it did work. I wasn’t expecting it to, but the Android platform is built around a Linux kernel and they did include the ogg libraries.

So, it’s time to go back to the collection of CDs, and start ripping them again. This time to ogg. I may not be able to do all that much with the podcasts I listen to, however there are a couple that do distribute in ogg.