Rusty's Blog

Thoughts and musings of someone who's not sure what 'normal' is…

Monday, August 24, 2009

DroidTrack – A tracks collector for Android phones.

Ok, first of all DroidTrack probably won’t be the final name for this collection of scripts. It would not surprise mi if someone els already had the name. That’s OK, we’ll work around that as the issue crops up.

This article will discuss a couple of methods for capturing location information and making it available.

If you went through college before Java became the popular Comp Sci language for writing programs in, you probably encountered Modula-II or earlier Pascal. Both are highly structured languages used to help teach structured programming. Lau has some similarities with Pascal. There are some good references out there, some of which have worked for me in the past.

I tend to write more in Python these days, and what I will present in a bit is more along those lines, but to show that most of this can be done in either Python or Lau, lets start with a sample of Lau.

require “android”
android.startLocating(“course”) –Obtain location from network.

android.sleep(5)  –Give the location sensor a moment to come online.

while true do
l = android.readLocation()
android.makeToast(“lat ” .. l.result.latitude .. “\nlon ” ..
l.result.longitude .. “\nalt ”  .. l.result.altitude)
android.sleep(5)
end

OK, starting with the first line. This is needed on every application that is going to interface with the Android phone.

The second line starts the locating intent. In othe words it tells the Android phone to start asking the network or the GPS receiver where the phone is and be ready to report it.

It takes a little bit of time for the service to kick in. For ‘network’ location that is because we are asking the cell phone towers to approximate where we are, and they simply take some time to get back to us. For the GPS receiver it is a bit more complicated. First the GPS receiver needs to determine what time it is. Oh, not in minutes and seconds, but at a much finer resolution. So it listens for GPS satelites and based on the time stamps that they are providing, determines approximately where all of them are relative to where the receiver is. That initial estimate gets the phone to within a certain distance of itself. Let’s estimate that to be the better part of a mile. Not really much help when trying to put us on a map, but it’s a start. Now that we have that estimate, the system improves the accuracy of the clock and gets a much tighter resolution of where it is on the globe (or above or below it.) This goes back and forth a few times. Ultimately using just this technique the GPS can estimate where it is to within about 80 feet. Well, that could put me on either side of this freeway. A major reason for this level of inaccuracy is that satelites do not have perfectly maintained orbits. So while we know that a sat is in orbit, it has a limited amount of accuracy involved as well. It may be hundreds of feet away from it’s predicted location.

Since we do like a little bit better resolution than that, they had to come up with a better way of getting an idea of where the satelites were so that we can get a better idea of where we are. The standard method of doing this is to use two GPS receivers, one at a fixed known location, that tells the mobile receiver what the inaccuracies are that the satellites are giving us. The problem is that this requires either a lot of extra hardware to get the two GPS receivers to talk to each other, it also requires that both GPS receivers work their location information out from the same satelites. That is not a given. Likewise this really only works when both receivers are seeing roughly the same sky. If you are over the horizon or around the world, you will probably see something different, in addition to having a bit of difficulty getting the current variation information.

The next best thing is to use a variation of this built into the GPS system. There are several satelites in orbit that provide ‘enhanced’ information to receivers that are designed to receive them. The enhanced information is part of a feature called WAAS – Wide Area Augmentation System. There are a collection of GPS receivers located on known locations that collect drift calculations for the satelites they ‘see’. This includes clock drift, and the like. That information is collected and sent to stations on both east and west coasts of the US, where the information is crunched and new calculations are made regarding where each satelite is in it’s orbit, as well as how far off from the standard time each clock actually is. A package of that information is sent to the enhanced capability satelites that then include it in their data stream. A WAAS enabled GPS receiver then uses that information to make improved calculations regarding it’s location. At best this gives a GPS receiver the ability to get their location to within a 2 foot circle, though on average 16 feet seems to be more common. (I’m at 45 degrees north, My experience is that the further south I get the better the accuracy is, and may get down to 5 feet or so near the equator, at which point I would suspect the resolution gets worse again.)

The big problem with GPS is that it requires a reasonably clear view of the sky. If you are in a building or in a metro area with lots of tall buildings. (Hey being in a canyon has an effect as well.) you can expect that GPS will not get you a usable location. As a result the Android phone, and most other 2nd generation and beyond phones also collect location information based on cell towers. Here the situation is a little bit different. Since the cell towers are not in orbit, and the phone is in contact with at least one, it can get an estimate of where it is by knowing what tower it is talking to, and the tower telling everyone where it is located. With 3G phones the phones are talking to multiple cell phone towers and can get an improved estimate of where it is by recognizin gthat it is probably somewhere between the cell phone towers that it can talk with. Also the phone can get some information based on how fast it gets a response from a tower. That said the accuracy is significantly lower than you see with good visibility of GPS satelites. However if I can figure out what block you are on, or what building you may be in, it makes getting emergency services to you that much easier.

So we’ve told the phone to start gathering location information, and we’ve put the application to sleep for 5 seconds for the phone to work out where it is. Let’s start a loop of what we want to do. That begins with ‘While True Do” and ends with”‘End”. (Purists will say that’s not like Pascal, where’s the ‘Begin’? I didn’t say it was pascal, just similar.)

Within the loop we need to do four things. Get the current location information, format what we want to present to user, present it, and go to sleep for some period of time. “l = android.readLocation()” does the first thing, putting it into a variable ‘l’. ‘l’ is a data structure with several blocks of information. We’re primarily interested in our longitude and latitude, which places us on the globe. In some cases we may be interested in our Altitude as well. So we’re going to construct a string: {lat ” .. l.result.latitude .. “\nlon ” .. l.result.longitude .. “\nalt ”  .. l.result.altitude} and wrap that up with a command to present it to the user, android.makeToast(). Finally we are going to let the script sleep again for 5 seconds.

Thank you to Shanjaq for the Lau script above. Lau also will alow you to ‘print’ the output. So if you want the output within the shell window, rather than as a pop-up that may obstruct whatever else you may be doing, you could replace ‘android.makeToast()” with “print()”.

As I say, I don’t know all that much Lau, and while it very likely is simple to edit and append files, I hadn’t originally started down that line. I tend to work in Pascal. ASE also support Ruby and BeanShell (not to be confused with BournShell or BournAgainShell) and both may also have similar capabilities.

So my script looks a bit different:

import android, string, time
droid = android.Android()
droid.startLocating()
time.sleep(10)
while 1:
  l = droid.readLocation()
  outstr = str(l['result']['longitude']) + ',' + strl['result']['latitude'])  + ',' + str(l['result']['altitude'])
  if l['result']['provider'] == 'network':
    droidfile = '/sdcard/droidtrack.cell'
  else:
    droidfile = '/sdcard/droidtrack.gps'
  fh = open(droidfile,'a')
  res= fh.write(outstr +'\n' )
  res = fh.close()
  time.sleep(10)

Note that you can see what the labels of each component of the data structure that droid.readLocation() provides by simply printing the result. so if ‘l = droid.readLocation()’ then you will get all the labels by doing ‘print l’ You may wish to modify the output based on what is in each portion. So for example I decided that it was important to distinguish between gps provided output and cell phone tower results. Cell phone provided results are taged with provider having a value of ‘network’. Since there are really only two ways that location information will be provided, that means that the alternative is ‘gps’.

Ok, there is also the possibility that all location information will be ‘wrong’ or ‘missing’. For the purposes of this script I’m not going to worry about those possibilities.

What I am looking for however is output in a csv format. Thats because I will be wrapping it up in a KML wrapper.

If you have been reading my earlier blogs, you might notice that you can also use resources on the internet to track this information. I can very likely make this script significantly more complex if I want to use those resources. As an example I can use xmpp to report my position to a jabber server where the information can update a “I’m located ‘here'” page if I wanted to do that. I could update the information with whether I was walking, riding the motorcycle, bicycle, do I have a radio with me, etc based on other parameters.

In APRS which is Amature Radio Positionin Reporting System, there is also a ‘smart beacon’ mode available. Currently this script tracks my position every 10 seconds, whether I’m traveling or not. That’s not quite enough data for OpenMap, but probably far more information than I need for APRS. One of the things to remember with APRS is that it is shared radio spectrum. Try not to monopolize it. Since there is more information I can work with here we can use some of the same rules that APRS uses. If I am stationary or moving at less than 5 miles per hour, a beaon every 30 minutes is probably sufficient. If I am moving from 5 miles per hour to 40 miles per hour, let’s give a position report every 10 min. Over 40 miles per hour once every 2 minutes. Since we also can see when those rates change, let’s grab a report any time we come to a full stop. Finally we probably want to know when our course changes. So if we see our bearing change by more than 15 degrees since the last time we had a bearing, we will send a report as well. If we want to be pedantic, we can also supress new bearing updates until we see a change in bearin gof less than 14 degres. That way we won’t send a report every second while we are navigating three right turns on the cloverleaf because we missed the right turn we needed to take earlier. It’s a feature we can add later.

So we start with the ‘speed’ option. This will give us a frequency of data collection of 30 min, 10 min or 2 min. :

getRate(speed):

if speed < 5:

rate=1800

else:

if speed < 40:

rate = 600

else:
rate = 120

return rate

Now we need a couple of functions to deal with major changes in speed, direction, etc. Conditions where we want to report new position information If speed crosses one of the thresholds for changing reporting, changes by more than 20 mph since the last report, or we come to a complete stop.

change_in_speed(old, new):

beacon = false

if (old = 0 and new <> 0) or (new = 0 and old <> 0) or (abs(old – new) > 20) or (old < 5 < new) or (old < 40 < new) or (new < 5 < old) or (new < 40 < old):

beacon = true

return beacon

How about a function to deal with change in bearing. This is a bit more interesting. First of all bearing is a number from 0 to 359. To do this we will set a ‘greater’ and a ‘lesser’ to compare. We then subtract the lesser from the greater. If the result is greater than 15, or less than 345 it’s time to beacon.

change_in_bearing(old,new)

if old < new:

lesser = old

greater = new

else:

lesser = new

greater = old

difference = greater – lesser

beacon = false

if (15 < difference < 345):

beacon = true

return beacon

Ok, as part of the loop we will set a counter, such that every second it is decremented by 1, we’ll start with 0 and any time we have a value of 0 we will send a beacon. So let’s use the ‘beacon_now’ variable to store this value in.

beacon_now = 0

while true:

l = droid.readLocation()

if beacon_now == 0:

Send_position_report(l[‘result’][‘longitude’], l[‘result’][‘latitude’] , l[‘result’][‘altitude’], l[‘result’][‘speed’], l[‘result’][‘bearing’])

beacon_now = getRate(l[‘result’][‘speed’])

oldPosition = l;  # Note that we only set this if we have had to beacon.

if change_in_speed(oldPosition[‘result’][‘speed’], l[‘result’][‘speed’]):

beacon_now = 1

if change_in_bearing(oldPosition[‘result’][‘bearing’], l[‘result’][‘bearing’]):

beacon_now = 1

beacon_now -= 1

time.sleep(1)

Now all we have to do is write the ‘Send_Position_Report()’ function. For that we can simply format a string and dump it to a text file. send an xmpp message to a tracking account, or update a twitter page.

I have taken a few liberties with the syntax for test in the change_in_speed() function. I will post an updated script as a comment. One thing I haven’t checked is how long it will take to run through the functions or comparisons. I suspect that if ASE converts this to bytecode that the response time should not be bad. Additionally I would suspect that some people will be looking for different threshods and rates. If you are a runner, you probably would not be all that pleased to get your rate of speed and location every 10 minutes. So perhaps you would rather set the rate to 120 for any speed over 5 mph. If you are a pilot, perhaps you want to set a threshold of 30 mph for ground speed, 200 for takeoff/landings and above that for cruising, with changes in speed of 50 mph to trigger track collection. You also may not be interested in getting samples as often at cruising speed as during takeoff and landings.

Ok, I’ve gone a bit affield of just a simple demonstration of how to get your location information to display on your phone. You may be looking to take this in an entirely different direction. If so, have a good time.

posted by Rusty at 11:04 pm  

3 Comments

  1. Have I mentioned how much I dislike Triganometry? Bearing is not something that the GPS in the android phone reports. So to use that value, must calculate it. Ok, we start with the arctangent of the change in latitude over the change in longitude.

    Python uses radians to measure angle. Since we are primarily going to work in degrees, we need to do a conversion which is to multiply the result by 180/pi. Since pi is defined in the math library, we have to prepend it with a ‘math.’ header. Similar for the atan2 function. I’m not entirely certain how math.atan(x) is supposed to work. Is ‘y’ presumed to be 1, or something else? Is ‘x’ something other than the change in x? Fortunately I’m not too worried about that, the atan2 function suits my needs.

    Ok, we have the ‘standard angle’. how to convert that to a bearing. Well, Standard angle is measured from the x axis counter clockwise, and the bearing is measured from the y axis clockwise. To get the bearing we subtract the standard angle from 450, and then take the mod(360) of the result.

    Finally since I would rather return an integer rather than a real (long or something like that) we’ll do an ‘int’ of the result and return that.

    import math
    def getBearing(deltaLat, deltaLon):
      theta = math.atan2(deltaLat,deltaLon)
      bearing = int((450 - theta ) % 360)
      return bearing
    

    Since bearing has no meaning for a single point, we set it to 0 as an initial value. This means that for 330 or of 360 times, we will be taking a snapshot immediately after we start moving.

    I still hate trig…

    Comment by Rusty — August 25, 2009 @ 1:45 am

  2. Ok, this should work now:

    DroidTracker.py

    import android, string, time, math
    
    # Send_Position_Report() will in this case save the current position to
    # the file /sdcard/aprstrack.gps
    def Send_Position_Report(longitude, latitude, altitude, speed, bearing):
      outstr = str(longitude) + ',' + str(latitude)  + ',' + str(altitude) + ',' + str(speed) + ',' + str(bearing)
      droidfile = '/sdcard/aprstrack.gps'
      fh = open(droidfile,'a')
      res= fh.write(outstr +'\n' )
      res = fh.close()
    
    def getBearing(deltaLat, deltaLon):
      theta = math.atan2(deltaLat,deltaLon) * (180/math.pi)
      bearing = int((450 - theta ) % 360)
      return bearing
    
    # getRate() will tell us how often to do reports
    def getRate(speed):
      if speed < 5:  # 5 mph is first threshod
        rate=1800    # under 5 mph, rate will be 30 min or 1800 seconds
      else:
        if speed < 40: # 40 mph is second threshold
          rate = 600 # over 5, but under 40 we'll set the rate to 10 min
        else:
          rate = 120 # over 40 mph well set the rate at 2 min.
      return rate
    
    # change_in_speed() decides if we need to record a position because
    # our speed has changed in some important way.
    def change_in_speed(old, new):
      false = 0
      true = 1
      beacon = false
      if (old == 0 and new <> 0):
        beacon = true
      if (new == 0 and old <> 0):
        beacon = true
      if (abs(old - new) > 20):
        beacon = true
      if (old < 5 < new): 
        beacon = true
      if (old < 40) and (40 < new): 
        beacon = true
      if (new < 5) and (5 < old): 
        beacon = true
      if (new < 40) and (40 < old):
        beacon = true
      return beacon
    
    # change_in_bearing() lets us know if we have changed direction by more
    # than 15 degrees since our last bearing report.
    def change_in_bearing(old,new):
      false = 0
      true = 1
      if old < new:
        lesser = old
        greater = new
      else:
        lesser = new
        greater = old
      difference = greater - lesser
      beacon = false
      if (15 < difference < 345):
        beacon = true
      return beacon
    
    # __main__ driver program. 
    if __name__ == '__main__':
      droid = android.Android() # define the variable droid from android
      droid.startLocating()     # start the locating tools
      true = 1
      time.sleep(10)            # let startLocating do it's job
      oldPosition = droid.readLocation() # set a starting position for oldPosition (initialization variable)
      while oldPosition['result'] == None: # no position received
        time.sleep(10)
        oldPosition = droid.readLocation() # try again
      oldBearing = 0
      beacon_now = 0            # initial state of beacon_now
      while true:
        l = droid.readLocation() # get current location
        while l['result'] == None: # no position received
          time.sleep(10)
          l = droid.readLocation() # try again
        deltalon = 0 - (oldPosition['result']['longitude']-l['result']['longitude'])
        deltalat = 0 - (oldPosition['result']['latitude']-l['result']['loatitude'])
        deltalat = 0 - deltalat
        bearing = getBearing( deltalat, deltalon)
        if change_in_speed(oldPosition['result']['speed'], l['result']['speed']):
          beacon_now = 0    # we need to beacon now
        if change_in_bearing(oldBearing, bearing):
          beacon_now = 0    # we need to beacon now
        if beacon_now == 0:
          Send_Position_Report(l['result']['longitude'], l['result']['latitude'] , l['result']['altitude'], l['result']['speed'], bearing)
          beacon_now = getRate(l['result']['speed']) # change rate only if we needed to beacon
          oldPosition = l  # Note that we only set this now if we have had to beacon.
          oldBearing = bearing
        beacon_now -= 1     # decrement the beacon_now counter
        time.sleep(1)	# nap for a second.
    

    As you can probably tell there are a few differences from what I originally put together. However this should work reasonably cleanly.

    A word of caution on running things as a service. If your script generates an error and tries to write to stderr, there is no such thing within the service portion. The result is that the script will hang and you will not be able to kill it at this time. The only way I've found to kill a hung script there is to power down the phone and restart it. Not the most elegant solution.

    The script itself is also available at http://www.beresourceful.net/~rusty/Samples/DroidTrack.py

    Comment by Rusty — August 25, 2009 @ 3:51 am

  3. And if you’re wondering why you are getting updates only once per minute, and you’re seeing two reports each time, with a bearing of 90 for the second report (each time) the reason is that ASE by default (under 0.10-2 and apparently 0.11) only collects a position update once per minute. Two consecutive location reports for the same spot will generate a bearing of 90, as the arctangent of 0/0 is 0 degrees in standard angle which is 90 degrees of bearing (pointing due east.)

    Fixes, set your ‘time.sleep(1)’ to ‘time.sleep(60)’ so that a different sample is collected each time. For the first sample collected, use ‘oldPosition = droid.getLastKnownLocation()’ which may be a cell location rather than a gps location, but for the moment I’m not going to be too worried about that.

    Long term issue 73 against ASE is a request to be able to make the resolution that ASE reports for time and distance user configurable. Which would mean that you can make the GPS report it’s location every hour, or hundreds of times per second. The latter would be rather processor intensive, and I think the former might be a bit too course unless you gave the phone a reliable power source and was asking it to give you updates of some thing’s position for a year or so. (I would think there would be better tools than a $399 phone, but who knows.)

    I have to update the script above as a result. (I’ll also fix the issue where the deltalat’s sign gets changed, and such. (the second “deltalat =” line isn’t needed, and the first one has ‘loatitude’ when it should be ‘latitude’.) Well, have to have fun some time, right?

    Comment by Rusty — August 27, 2009 @ 2:26 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress