Archive for the Mobile Category

Updating a Database in a Deployed Mobile App

May 22, 2013

One of the issues I had to deal with in building my mobile app was the evolving database schema as more features were added and the app grew.  I needed a way to update the database scheme without blowing away the old one and forcing the app to redownload all the data.  While I could have done that without any loss of data (there is no user supplied data to be lost), the main data set is over 8 MB in size and that isn’t nice to users’ wireless plans.  What I needed was a way to incrementally change the schema as the app was updated.

First Attempt

My first attempts just used if() statements.  i.e. if the current app version was one value and the new app version was a different value, apply this update.  This worked well enough for my development versions where I was strictly controlling the upgrade versions.  But I quickly realized that if I were to try to do that with versions of the app out in the wild, where I had no control over when it was updated, I’d quickly have a mess of conditionals to deal with.  So I scrapped that idea and went looking for something new.

Final Solution

I don’t think this solution is new to me as I’m sure I saw it somewhere to give me the idea but I couldn’t tell you where I saw it off the top of my head.  The answer was a switch() statement, with each case() block having a bit of the upgrade path from old to new schemas.  In order for this to work, you need to track a version number for the database schema that increases each time you make a change.  Simply start at one and every time you change the schema increment the value.  This value, which I called current_version, is simply a coded variable.

Additionally, the app needs a second value that it stores as part of the application data which contains the database version that is currently installed (installed_version).  Then, each time the application is run, you check these values.  There are three possibilities[1]:

  1. There is no stored value for installed_version. This should only be true the very first time the application is run.  In this case, you simply need to create the entire database schema as it now stands and store the current_version as the installed_version.
  2. The current_version equals the installed_version.  This is the simplest case of all.  Since the database schema is all up-to- date you don’t have to do anything.  Continue on to the main part of the application.
  3. The current_version is greater than the installed_version.  In this case there is work to do and we now need to do the update.

To make the update we simply enter a switch() statement based on the installed_version.  Each case() block should have arguments that start at 1 and increase by one each time.  The first case() block (i.e. case 1:) would hold the commands that were necessary to upgrade from version 1 to version 2.  The second would hold the commands modify the schema from version 2 to version 3, and so forth.

None of the case statements should have the break command associated with them so that after execution they would fall through to the next one and cascade all the way to the end.  That way if a user skips one or more app updates that include database upgrades, each one will be applied in turn and none will be skipped.

Finally, the default() block at the end of the switch statement should contain the code up update the value of the stored installed_version so that it is also up-to-date for the next time the application is run.

Here is a bit of sample code that shows how this works.  It is lifted in modified (with additional comments added) from my application which is written in using Titanium Studio’s Mobile SDK.  It should be noted that the c object is a configuration object that holds a lot of information such as the database name, the table names, and column names among other things.

// Check that we have a stored value for the installed database version
if(!Ti.App.Properties.hasProperty('installed_db_version')){
  // if not, the database has never been created
  createDB();
} else {
  var installed_version = Ti.App.Properties.getInt('installed_db_version');
  if (c.current_version > installed_version){
    Ti.API.info("Performing an upgrade from database version "+old_version+" to version "+c.version);
    var db = Ti.Database.open(c.name);
    // We need to do an upgrade to the new version based on the old version.
    // We'll do this incrementally adding in all the changes since the installed
    // version was created.
    switch(installed_version) {
      case 1:
        db.execute("REPLACE INTO daily_data (source,start,duration,flux_1000_300000,error_1000_300000,ul_1000_300000," +
        "flux_300_1000,error_300_1000,ul_300_1000,flux_100_300000,error_100_300000,ul_100_300000,test_statistic)" +
        "VALUES (1,56000,86400,10,1,0,9,2,0,1,0.5,0,32)");
        db.execute("REPLACE INTO daily_data (source,start,duration,flux_1000_300000,error_1000_300000,ul_1000_300000," +
        "flux_300_1000,error_300_1000,ul_300_1000,flux_100_300000,error_100_300000,ul_100_300000,test_statistic)" +
        "VALUES (1,57000,86400,12,2,0,10,1.5,0,2,0.5,1,32)");
      case 2:
        db.execute("DELETE FROM " + c.T_DAILY);
        db.execute("DELETE FROM " + c.T_WEEKLY);
        Ti.App.Properties.setString('last_lc_data_update','0');
      case 3:
        db.execute(DATABASE_CREATE_GCN);
        Ti.App.Properties.setString('last_GCNNotice_date', '0');
      case 4:
      ...
      ...
      default:
        Ti.App.Properties.setInt('installed_db_version',c.current_version);
        break;
    }
  db.close();
  }
}

In version 1, I was using some dummy data that I was setting explicitly.  In version 2, I had updated the code to get the data from a web service so I needed to remove all the old data and add a stored variable to track the date of the last data point.  In version 3, I added a table to the database and an internal stored variable to track the date that data was last updated.

At the beginning of the code block, I check to see if the database is installed at all.  If not I call a createDB() function.  This function simply contains the code that creates the current version of the database schema.  Another option would be to just set the installed_version to 0 and have a case 0: block that creates the original database schema that all the modifications are applied to but I believe it is faster and cleaner to just create the current schema rather than run through all the old ones applying one updated at a time.  While probably not an issue if you only have a few changes, as the number of modifications grow, so does the time involved in the creation of the new database.

And there you have it.  A simple straightforward way to make sure that your database updates go smoothly across updates of your application code.

One final note is that you don’t have to call this each time the app runs, but only the first time the database is used.  Now I suspect in most cases, the database is used every time the application is run but if there are sections of your app that don’t use the database, this check can be delayed until first use of the database.  This allows the app to load up a bit faster, although the difference may not be noticeable.


[1] There is a fourth possibility and that is that current_version is less than installed_version.  But if you’re properly incrementing current_version then this should never happen unless you roll back a distributed version of your application.  If you do that, all bets are off and you’ll need to find another way to handle the update.  Of course this is only a worry if the older code can’t work with the newer schema.

Going Non-Native

April 25, 2013

Supporting your application on multiple operating systems requires some tough decisions.  Do you develop natively for each OS and maintain multiple code bases?  Do you use a cross platform development framework?  Or do you do something else?

This is the fourth of several articles about my  journey developing an application for NASA’s Fermi Gamma-ray Space Telescope mission.  Earlier articles can be found here:

I knew from the start that in the end the application needed to run on both Android and iOS devices as both are widely used in the scientific community the app targeted.  I chose Android for the prototype simply for practical reasons:  I owned an Android phone and tablet but the only Apple devices in my house were my old MacBook from work and my daughter’s 4th gen iPod Nano, neither of which ran iOS.  I didn’t have a device to test an iOS prototype on.

I originally toyed with the idea of just doing two different apps, one in Java and the other in Objective C to target the two platforms.  Of course that would require learning Objective C as I had (and still have) zero experience with that language.  More appealing would be to find some cross platform development framework that would allow me to target both platforms simultaneously.  And so I started looking around.

I new the app was going to be fairly basic, it would make plots, have some lists and buttons and serve up webpages and data tables.  So the cross platform environment didn’t need to be that spectacular.  I looked at several.  Many of the cross platform systems seemed to be focused on game development (or at least their sales pitches on their websites were).  Others were a little too pricey.

In the end, I settled on Titanium Studio from Appcelerator.  It allowed development for Android and iOS, code was written in JavaScript (which I had some familiarity with), and it was free.  Using this tool, I would be able to write a single program that would compile down to native code on both platforms, and it would look and behave similarly on both.

Switching Development Environments

Up to this point all my work had been done in my primary work environment, namely Linux and the Eclipse IDE.  Titanium uses a customized version of Eclipse for their IDE so that wasn’t an issue, but in order to develop for iOS you have to build and compile on a Mac.  If there is one thing I don’t like about writing iOS apps, it’s that you have to do it on the Mac.  I understand why, but it is still frustrating.

Unfortunately for me, my MacBook is pretty old.  And I definitely felt it.  The iOS simulator was snappy enough but the Android Emulator was painfully slow.  In fact, in the end, it was faster (by nearly an order of magnitude) to deploy test versions to my phone (A Motorola Droid X) and test there instead of trying to use the emulator.  There were times I could definitely relate to this XKCD comic.

There were other issues with working on the Mac, but I’ll save those for another post.  Beyond the slow computer and minor interface dissimilarities, the switch was relatively painless and worked just fine.

One Code Base?

One of the draws of working in Titanium Studio was the promise of being able to develop in a single code base for both mobile OSes.  I knew that there would have to be some OS specific coding in the app to deal with different features (i.e. no hardware back or menu buttons in iOS) but at least it would all be in the same language.

This aspect of the development turned out pretty much as advertised.  I have a single set of classes/files that work just fine on both iOS and Android.  I don’t have to worry about implementing a feature for one OS and then going and reimplementing it for the other.  I just have to do it once and it is there for both.  I believe that this definitely cut down on the total development time it would have taken to build it as two separate apps.

In addition, it saved having to deal with context switches.  I was working in JavaScript with the Titanium APIs.  While developing I didn’t have to switch out of that mode.  Had I tried to do native Java and Objective C applications, I would have been constantly switching between the Java/Android APIs and the Objective C/iOS APIs.  That’s a lot more you have to keep straight in you head and many more chances for errors.  I think of all the benefits, being able to work in a single context and not having to continually switch is probably one of the best benefits of the cross platform framework.

Does that outweigh the issues encountered?  That remains to be seen.  From the point of view of the programmer, I think so.  From the point of view of the functionality provided and the user experience, I haven’t decided yet.  There were several gotchas that I had to deal with where something would work on one OS and not the other that I had to work around but for the most part these were minor.  And there was one major issue that I encountered that I had to work around.  But those issues will be the topic of a future post.

Conclusions

Looking back, maybe Titanium wasn’t necessarily the best choice of a cross-development framework.  It’s been a rocky road that I’ll detail in a future post and give my reasons for that assessment.

However, the decision to use a cross platform development framework was, I believe, a good one.  It kept all the code in a single place.  I didn’t have to worry about implementing a feature in Java and then making sure I remembered to go and reimplement it in Objective C or vice versa,  I could implement it once and it would be there. I think it definitely cut down on the development time, if not the testing time.

So, if you want to target both iOS and Android and possibly others, I think it is definitely worth the effort to look at the existing cross platform development frameworks.  Look closely at the features that you need and what they provide.  And pick a framework that provides what you need/want.  It could save you a lot of work.

Building a prototype

December 3, 2012

Once you have an idea for an application, it really helps to get a prototype up and running to understand better how the application will function and even if it is possible.  Building the mobile data application for the Fermi Science Support Center was no different.

This is the third of several articles about the journey of developing an application for the Fermi Gamma-ray Space Telescope mission.  Earlier articles can be found here:

When I started this project my mobile app development experience comprised a sum total of about two weeks of playing around with the Android SDK.  I had it up and installed, had read a few tutorials, and started in on a small app that was to be a character generator for an old role-playing game to have something (semi-)practical to work on.  But beyond that I was a complete novice.

However, that was infinitely more experience than I had with iOS development and that fact, combined with the fact that I owned an Android smartphone (a Droid X) and the only Apple devices in my house were my daughter’s 3rd gen iPod Nano and a work MacBook that I used occasionally to test out builds of our science software, I decided to try the prototype as a native Android app written in Java.

Breaking New Ground

This was a completely uphill journey, although not necessarily a difficult or unpleasant one.  First was the fact that I was working in Java.  I’m primarily a C++/Perl programmer although I’m familiar with many other languages.  I have a passing knowledge of Java but have never really used it for any real project.  So working in Java required more than a bit of looking up syntax and functions.

Doing event driven, GUI programming wasn’t completely new to me.  On of my side projects that I work on off-and-on is a computer version of the spaceship combat game from the old Star Frontiers role-playing game.  This game is GUI based and so the concepts of responding asynchronously to input events isn’t new.  In fact to a lesser scale, the main data server that I built for the mission does the same thing but the events are coming from sockets instead of a user interface.

And of course the entire Android development process was new.  Screen layouts, the SDK, etc was virgin territory.  I spent a lot of time looking at the developer documentation to figure things out and learn about the various classes, what they did and how they interacted.

What to Build in the Prototype

The next question was what the prototype should contain.  The goal of the prototype was two-fold.  First, I wanted it to demo the basic functionality of the application so I could show it to the project scientist to get approval to do the full application.  And second, it was a way for me to explore a bit of how to do thing in the mobile space and how the whole mobile development paradigm worked.

In the end I decided to implement three major components:

  1. Plotting the light curves – This was to be the major functionality of the application to begin with so it made sense to try this out from the get go.  It didn’t have to pull data from the web or perform updates, but it did need to plot regular data points with errors as well as upper limits and give source and data range selection options.
  2. Show the Fermi Sky Blog – I mainly did this one because it was easy as it was effectively just providing a link to the web page of the blog.  This provided some experience in launching other tasks (Activities in Android parlance) that were external to the application I was writing.  However, while everything I learned here could have been learned from implementing the ATels (#3 below) it did provided a little more functionality to have in the demo.
  3. Building and displaying the list of Fermi related Astronomer’s Telegrams – The main draw to implementing this one was that it provided some experience with reading an RSS feed and parsing the related XML.  Both of which I’d never done before.  It also provided some experience with the process of using the network interface to go out and get data.

Putting It All Together

It took me about a week and a half to get the main application screen and get the basic plotting implemented.  I know because a week and a half after I got back from the AAS meeting I was on the plane headed out to NASA Goddard.  The trip was to help train a new staff member on our data server but I would also be pitching the mobile application to the project scientist.

One thing I was reminded of during that time is how effective and efficient you can be when you are really excited about what you are working on.  I was really excited about this project and wanted to pursue it and so I found myself very focused and working hard to get this done.  It’s amazing what you can accomplish when you don’t let yourself get distracted.

While at Goddard, I had a chance to show the prototype as it existed to the scientist who I had worked on the design with as well as the project scientist.  Both of them liked what I had so far and the project scientist gave the green light for me to work on this officially and implement the full version.

I still had work to do on the prototype to fulfill it’s role as an exploratory tool for some of the techniques in building the app but it had fulfilled at least one of its main purposes as a demo tool.  In the end, the functionality of the prototype was expanded some what to include other things like notifications, automatic data updates, and pulling live data from our servers.  Especially in the latter case, it served as a test client to back end services that I needed to develop for the final applications but that that is the topic for another post.  But all along the way it was a tool to help me learn what was possible, what wasn’t, and how to implement those things.  And in the end, that was really its primary purpose which it fulfilled admirably.

My First Mobile App

November 15, 2012

For the past few months I’ve been working on a mobile application for work.  While it’s nothing fancy, considering the fact that I had no experience with mobile development when I started, I think this first version of the app turned out pretty well.  Although the real test will come as the number of users increases.

The journey from concept to initial version has been fun,  interesting, and educational as well as filled with frustrations and roadblocks along long the way.  This post is only an introduction to a  series of posts  that I’m planning on writing to document a bit of the journey.  For now I just want to tell you a little about the app.

The app is geared towards scientists who use the data from the NASA mission I work on, the Fermi Gamma-ray Space Telescope, although there is nothing that prevents others from using the app to see what is going on.  As part of the mission, we provide a variety of “high level” data on various sources as well as publish alerts and news about sources and events in the gamma-ray sky.  The goal of the app was to aggregate some of this information onto the computer you’re always carrying with you and make it available even when you are off-line for some reason.

Currently the application provides access for four bits of data:

  • Light curves of monitored gamma-ray sources – These are sources that we’ve either been monitoring since the beginning of the mission or which have flared up to be very bright since launch and we’ve been monitoring since they first crossed a predefined luminosity threshold.
  • Gamma-ray Coordinate Network (GCN) notices produced by Fermi – When there is a gamma-ray burst or other very bright transient event (e.g. solar flares among other things) the satellite sends down a real time alert about the event and its location on the sky.  We aggregate these alerts to provide a summary of the best possible data for each event.
  • Fermi related Astronomer’s Telegrams – These are notices sent out by astronomers about objects of interest that have been detected and analyzed using the Fermi data.  While there are dozens of telegrams each week, the app filters and collects the Fermi related ones.
  • Access to the Fermi Sky Blog – This is a weekly summary of events and objects of interest in the gamma-ray sky.

If you’re interested in checking the application out, it is called the Fermi Data Portal and is available on both the Google Play Store and the Apple AppStore and works on both phones and tablets.  Check back for more about the development process and lessons learned from building the app.

Switch to our mobile site