Sybase Business Intelligence Solutions - Database Management, Data Warehousing Software, Mobile Enterprise Applications and Messaging
Sybase Brand Color Bar
  blank
 
 
 
 
 

The Encryption Control Saga v12

Unfortunately, I’ve committed the cardinal sin of testing on one version then doing a cut-n-paste for the other version when I wrote More On UltraLiteJ Security on BlackBerry. Now that I have joined Tom on this blog, I’ve edited the original post to correct the code to include a correct sample for version 11.0.x, and here below I include the correct sample for UltraLiteJ 12.0.0.

AES CBC EncryptionControl for 12.0.0

While the sample provided in my original article does encrypt and decrypt correctly, it uses Electronic Code Book (ECB) mode. In this mode, blocks of data that are identical will always encrypt to the same encrypted block. Here I provide an EncryptionControl that uses Cipher Block Chaining (CBC), which uses the page number to seed the encryption and where identical blocks in a page will encrypt to different values based on where in the page the blocks are.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
import net.rim.device.api.crypto.*;
/** Class to implement encryption/decryption of the database
 * using Cipher Block Chaining (CBC).
 */
final class Encryptor implements EncryptionControl
{
    private static final int SOME_PRIME1 = 88897;
    private static final int SOME_PRIME2 = 3847;
 
    private CBCEncryptorEngine _encryptor;
    private CBCDecryptorEngine _decryptor;
    private int _block_length;
    private byte[] _iv;
 
    /** Generate an initialization vector based on the page number
     * @param page_no Page number
     * @return initialization vector for the page
     */
    private InitializationVector buildIV( int page_no )
    {
        for( int i = _block_length; i > 0; ) {
            page_no *= SOME_PRIME1;
            page_no += SOME_PRIME2;
            _iv[ --i ] = (byte)(page_no & 255);
        }
        return new InitializationVector( _iv );
    }
 
    /** Encrypt a page stored in the Database.
     * @param page_no the number of the page being encrypted
     * @param src the encrypted source page which was read from the Database
     * @param tgt the unencrypted page (filled in by method)
     * @param num_bytes the number of bytes to decrypt
     */
    public void decrypt( int page_no, byte[] src, byte[] tgt, int num_bytes )
        throws ULjException
    {
        try {
            _decryptor.setIV( buildIV( page_no ) );
            for( int i = 0; i < num_bytes; i += _block_length ) {
                _decryptor.decrypt( src, i, tgt, i );
            }
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
 
    /** Encrypt a page stored in the Database.
     * @param page_no the number of the page being encrypted
     * @param src the unencrypted source
     * @param tgt the encrypted target page which will be written to the Database (filled in by method)
     */
    public void encrypt( int page_no, byte[] src, byte[] tgt )
        throws ULjException
    {
        try {
            _encryptor.setIV( buildIV( page_no ) );
            for( int i = 0; i < src.length; i += _block_length ) {
                _encryptor.encrypt( src, i, tgt, i );
            }
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
 
    /** Initialize the encryption control with a password.
     * @param password the password
     */
    public void initialize( String password )
        throws ULjException
    {
        try {
            byte[] bytes = password.getBytes( "UTF8" );
            SHA256Digest sha = new SHA256Digest();
            sha.reset();
            sha.update( bytes, 0, bytes.length );
            byte[] key_bytes = new byte[32];
            sha.getDigest( key_bytes, 0, false );
            AESKey key = new AESKey( key_bytes );
            _encryptor = new CBCEncryptorEngine(
                    new AESEncryptorEngine( key )
                );
            _block_length = _encryptor.getBlockLength();
            _iv = new byte[ _block_length ];
            _decryptor = new CBCDecryptorEngine(
                    new AESDecryptorEngine( key ), buildIV( 0 )
                );
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
}

UltraLiteJ BlackBerry Connectivity

Recently, I struggled with a nasty device operating system bug which caused me to delve deeply into the various connectivity paths available to application developers using UltraLiteJ on BlackBerry devices.  Although I knew most of the details already, I realized that this knowledge may not be universal, and since I don’t hate my users (quite the contrary), I figured I would post a summary of the options, and save you some time interpreting the Research In Motion (RIM) white papers.

To those of you who want the final word on this topic, download http://na.blackberry.com/eng/developers/resources/Network_Transports_tutorial.pdf from RIM.

So here goes…

UltraLiteJ on a BlackBerry device can communicate to a MobiLink Synchronization Server through many different paths.  RIM really only considers two (BES and BIS) to be first class options while the others are last resort.

  1. The BES path.  This is the preferred route.  It applies only to devices that are paired with a BlackBerry Enterprise Server (BES) and requires the BES to have MDS service enabled.  With this networking option, connectivity gets proxied through the MDS server attached to the BES.  This option is carrier independent (works when roaming).  The bonus of this path is that if the device detects a Wi-Fi route to the BES, it will use Wi-Fi instead of the cellular network and whether it uses the cell network or Wi-Fi, all communication between the device and the BES are encrypted.For UltraLiteJ users, you can ensure you are using the BES path by using:

    mySyncParms.getStreamParms().setURLSuffix( “;deviceside=false” );

    If you are using a relay server, your URL suffix might look like this:

    mySyncParms.getStreamParms().setURLSuffix(
            “ias_relay_server/client/rs_client.dll/MyRelayUser.MyRelayProfile;deviceside=false”
        );

  2. The BIS path.  This is the next best thing.  When the device is not paired with the BES, devices can communicate through the BlackBerry Internet Service (BIS – which is essentially a MDS/BES hosted by RIM on the carrier’s behalf).  Like the BES path, this path works when roaming and will route through Wi-Fi if it is available.  Unlike the BES path, communication is not encrypted (use HTTPS to secure the synchronization stream to the MobiLink server) and if you need to access servers behind a corporate firewall, you need to add a hole in your firewall, use some tunneling technology from a public server to the firewalled server, or use the Sybase relay server (easiest option).  This network path requires you to be a RIM Partner and requires you to apply for permission to use it for each application.
  3. The Direct TCP path.  This option uses a publicly available IP gateway accessed from the cellular network.  It may require you to specify gateway information in one of the device option screens.  It will not attempt to route through Wi-Fi and communication is not encrypted (again use HTTPS to secure if needed).  UltraLiteJ user should add “;deviceside=true” to the URL suffix:

    mySyncParms.getStreamParms().setURLSuffix( “;deviceside=true” );

  4. The Direct Wi-Fi path.  RIM literature usually considers this as part of the Direct TCP path.  To force Wi-Fi use instead of the cellular network, add “;interface=wifi” to the URL. UltraLiteJ user should add “;deviceside=true” to the URL suffix:

    mySyncParms.getStreamParms().setURLSuffix( “;deviceside=true;interface=wifi” );

  5. The WAP 2.0 path.  Devices with OS 4.2.0 or greater can connect through a cellular carrier’s WAP 2.0 gateway. Applications must look up the Service Book record, if any, for the WAP 2.0 gateway to retrieve the Connection UID (sample code is provided in the RIM White Paper under the section Wireless service provider WAP 2.0 gateway) and then set the UltraLiteJ URL suffix appropriately:

    mySyncParms.getStreamParms().setURLSuffix( “;ConnectionUID=” + uid );

    I’m not sure how portable this is – that is, if roaming will the service books get updated to find the new gateway? Exercise left to the reader :-)

  6. The WAP 1.x path.  This is the least preferred path according to RIM and should only be used in third world countries and old devices.  An application developer would need to look up the APN information for each carrier they intend to use, then create the appropriate URL suffix.  You could borrow some code (RIM won’t mind, they even recommend that you use it) from RIM’s Network Diagnostic Tool to use the service books to try to identify what carrier you are on but the gateway parameters would still have to be hardcoded in the application for each supported carrier.  Below is a string for Rogers (for others search for WAP Gateway APN” or see http://www.blackberryfaq.com/index.php/Carrier_specific_APN/TCP_settings):

    mySyncParms.getStreamParms().setURLSuffix(
            “;WapGatewayAPN=internet.com;WapGatewayIP=208.200.67.150;WapGatewayPort=9201″
        );

    A complete list of WAP 1.x parameters is available in the RIM White Paper.

Conclusion

Be explicit!  If using BES, add the “;deviceside=false” string to the URL suffix.  If use Direct TCP, use the “;deviceside=true”.  The default behaviour (yes I’m Canadian so I spell it with a “u”) is different whether you are on an IDENT network or not.  Always better to be clear what you want (if just for the security aspect).

Make your life simple!  Use BES or BIS – you can join at the base level for free and then apply for permission to use BIS transport.  If you need help, contact Tom Slee – he will put you in contact with the right person at RIM and he would love to know how you are using UltraLiteJ so we can make it better.

This last point (let us know what you are doing) will probably be a subject of a separate post :-)

More on UltraLiteJ Security on BlackBerry

Philippe Bertrand, who wrote this recent post on BlackBerry security, has some an improved code sample for us.

An Improved EncryptionControl for BlackBerry Devices

While the sample provided in my original article does encrypt and decrypt correctly, it uses Electronic Code Book (ECB) mode. In this mode, blocks of data that are identical will always encrypt to the same encrypted block. Here I provide an EncryptionControl that uses Cipher Block Chaining (CBC), which uses the page number to seed the encryption and where identical blocks in a page will encrypt to different values based on where in the page the blocks are.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
import net.rim.device.api.crypto.*;
/** Class to implement encryption/decryption of the database
 * using Cipher Block Chaining (CBC).
 * For UltraLiteJ 11.0.x
 */
final class Encryptor implements EncryptionControl
{
    private static final int SOME_PRIME1 = 88897;
    private static final int SOME_PRIME2 = 3847;
 
    private CBCEncryptorEngine _encryptor;
    private CBCDecryptorEngine _decryptor;
    private int block_length;
    private byte[] _iv;
    private byte[] _working_page;
 
    /** Generate an initialization vector based on the page number
     * @param page_no Page number
     * @return initialization vector for the page
     */
    private InitializationVector buildIV( int page_no )
    {
        for( int i = _block_length; i > 0; ) {
            page_no *= SOME_PRIME1;
            page_no += SOME_PRIME2;
            _iv[ --i ] = (byte)(page_no & 255);
        }
        return new InitializationVector( _iv );
    }
 
    /** Encrypt a page stored in the Database.
     * @param page_no the number of the page being encrypted
     * @param src the encrypted source page which was read from the Database
     * @param tgt the unencrypted page (filled in by method)
     */
    public void decrypt( int page_no, byte[] src, byte[] tgt ) 
         throws ULjException
    {
        try {
            // WORKAROUND RIM bug that results in first 32 bytes
            // not being valid - required in UltraLiteJ 11.0.x not 12
            if( _working_page == null
                    || _working_page.length != src.length
                 ) {
                 _working_page = new byte[ src.length ];
             }
             System.arraycopy( src, 0, _working_page, 0, src.length );
             int start = (page_no == 0 ? 32 : 0);
            _decryptor.setIV( buildIV( page_no ) );
            for( int i = start; i < src.length; i += _block_length ) {
                _decryptor.decrypt( src, i, tgt, i );
            }
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
 
    /** Encrypt a page stored in the Database.
     * @param page_no the number of the page being encrypted
     * @param src the unencrypted source
     * @param tgt the encrypted target page which will be written to the Database (filled in by method)
     */
    public void encrypt( int page_no, byte[] src, byte[] tgt ) 
        throws ULjException
    {
        try {
             int start = (page_no == 0 ? 32 : 0);
            _encryptor.setIV( buildIV( page_no ) );
            for( int i = start; i < src.length; i += _block_length ) {
                _encryptor.encrypt( src, i, tgt, i );
            }
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
 
    /** Initialize the encryption control with a password.
     * @param password the password
     */
    public void initialize( String password ) 
        throws ULjException
    {
        try {
            byte[] bytes = password.getBytes( "UTF8" );
            SHA256Digest sha = new SHA256Digest();
            sha.reset();
            sha.update( bytes, 0, bytes.length );
            byte[] key_bytes = new byte[32];
            sha.getDigest( key_bytes, 0, false );
            AESKey key = new AESKey( key_bytes );
            _encryptor = new CBCEncryptorEngine(
                    new AESEncryptorEngine( key )
                );
            _block_length = _encryptor.getBlockLength();
            _iv = new byte[ _block_length ];
            _decryptor = new CBCDecryptorEngine(
                    new AESDecryptorEngine( key ), buildIV( 0 )
                );
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
}

The above sample was corrected to work with the latest version of UltraLiteJ 11.0.1. For UltraLiteJ 12 please see The Encryption Control Saga v12.

UltraLiteJ Security on BlackBerry Devices

This post is by Philippe Bertrand, staff software developer working on UltraLiteJ, and answers several questions we have had about database security on BlackBerry.

Introduction

In this article I present an overview of security issues on BlackBerry devices and discuss how an UltraLiteJ application developer may address them based on their security needs. I cover databases residing in the Persistent Store (Persistent Object Store), on-board flash memory (accessed as file:///store/ and hence referred to as flash file store), and media cards.
This article is based on the SQL Anywhere 11.0.1 release on a BlackBerry Device OS 4.2 – 5.0, although it applies to all known versions of UltraLiteJ at the time of writing. No knowledge of cryptography is required, nor is a developer background. However, to make use of the included code sample the reader should be familiar with the UltraLiteJ API and BlackBerry development.

Basic Security

The most basic level of security prevents theft of information by nosey individuals, such as those who can’t resist peeking at an open pay stub that a co-worker has left out in the open. We could also consider the value of the data: data that has little value to a prospective thief requires only minimal security. All data should at least be protected from accidental damage, such as when your six year old gets hold of your BlackBerry and starts playing with it. In addition, in a synchronization environment where an UltraLiteJ database is synchronized with a MobiLink server, you should also consider the risk of bad data being uploaded to the consolidated database.
For this kind of data, the BlackBerry device should be protected with a device password and the UltraLiteJ database should also be protected with a non-default password. The database password prevents other UltraLiteJ applications from accidently accessing a database. Databases in the BlackBerry Persistent Store are protected from non-UltraLiteJ applications using the type-checking mechanism, however files in the flash file store or on a media card can be viewed by other applications.

Protecting Sensitive Data

The next level of security protects data from malware and opportunistic thieves (stealing the device or media card then seeing if there is anything of value on it). While the basic security (device password) of the BlackBerry device is quite effective and will deter most thieves from accessing the persistent store (attempts to re-install the OS or too many attempts to guess the password will result in a security wipe of the Persistent Store), there is increased risk for data on flash file storage and data on media cards is completely unprotected (with the basic level of security). The value of the data determines how much effort the thief may take to acquire it – for instance, credit card and information used in identity theft is highly valued. You should also consider legislative privacy requirements (for personal health information for instance) and how embarrassing it could be to your company if customer or corporate data were leaked.
UltraLiteJ applications provide several options with respect to protecting this kind of data.
1. Encrypting just the sensitive data
Sensitive data such as credit card numbers, social security numbers, driver’s license numbers, etc. can be directly encrypted before they are placed in the database. While this approach may be the least CPU-intensive, it is more problematic to program, and such values may not be used in indexes in any order-preserving way (lookups can still be performed using the encrypted value). Access to column values is achieved through a simple accessor class (code not included) very similar to the Encryptor class described below, the only challenging part being to make sure to access all values through such a class.
2. Encrypting the entire database
An easier and more encompassing approach is to define a class that implements the UltraLiteJ EncryptionControl interface (sample below). This class is passed to an UltraLiteJ configuration object to create the database and is used for all subsequent connections that open the database. UltraLiteJ will use it to encrypt and decrypt all database pages.

Update: this code has been updated to show that you need to loop through the array of bytes you want to encrypt, passing them a chunk at a time to the encrypt or decrypt methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
/** Class to implement encryption/decryption of the database.
 */
class Encryptor
    implements EncryptionControl
{
    private BlockEncryptorEngine _encryptor;
    private BlockDecryptorEngine _decryptor;
    private int block_length;
 
    /** Encrypt a page stored in the Database.
     * @param page_no the number of the page being encrypted
     * @param src the encrypted source page which was read from the Database
     * @param tgt the unencrypted page (filled in by method)
     */
    public void decrypt( int page_no, byte[] src, byte[] tgt )
        throws ULjException
    {
        try {
            for( int i = 0; i < src.length; i += _block_length ) {
                _decryptor.decrypt( src, i, tgt, i );
            }
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
 
    /** Encrypt a page stored in the Database.
     * @param page_no the number of the page being encrypted
     * @param src the unencrypted source
     * @param tgt the encrypted target page which will be written to the Database (filled in by method)
     */
    public void encrypt( int page_no, byte[] src, byte[] tgt )
        throws ULjException
    {
        try {
            for( int i = 0; i < src.length; i += _block_length ) {
                _encryptor.encrypt( src, i, tgt, i );
            }
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
 
    /** Initialize the encryption control with a password.
     * @param password the password
     */
    public void initialize( String password )
        throws ULjException
    {
        try {
            byte[] bytes = password.getBytes( "UTF8" );
            SHA256Digest sha = new SHA256Digest();
            sha.reset();
            sha.update( bytes, 0, bytes.length );
            byte[] key_bytes = new byte[32];
            sha.getDigest( key_bytes, 0, false );
            AESKey key = new AESKey( key_bytes );
            _encryptor = new AESEncryptorEngine( key );
            _decryptor = new AESDecryptorEngine( key );
            _block_length = _encryptor.getBlockLength();
        } catch( Exception e ) {
            throw new EncryptionError();
        }
    }
}
 
/** Error class for encryption errors.
 */
class EncryptionError
    extends ULjException
{
    /** Constructor.
     */
    EncryptionError()
    {
        super( "Encryption Error" );
    }
 
    /**
     * Get the error code, associated with this exception.
     * @return the error code (from the list at the top of this class) associated
     *  with this exception
     */
    public int getErrorCode()
    {
        return ULjException.SQLE_ERROR;
    }
    /** Get exception causing this exception, if it exists.
     * @return null, if there is no causing exception; otherwise, the exception causing this exception
     */
    public ULjException getCausingException()
    {
        return null;
    }
    /** Get offset of error within a SQL string.
     * @return (-1) when there is no SQL string associated with the error message;
     * otherwise, the (base 0) offset within that string where the error occurred.
     */
    public int getSqlOffset()
    {
        return -1;
    }
}

Benefits
Both of these methods provide good security while providing excellent flexibility. Databases can be created and pre-loaded onto media cards for distribution or copied into the flash file store using the BlackBerry USB mass media mode while still providing data security. This option also allows for applications to run background synchronizations.
3. Using BlackBerry encryption mode for flash file store and media cards
A final approach (for flash file store and media card databases only) is to rely on the BlackBerry Device Options > Memory > Encryption Mode (or equivalent IT Policy) setting to specify the level of security for the database that is stored on either the flash file store or the media card. This approach requires the device to be unlocked whenever the application requires access to the database. Some values for this setting require the database to be created on the BlackBerry it will be used on, and a database cannot be imported with this security enabled, either through media card distribution or by the USB mass storage mode. While the Security Password encryption mode is more portable (between BlackBerry devices), the algorithm is not public, nor are there any desktop tools to create compatible encrypted files (Media Manager only handles media files and USB mass storage mode does not encrypt and decrypt files going to or from the device). For all enabled security modes, applications must implement a PersistentContentListener with a persistentContentStateChanged() method that closes all connections to the database when the device locks (strong references to the listener may cause memory leaks and other problems should the application fail and not clean up properly). After locking completes and the device key is released by the operating system, no database access, including background synchronization with a MobiLink server, is possible until the device is unlocked.
There is a risk if the user somehow installs malware on their device. When the device is unlocked, any application on the device can read or copy the database file.
Encryption is easily turned off
It should be noted that if a database is created without any encryption, changes to the Encryption Mode (or equivalent IT Policy) setting will have no effect on the security level of the database. If the database is created with any kind of encryption and the Encryption Mode is changed to None (while the card is still in the device), the database will become viewable by any BlackBerry even though it is still wrapped in an encrypted wrapper!

Critical Data and Determined Thieves

There is a select group of thieves that, given a locked device, can break into it and find the encryption keys that are in memory. This is a sophisticated attack known as a cold boot attack (http://en.wikipedia.org/wiki/Cold_boot_attack), which involves breaking into a device without removing power to the memory to get the device’s encryption key and then having access to everything stored in persistent store and other forms of memory. BlackBerry OS applications are protected by disabling access to the device encryption key when the device locks or is connected to the desktop.
An UltraLiteJ application would protect itself by implementing a MemoryCleanerListener, which drops all connections to the database and destroys all references to configuration objects used to connect to the database. The memory cleaner can be configured to run after a specified amount of time after the device is locked. A fully secure solution which does not require a user to re-enter a database password after the device is unlocked would do the following:
• Application implements an EncryptionControl for encrypting the UltraLiteJ database
• When first run, the application prompts for a database password
• The password is securely stored in the Persistent Store using PersistentContent.encode()
• On connect, the application uses the password and the EncryptionControl to create a connection to the database (do not keep extra references to the configuration object or password)
• When the MemoryCleanerListener is invoked, the application drops all connections to the database, destroys all configurations, EncryptionControl objects, and all copies of the password (not the one in Persistent Store)
• Once the user unlocks the device, the application can retrieve the password from the Persistent Store and reconnect to the database.
One interesting opportunity that arises when you implement a MemoryCleanerListener and a PersistentContentListener is that they define the time between when a device is locked and when it should be completely secure – an ideal time for a background synchronization!
If you use the preceding security solution, ensure either that the device Options > Memory > Encryption Mode (and associated IT Policy) are turned off, or that the database has first been created on the desktop to maintain the full features and functionality described above.
It should be noted that even RIM’s solution has one weakness since applications may hold onto a “ticket,” keeping the device key in memory until they release the ticket. A single bad application may keep the device in a vulnerable state. Users should pay attention to the little padlock icon on the locked screen. It only ”locks” when the key is cleaned up.

Persistent Store Security Options

The device Options > Security Options > General Settings > Content Protection (and the associated IT Policy setting) affects only the Persistent and Runtime Store of the device. Content protection is not automatic but requires applications to do the right thing with respect to their data. At this time, UltraLiteJ does not provide any support for this setting.
Please note that the device Options > Memory > Encryption Mode is independent of the Options > Security Options > General Settings > Content Protection setting. Encryption Mode controls the security of the flash file store and media card.

IT Policy Versus User Settings

All device options mentioned in this article may be overwritten by the IT policies of the BlackBerry Enterprise Server (BES) a device is paired with, in which case the most restrictive setting (user or IT) takes precedence. Thus, a user may not choose a less restrictive Encryption Mode than what is mandated by the corresponding IT Policy setting.

Wandering into Carnival of the Mobilists

A rotating collection of links called Carnival of the Mobilists has been running for a couple of years now, pulling together the best in mobile blogging. I’m happy to say that this blog appears in its 206th edition, no less, hosted by mobile marketing company mobiThinking. You can find the carnival here you can find a set of posts that are, not surprisingly given the time of year, looking forward to see what’s coming next in the worlds of mobile computing.

Mobile Data Apocalypse?

This piece was written by Ben St. Pierre, a co-op student from the University of Waterloo who spent a very productive term with us in the fall. He wrote this in December, just before taking off for the wintry north of Ontario. Thanks for a great term Ben.


With the popularity of smart phones on the rise, and the ever increasing consumer hunger for bandwidth, the sustainability of the mobile data network has come into question. Recently, AT&T has come under fire for its lackluster wireless 3G coverage in the two most heavily used wireless data networks, San Francisco Bay and Manhattan [link]. Is this the beginning of the mobile data apocalypse, or will carriers and consumers adapt?

The Current Trend

Earlier this year, Cisco published a white-paper which presents their mobile traffic projections and growth trends [link]. They project that mobile data traffic will double every year, increasing 66 times over from 2008 to 2013 with Latin America having the strongest growth, followed by Asia-Pacific. This growth will mostly be driven by mobile video streaming and carrier-subsidized 3G-equipped laptops and netbooks [link].

This growth may seem wonderful at first, but the network infrastructure is not equipped to handle such a large jump in usage and the projected profits from this usage are expected to grow at a much slower rate [link].

To make matters worse, FCC chairman Julius Genachowski is warning of a “looming spectrum crisis” [link]. Available spectrum is scarce, and reassigning spectrum is a process that takes years to implement.

Mitigating Factors

Although the current projections show unsustainable growth, some factors may help keep mobile data technologies in the hands of the mass-market.

Public Education

AT&T has cited that 3% of their smartphone users are currently consuming 40% of the network capacity [link]. This seems to be due to the lack of education of their users. When contacting these customers, AT&T found that they were using their smartphones by streaming audio around the clock or watching large numbers of streaming videos, both of which consume excessive amounts of data.

The End of Unlimited Data Plans

The easiest way for carriers to educate their users is by giving them a comprehensive data plan. Unlimited plans make it easy for users to abuse the network since they don’t need to be aware of their usage. If users need to be aware of how much data they are using due to usage-based plans, excessive use would likely be less of a problem. Carriers have already hinted that unlimited plans are not sustainable and that in the long term, a switch to usage-based plans is likely [link].

Long Term Evolution (LTE) and 4G

4G wireless technology promises faster speeds and better scalability than the current 3G networks. The FCC has a four-part plan to help ease the deployment of 4G which includes unleashing spectrum and removing obstacles for ubiquitous 4G deployments [link]. However, the projected bandwidth requirements still exceed the capacity of 4G technology [link] and a widespread deployment is still years away from available.

Femtocells

Femtocells are small network appliances (similar to network routers) which use a 3G antenna and a regular broadband internet connection (such as DSL or cable) to connect 3G wireless devices to their mobile carriers. AT&T has already released a limited number of 3G Femtocells as a test program which supports up to 4 wireless phones in a residential setting [link]. Sprint-Nextel also has its own femtocell offering through its AIRAVE product [link].

The next generation femtocells are also already in development. LTE groups have indentified femtocells as a priority item for 4G deployment. This could increase early 4G adoption by consumers who wish to get 4G handhelds in areas where 4G is not yet available and thus help alleviate 3G network congestions.

Wifi

Many consumers already know that a Wifi network is preferable to using 3G. They know this due to the visible speed difference in bandwidth-hungry applications. This has lead to consumers looking for wireless devices which also includes a Wifi interface. In a recent study, it was found that 77 percent of mobile phone users want Wifi on their next handset and 75 percent of users who already have Wifi on their phone use it regularly [link] The same study also projects that 90 percent of smartphones will have Wifi by 2014, up from the current 44 percent.

Conclusion

Because of the enormous expected growth, many opportunities have arisen for businesses to solve and although the traffic projections look apocalyptic in growth, there are many mitigating factors which may lessen the impact. Whether or not these factors will be enough to keep the mobile data networks running smoothly however, has yet to been seen.

SQL Anywhere for iPhone Beta Program Now Open

Not only are our colleagues in the Afaria team releasing their Afaria for iPhone product, but today the SQL Anywhere for iPhone Beta Program is open.

It’s been a long time since we announced that we’d be providing SQL Anywhere software for the iPhone but today, after a long and winding road, we are finally launching the SQL Anywhere for iPhone Beta program. If you register here, then you’ll be up and going quickly. Anyone who registered previously will hear from us today. The official announcement is here.

Here are some questions and answers about the release.

Q. What are the key features of SQL Anywhere for iPhone?

  • Data synchronization between device and enterprise. With SQL Anywhere for iPhone, developers can build rich applications that use a local database on the device and that share data between the device and a server.
  • Integration with enterprise databases. Developers can build applications that share data between the iPhone and relational databases such as Oracle, Sybase ASE and SQL Anywhere, IBM DB2, Microsoft SQL Server, and MySQL.
  • Integration with enterprise systems. Developers can build applications that share data between the iPhone and enterprise systems.
  • A common infrastructure for multiple devices. SQL Anywhere already supports Windows Mobile and BlackBerry. With iPhone support, SQL Anywhere now supports the three major smartphone platforms for business applications.

Q. What are the key benefits of SQL Anywhere for iPhone?

Businesses looking to provide line-of-business applications to their workforce can bring those applications to the iPhone using the same SQL Anywhere technology deployed on hundreds of thousands of other mobile devices.

Application developers looking to sell line-of-business applications can now use a single platform for all three major smartphone platforms for business applications.

Q. How is this different from other SQL Anywhere offerings? Why?

SQL Anywhere for iPhone brings most of the data management and data synchronization features available on other mobile platforms, such as Windows Mobile and BlackBerry, to the iPhone.

SQL Anywhere for iPhone includes the UltraLite database management system and synchronization client. This is a port of the UltraLite database for Windows Mobile already used in many enterprise deployments: the features and implementation are the same between the two platforms, so customers can be assured that this is already a mature and well-recognized technology.

At the server, a single SQL Anywhere MobiLink server can serve clients on all kinds of mobile devices, including the iPhone. The same set of enterprise systems are supported across all mobile devices.

To develop applications for the iPhone using SQL Anywhere, you need a Mac and a copy of the iPhone SDK. When it is reaches general availability, SQL Anywhere for iPhone will be distributed as part of SQL Anywhere for Mac OS.

Database encryption on device is supported, but encryption between the iPhone and the server is not currently supported. Push synchronization is not currently supported.

While UltraLite on Windows Mobile can be run either in-process or as a separate process, only the in-process version is available for iPhone, as iPhone does not support multiple processes running at the same time.

Q. How will developers deploy applications that include SQL Anywhere for iPhone?

Developers compile SQL Anywhere functionality into their application, and then deploy their application in the same way as any other application – through the App Store or through the same enterprise deployment mechanisms Apple recommends for other mobile applications.

Q. What iPhone versions does SQL Anywhere support?

This release supports iPhone version 3.1 and back as far as version 2.0.

Q. What version of SQL Anywhere is needed?

To synchronize data between the iPhone and a server, developers must have the SQL Anywhere 11 MobiLink synchronization server.

Google Street View reaches Sybase Waterloo

And here we are…


View Larger Map

Finally, some CPU improvements for mobile devices

Two or three years ago, the state of the art CPU for mobile handsets was the PXA270, an XScale ARM processor originally from Intel and then sold to Marvell. It chugged along at 625MHz or so.

Nowadays, we are far from that. Today’s cutting edge phones and mobile devices use… oh right, the same chip. Yes, for about three years, Moore’s law has taken a break when it comes to mobile devices. I don’t know enough about hardware to know the reasons – is it something about the chip architecture? is it having to deal within the limits of the crappy battery technology we are plagued with? Well, I don’t know, but it certainly puts a crimp in mobile software.

But maybe things are finally changing. A couple of weeks ago Motorola released their latest round of enterprise handhelds (the old Symbol line), and the MC9500 is based on the Marvell PXA320 chip, which runs at 806 MHz and yet which apparently has better power consumption than the last lot. And now it looks like faster chips may be reaching consumers as the Acer A1, a new Android-based phone, sports a Qualcomm 8250 processor running at 768MHz. So it looks like the 625MHz barrier is finally broken, and that’s good news for mobile computing.

So these new devices will scream. At least, pre-release versions of the Motorola one did. It has a built-in accelerometer and emitted a scream when dropped, as a safety feature. Unfortunately, according to the Register, that just made it too much fun to drop so it was kind of counter productive. Production units will be boringly silent when they crash, unharmed, to the ground off the top of whatever surface you want to knock them from.

UltraLiteJ now supports databases on BlackBerry Flash Memory and SD Cards

SQL Anywhere is Sybase’s mobile database, and UltraLiteJ is included in that product as a mobile database for the BlackBerry.

We’ve been working for some time to get UltraLiteJ to support the internal flash (device memory) and SD Card stores on BlackBerry devices, and now I’m happy to say that it’s done and available. Right now, existing SQL Anywhere users can get this technology by downloading the latest Express Bug Fix (EBF) release. We’ll be upgrading the free SQL Anywhere Developer Edition download soon and I’ll post a link when that’s done, but for now you have to go the EBF route.

As this technology is being provided in an EBF we haven’t updated the documentation, so here is how you use it. In this post, I’m assuming you know how to use UltraLiteJ already.

First, get the software. Assuming you have SQL Anywhere installed, click here for the EBF (you need to sign up for a free Sybase account). Click SQL Anywhere, scroll down to the Windows x86 platform, and click “SQL Anywhere – Express Bug Fix, Update of 11.0.1 to build 2308 (all components)”, or any later build as they become available.

Once you’ve installed it, you will see that under the SQL Anywhere\UltraLite\UltraLiteJ folder there is now a BlackBerry 4.2 folder. That’s the folder containing the JAR and COD files you need.

There’s not a separate JAR or COD file for the flash memory or SD Card support, the same software can access databases in the persistent store and in file-based locations. It works by providing a new subclass of the ConfigPersistent object, which is what you use when connecting to or creating a database. The new class is ConfigFileME, and it inherits many of the same methods that ConfigObjectStore does, so anything you have about cache size and so on remains the same. The only difference is that when creating the configuration object the database name is a bit different.

For a database in the persistent store (all we have had until now) you would do something like this to connect to a database, using a string (no path of course) to specify the database in the persistent store:

String dbName = "my_persistent_store_database";
_config = (ConfigObjectStore)DatabaseManager.createConfigurationObjectStore(dbName);
Connection conn = DatabaseManager.connect( _config );

For a database in the internal flash (device memory) you would do this instead:

String dbName = “file:///store/user/home/my_file_database.jdb”;
_config = (ConfigFileME)DatabaseManager.createConfigurationFileME(dbName);
Connection conn = DatabaseManager.connect( _config )
;

And that’s it. Once you have a connection, all the other database operations are exactly the same as if you were using the persistent store. Now here are a few more details, stolen from my colleague Philippe Bertrand:

Databases names must follow the format for a fully qualified, absolute path file name as described in the file URL format as part of IETF RFCs 1738 & 2396. That RFC dictates that a file URL takes the form:

   file://<host>/<path>

In the form above, <host> is the fully qualified domain name of the system on which the <path> is accessible, and <path> is a hierarchical directory path of the form
<root>/<directory>/<directory>/…/<name>.

For BlackBerry devices, the internal flash is accessed using URIs starting with file:///store/ (case sensitive) while the SD media card is accessed using URIs starting with file:///SDCard/ (case sensitive).  File paths are case insensitive except for the host portion which is case sensitive.  Percent (%) characters have special meaning and relative paths (directories "." and "..") are not allowed.  For more information on file name restrictions, please consult the BlackBerry JDE (4.2 or later) API Reference for the javax.microedition.io.file package.

You will find that, for data access, persistent store is faster than internal flash and internal flash is faster than SD cards.  On the other hand, databases stored on internal flash or media cards can keep more rows in memory and/or use a bigger cache to improve performance.