Teambuilding

Last weekend our team went to play Paintball near Prague. It was very good fun, as you can see from the pictures. The team enjoyed the game almost as much as their work! ;)

Core data - metadataForPersistentStoreOfType fails after iPhone restart.

In our application, we often need to do core data migration when we create a new release. To do this we first need to get the metadata from our app’s persistent store,  so that it can be migrated. We use the following line of code for this:

[NSPersistentStoreCoordinator metadataForPersistentStoreOfType:NSSQLiteStoreType URL:self.storeURL error:&error];

However, we noticed that after restarting the iPhone, our application wasn’t able to run. We discovered this error in the log:

Error Domain=NSCocoaErrorDomain Code=256 "The operation couldn’t be completed. (Cocoa error 256.)" UserInfo=0x146570e0 {NSUnderlyingException=error during prepareSQL for SQL string 'SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = 'Z_METADATA'' : attempt to write a readonly database, NSSQLiteErrorDomain=264}

We determined that this is a core data bug, if you do not need to call metadataForPersistentStoreOfType you won’t see that there is a problem.  We noticed that this bug occurs only if we use WAL journal mode in SQLite, when we changed it to DELETE journal mode the app worked as expected.

Using SQLite database in Portable Class Library

I am a member of the team who are developing the C# .NET version of the secure messaging application called Babel.

One of the components which we need is database.  We chose SQLite for this component because it is optimal for our project and recommended for the Portable class library (PCL). We also want to use a framework for Object-Relation Mapping (ORM). After trying several options we decided to use SQLite-NET and its extension for object relationships SQLite-NET Extensions . This setup initially looked good but after some testing we discovered that the SQLite components target .NET 4.0, Win 8, WinPhone 8.1, Silverlight 5 and WinPhoneSilverlight 8.1. However our PCL needs to target .NET 4.5, Win 8.1 and WinPhone 8.1. This is a big problem because we are unable to use the SQLite component in the current build of our PCL.

We download both projects from GIT and delete every project which we cannot use (there are projects targeting every platform) then change the targeted platforms for the PCL projects to .NET 4.5, Win 8.1 and WinPhone 8.1. This will generate many errors in the code that we will need to repair before the project will build. But once the project does build the PCL components will all have the correct target platforms that we can use in our project.

If you want this component with the same target platforms here is a zip containing the compiled C# .NET libraries.

New version of Babel Business Edition with attachments is out now!

So what is new?

We have just released a new version of our app which now includes some cool new features. The most important change is the ability to send and receive attachments, this includes any type of file but you can also capture images, video and audio to send as an attachment if your device has the capabilities.

Other improvements include displaying a contact's current status, a green icon next to the contact's name indicates they are online and a grey icon indicates they are offline. We have also introduced a new feature called 'message expiration', which gives you the opportunity to set a time period after which your message should expire if not delivered. Lastly you can set to be notified if your message has fallen into an "unsent" status – making sure you don’t miss out on actually sending your message!

Go to the App Store and Google Play to update your current version of Babel and get these cool new features.

Ignore certificate errors on Windows & Windows Phone

Introduction

Our team is also currently working on .NET version of secured messaging application Babel. To put it simply, main goal is to port great and useful Android & iOS app to Windows desktop (Vista, 7, 8), Windows 8.1 (Metro) and Windows Phone 8.1 - both known as "Universal apps".

At first, I'll try to explain the problem. Except the mentioned Babel, we've got also the second version, targeted to corporations and called Babel Bussiness Edition. This one is based on fact, that every company owns their private server and all communication comes through this one, so there is no third-party server. There is an authentication based on certificate, which is signed by our company. Essentialy, when user connects to server we needs to verify server identity and depending on that enable or disable the communi- cation. Of course, .NET framework does not trust certificates, which he cannot find or successfully compare with any from root trusted certification authorities. When you will try to initialize server connection secured with these type of certs, an exception will be raised. In this blog post, I will try to describe how to easily ignore certificate errors on:

• Windows desktop client (WPF, WinForms etc.)
• Win 8 (Metro), Windows Phone 7, WP8, WP8.1 Silverlight
• Windows 8.1, Windows Phone 8.1 RT - Universal apps

Windows desktop client

Pretty easy, well known stuff:

//Set certificate validation delegate
private void SetCertValidationDel()
{
  //you can use ServicePointManager class
  System.Net.ServicePointManager.ServerCertificateValidationCallback += CertValidation;
}

//This method will be raised, when initializing https connection
private static bool CertValidation(
object sender,
System.Security.Cryptography.X509Certificates.X509Certificate cert,
System.Security.Cryptography.X509Certificates.X509Chain chain,
System.Net.Security.SslPolicyErrors sslPolicyErrors)
    {
      return true;
      //All cert will be considered as valid - recommended for testing.
      //Or your own implementation of cert validation.
    }
}

Win 8, WP7, WP8, WP8.1 (Silverlight)

You simply cannot use previous example, because missing some classes like ServicePointManager, X509Chain etc. I've found two solutions for this problem:

• Install certificate to root cert. store of device.

Upload the certificate somewhere on the internet (mail, cloud service etc.) and access it directly from concrete device. The cert will be installed automatically to device root certification store.

• Install certificate to root store of application.

Using certificate extensions destribed here.

You'll need to export server cert to DER file, copy to application, set build action to "Content" and always copy to output directory. The last step is to edit package.appxmanifest file.

Windows 8.1, WP8.1 RT - Universal apps

Up until Windows 8.1 & WP8.1 you could not ignore certificate errors in Windows Store apps with classic System.Net.Http.HttpClient class. Now you can do this with new  namespace Windows.Web.Http and Windows.Web.Http.Filters.

//Initialize new instance of HttpBaseProtocolFilter, which implements IHttpFilter.
var filter = new HttpBaseProtocolFilter();
//Untrusted and Expired cert. errors will be ignored in this example:
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Untrusted);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Expired);
//Initialize new instance of HttpClient with IHttpFilter parameter.
var httpClient = new Windows.Web.Http.HttpClient(filter);

The ChainValidationResult is simple Enum and a list of possible values can be found here. Note that you should use this technique only in very limited type of scenarios (tests etc.). You should ever ensure that target you are hitting is the correct endpoint instead of ignore certificate errors.

Applet in a spring web application

Another application our team is developing right now is a card access system. The system is quite complex, but the main idea is very simple. User is registered through the web application and receives a card. The card contains access information and when he comes to the door and uses the card, permissions are checked and he is either allowed to open the door or the access is denied.

The web application uses spring framework (3.2.4) and we use maven as the build tool. The main problem was to figure out how to communicate with the card through the web browser. Finally we found out that our only solution is to use java applets. But it’s much easier said than done.

In this article we’d like to cover all the steps needed to run java applet which uses clients’ system resources (such as a smart card reader), which may help you when you decide to use this deadly combination, too.

JNLP

Suppose, you have an applet built to a jar file called test-applet.jar (you can try to write some simple applet as an exercise).

The recommended way to launch applets on the webpage is by using the Java Network Launch Protocol (JNLP). Our test.jnlp file for this simple example looks like this:

<?xml version="1.0" encoding="UTF-8"?>

<jnlp spec="1.0+" codebase="" href="">

  <information>

    <title>Test applet</title>

    <vendor>OKsystem s.r.o.</vendor>

    <offline-allowed/>

  </information>

  <resources>

    <!-- Application Resources -->

    <j2se version="1.6+"

                  href="http://java.sun.com/products/autodl/j2se" />

    <jar href="/SpringAppletTest/applet/test-applet.jar" main="true" />

  </resources>

  <security>

    <all-permissions/>

  </security>

  <applet-desc

        name="Test Applet"

        main-class="cz.oksystem.spring_applet_test.testapplet.TestApplet"

        width="800"

        height="550">

  </applet-desc>

  <update check="background"/>

</jnlp>

It contains basic information about the application, path to the .jar file, security information and the applet-desc element to run the applet.

Since we need our applet to work with a smart card reader, we also need the permissions to user’s computer resources. That’s why the all-permissions option is setup (the other possible option is sandbox).

JSP

The next step is to prepare the applet.jsp page which uses the jnlp file to run the applet:

<%@page contentType="text/html" pageEncoding="UTF-8"%>

<!DOCTYPE html>

<html>

  <head>

    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

    <script src="https://www.java.com/js/deployJava.js">             </script>   

    <title>Test JSP Page</title>

  </head>

  <body>

    <h1>Hello World!</h1>

       

    <div>

      <script>                   

        var attributes = {

          code: 'cz.oksystem.spring_applet_test.testapplet.TestApplet', archive: '/SpringAppletTest/applet/test-applet.jar', width: 800, height: 550};

          var parameters = {jnlp_href: '/SpringAppletTest/applet/test.jnlp', java_status_events: true};

                deployJava.runApplet(attributes, parameters, '1.7');                   

      </script>

    </div>

  </body>

</html>

The core functionality of the page lies between the script tags. First, we import the deployJava javascript from the java web page. Deploy-java.js (Deployment Toolkit script) contains useful functions that can be used to deploy applets in a web page, in this example the runApplet function. You pass to this function attributes and parameters containing information about the applet code base, archive which contains the compiled source and the jnlp location.

If you need to pass some additional parameters that the applet should use, just add them to the parameters object as the key-value pair and use the getParameter() function in your applet java code.

Manifest

When you create a JAR file, a default manifest file is created automatically (MANIFEST.MF). It simply contains the “header: value” pairs information about the file. To modify the manifest file, you must first prepare a text file containing the information you wish to add to the manifest.

Our sample manifest file manifest.txt contains following information:

Permissions: all-permissions

Codebase: *

Application-Name: test-applet

Trusted-Only: true

Trusted-Library: true

Caller-Allowable-Codebase: *
Application-Library-Allowable-Codebase: *

We will not give the full description of the file properties, because it’s not the aim of this article. You can find all the possible attributes with their detailed description in the oracle documentation page: 
http://docs.oracle.com/javase/8/docs/technotes/guides/jweb/security/manifest.html

Maven manifest

As I have already mentioned in the beginning of this tutorial, we use maven as our build tool for this project. By default, maven archiver generates the manifest file for you. But in this particular situation we need to use our own manifest file that we have created in the previous step. This is done with the <manifestFile> configuration element by setting its value to the location of the manifest file. The content of this file will be merged with the entries generated by Maven Archiver. 

<build>

    <plugins>

        <plugin>

            <groupId>org.apache.maven.plugins</groupId>

            <artifactId>maven-jar-plugin</artifactId>

            <configuration>

                <archive>

        <manifestFile>src/main/resources/META-INF/manifest.txt</manifestFile>

                </archive>

            </configuration> 

        </plugin>

    </plugins>

</build>

Spring-servlet.xml

The only 2 lines you have to add to your usual spring configuration file are the resources where the applet and javascript are located:

    <mvc:resources mapping="/applet/**" location="/applet/"/>   

    <mvc:resources mapping="/js/**" location="/js/"/>    

Complete project structure

Just for completeness, we attach also the structure of our applet and web application projects.

Signing the applet

At this point, when you build, deploy and run your web application, the following error is going to appear:

This happens because the applet you are using have to be signed. To self-sign the applet follow these 3 steps (don’t forget to use real certificates for production):

   

     1. keytool -genkey -keystore myKeyStore -alias testuser

     2. keytool -selfcert -keystore myKeyStore -alias testuser

     3. jarsigner -keystore myKeyStore test-applet.jar testuser

Because in this example the applet is self-signed you have to change the java settings to allow running also applications which are not signed by the trusted authority. To do this, go to Control panel -> Java -> Settings -> Security and change the security level to medium.

As the note under the security level slider says, “All Java applications will be allowed to run after presenting a security prompt”. I know it’s a very uncomfortable step, but when you realize that the applet has access to your system resources (in our case the card reader connected to the users’ computer), you also realize that this can be a perfect means to attack the clients computer. And that’s the main reason behind it.

After completing these steps you will be able to successfully deploy and run the application.

Futher notes

After completing these steps you should be able to run the applet in a spring web application. Similar steps may be used to make it run on the classic JavaEE platform.

Before we finish this tutorial, we want to point out 2 more issues you would most probably encounter while developing applets for the web applications.         

Applet cache

The first issue is the java applet cache. When developing java applets, you may come to a situation when you change some applet code, but when running it in the web browser, nothing changes even when you restart the browser to clear its cache (or clear it manually). The thing about this is a fact, that your browser is not responsible for caching java applets, java does it by itself. The reasons for caching applets are obvious. There is no need for downloading applets every time a user references them, which results also in their startup performance (but sometimes causes headaches to us, programmers). 

To clear java cache go to Control panel -> Java -> Settings -> Delete files. Load your web application again and your applet gets fully loaded again, including all of the changes.

Multiple resources in JNLP

The second issue is that sometimes you may need to use multiple resources in your JNLP file. In our case our card reading applet is using a common module with some cryptographic functions implemented there. The solution is to sign the module, build it and add it to the applet folder of your web page. Then reference this jar in resources element in the JNLP file.

...
  <resources>
    <!-- Application Resources -->
    <j2se version="1.6+" href="http://java.sun.com/products/autodl/j2se" />
    <jar href="/SpringAppletTest/applet/test-applet.jar" main="true" />
    <jar href="/SpringAppletTest/applet/some-module.jar" />
  </resources>
...

We hope this tutorial helps you at least a bit and we wish you a happy coding.

Android HTTP connection reuse bug

Despite of summer, hot weather, or holidays, our team is still hard working on secured messaging application Babel. We have been developing new feature last month, attachments. The goal is to allow users sending attachments, as pictures, videos, documents or other data files. The attachments are stored at Babel BE server and the users should be able to download them whenever they want to. And moreover, just like text messages, the attachments are end-to-end encrypted and secured the whole time of the transfer.

While developing this on Android, an interesting problem was found. The data are transferred divided to chunks and javax.net.ssl.HttpsURLConnection is used to transfer them, one by one. The server has REST interface and to transfer of every chunk is required to create a new Uri, with the right offset.

After implementing, on some devices exception as follows was thrown:

   java.net.SocketException: socket failed: EMFILE (Too many open files) 

After some research, we found this Android issue, which could have been our problem:

https://code.google.com/p/android/issues/detail?id=38817

For better understanding, I’ll give you little explanation what reusing of HTTP connection is.

The idea of HTTP persistence connection, also called HTTP keep-alive is to use a single TCP connection to send and receive multiple HTTP requests/ responses instead of opening a new connection for every request/ response pair.

Then we figured out the solution, which was simply disabling the reusing of HTTP connection by:

     connection.setRequestProperty("Connection", "close");

After disabling it, there is no effort to reuse the connection, so every connection which is made is simply closed after every request.

Crashing of the app was fixed so it remained to find out whether there were any performance issues with having the reusing of HTTP connection disabled.

1.  At first, we were measuring the speed of data transfer. We didn´t find any relevant difference between data transfer speed with reusing enabled and disabled.
2.    Then we have been measuring the data transfer volume. The results are as follows:
1.  Upload: the cost of transfer is increased by 60 B with reusing disabled.
2. Download: the cost of transfer is increased by 200 B with reusing disabled.

The increasing of data transfer cost may look as a problem, but when you consider that size of our chunk is 2 MB when connected to WIFI or 100 KB when connected to mobile network, it isn’t problem at all.

For example, if you would download 100 MB file while connected to mobile network, the increase of data volume would be 200 KB, while connected to WIFI, the increase would be only 10 KB.