Syncing Contacts with an Android Application

It is nice to be able to access everything from one place. This is analogy does not apply only in the real but in the world of IT as well. This blog features a topic that is based on this analogy. You might have noticed how various famous applications that connect to your contacts provide you with the option to launch various operations from your phone contacts app itself. For example if you go to your contacts app, you can see that in your contacts there are the options to send message, voice call or video call with any famous android app you like. You can send a message or make a call from one place.

In this post, I will explain an android application that achieves the above task along with code snippets. It is an android application that will sync contact numbers in the phone book with a server and also provide options such send message and calls from the phone book itself. The application code is available here. Following is a screenshot of a contact synced with the application.

Introduction

Contacts syncing has been around for very long but still there are not many open-source implementations of it out there. I have already written a blog about it some years back but it mostly works to introduce the topic. In this post, I intend to share some more details about the code that runs to achieve the goal. And also this post includes some interactions with server to sync data.

Let's start by describing the main components that you will need to implement for syncing. These are:

1. Contacts Provider - Contacts provider is a content provider that acts as the central repository of all data about the people on the android device.
2. Authentication Service - Authentication service works to authenticate the user usually through an OAuth framework when we create an account for the app on the device.
3. Sync Service - Sync service is the service that synchronizes data between the device and a server.

Now I will describe each component in detail.

Contacts Provider

As mentioned before, it is a content provider basically and acts as the central repository for all data about people. It maintains three types of data about a person, each of which corresponds to a table in it. Hence we have three-tier data model:

                                                  Contacts provider table structure (Source)

These tables go as follows:

• A row in the Data table stores personal data like phone number or email address. This table has predefined set of common kinds of data but you can define your custom kind of data as well.
• A row in RawContacts table is an aggregate of rows in Data table. It represents the data of a person which is associated with a single account type.
• A row in Contacts table is an aggregate of rows in RawContacts table. It represents a person in the devices phone book.

Contacts Provider table relationships (Source)

Other than the above mentioned tables, we are interested in another table called AggregationExceptions which is useful for manual aggregation of contacts. We will find out more about this later.

ContactsContract class acts as the contract between the Contacts Provider and the applications. Through this class the tables that are used by the provider are modified. ContactsContract.Contacts, ContactsContract.RawContacts and ContactsContract.Data are the classes that the applications can use to modify the tables mentioned above.

ContactsContract.Data

Here is a code snippet from the app that will help understand Data class better:

Snippet
Now lets see what is happening here. You will notice that we are running a query first. We are querying the ContactsContract.Data.CONTENT_URI which is the uri for the data table. We are looking for all the numbers that belong to a contact id. First of all how is the data stored in Data table?  Each row in data table contains MIME type of the data, the data itself in columns DATA1 through to DATA15 and the contact id, rawcontact id etc. So you see in the code we are querying for the number where CONTACT_ID matches our contact id and MIMETYPE is of type Phone.

But.. wait a minute what are we querying for? Shouldn't it be something like DATA1, DATA2.. so on. Yes it should be but this is where we can make use of the CommonDataKinds class which is a container for Data kinds in the Data table. So when we query for CommonDataKinds.Phone.NUMBER we are actually querying for DATA1 type because the number is top priority for Phone MIMETYPE. For CommonDataKinds.Email, CommonDataKinds.Email.ADDRESS will be DATA1. So these are nice aliases we can use instead of having to delve into too much data1, data2 etc.

ContactsContract.RawContacts

So that was the Data class, now we see RawContacts class:

Snippet
In the same way as the Data class, we query the RawContacts class uri to get all rawcontact id's for a contact id.

ContactsContract.Contacts

Finally let's see the code for Contacts class:

Snippet
Here we query the Contacts class uri to get the name for a contact id. Now I hope we have a better understanding about how the Contacts Provider stores the data on the device.

ContentProviderOperation

So far we were just observing data with queries, now we do more than just spectate by inserting, updating and deleting data. We start by introducing ContentProviderOperation class which represents a single operation to be performed on the Contacts Provider tables. We can do an insert, an update or a delete operation. We'll see some code from the app explaining these operations:

ContactsManager.kt
Above code is the ContactsManager class from the app. It performs insertion and deletion of a rawcontact corresponding to a number.

Insertion is performed in the registerNumber() method through a list of multiple newInsert() methods. First insert occurs on the RawContacts.CONTENT_URI where we specify the account name and type to be inserted along with aggregation mode. Then we insert into Data.CONTENT_URI where we specify the MIMETYPE as Phone.CONTENT_ITEM_TYPE and Phone.NUMBER as the number value to be inserted. Here you see Phone.NUMBER is simply an alias for DATA1. Finally, we insert into Data.CONTENT_URI where we specify MIMETYPE as a type we specify in MESAGE_ITEM_TYPE and DATA1, DATA2, DATA3. Similar to MESSAGE_ITEM_TYPE, there are also VOICE_ITEM_TYPE and VIDEO_ITEM_TYPE.

So what is happening in the final three inserts. We see that we are inserting a custom mimetype along with our own values for data1, data2 and data3. This will create an entry in data table for our app's custom type of data. Now that we have added insert operations to the list, let's see what is happening overall. First insert created an entry in rawcontacts table which is the new entry we want to create for our app in the device's phone book. Second insert is the number we want to associate our insert with. You see android OS will check in the background that does the number match any previous one and if it does it will aggregate them together which is to say we will see them under the same contact in phone book. The final three inserts are the app data we want to show in the phone book. (These are what you saw in the first screenshot).

Another important part of the insert operation is withValueBackReference() which I have specified as 0. Basically what I understood from the android developer docs is that we pass the index of the operation in which we create the rawcontact id. So I did the rawcontact insert in the first operation thus the value of 0. Finally, there is the addCallerIsSyncAdapter() method where if isSyncOperation is true then android OS knows that the contacts are being updated with values from the server and not local values. Hence it will not try to perform a redundant sync back to server. I set it to true because my caller is indeed SyncAdapter.

Onward we see that the class also contains a deleteNumber() method which is the deletion of a rawcontact from the RawContacts table. We first find all rawcontact id's for the number and then we filter them for the one with our app's account type. Once we have the rawcontact id we perform the newDelete() operation where we specify it as the RawContacts._ID.

Once we have formed our list of ContentProviderOperations, we call the ContentResolver.applyBatch() method to apply all the operations as a batch. It takes two arguments ContactsContract.AUTHORITY and list of operations we built earlier. Contacts Provider is as I mentioned before is a Content Provider so whenever we want to do changes we must supply the authority of the provider.

There are other alternatives to batch operations but I find batch operations rather handy.

So all that was good by the books process of operating on Contacts Provider. But something I learned from my previous post is that going just by the books doesn't work as often as we would want it to. People reported to me that there new contacts created in the phone book for the rawcontacts that they added. I myself found it evident on quite a few devices when I looked into it. Reason was obvious, they we not being aggregated properly. So I tried to find a solution to this which was AggregationExceptions. The purpose of this table is to store rawcontacts that are meant to be aggregated. It's each row stores two column's which are RAW_CONTACT_ID1 and RAW_CONTACT_ID2 which are the two rawcontact id's to be aggregated. Also there is the TYPE column that should store TYPE_KEEP_TOGETHER if you want to aggregate the two rawcontacts.

What I basically did was take things into my own hands and did a manual aggregation. So far so good, now we know how to do manual aggregation but what to aggregate to achieve what we want. Let's focus on the registerNumber() method and go to the last part where you would notice that we derived the new rawcontact id for the operations we performed. Okay, so we have one rawcontact id. Now what about the other one. The other one is the already existing rawcontact id of the number we want to sync. Let's see a code snippet:

Snippet
In the above code, I am extracting the names and numbers of all contacts in the phone book. Also we are getting rawcontact id's map for all numbers. What is happening is I am finding the first rawcontact id that corresponds to the number. Now if I aggregate this rawcontact id with the new one in the registerNumber() method we have the result we want. No more obnoxious new contacts in the phone book and perfectly synced contacts.

Note: I found out that if you aggregate the new rawcontact id with any of the rawcontact id's of the number it will work just fine but still I just aggregated them to first rawcontact id of the number.

I have built the app around numbers but you can also build it around email address. If you go for email address just replace newInsert() for number with email address and when you find the rawcontact id to aggregate with the new one, use the the rawcontact id for email address instead of number.

Authentication

Authenticator is usually used for services that use OAuth for user verification. Although my application does not use OAuth, you still have to implement it as it is required by the Sync service. For my application, I made a stub Authenticator class which will allow the application to work properly. Here is the code:

Authenticator.kt

Now we need a service that binds to the Authenticator to listen for the AccountAuthenticator broadcast.

AuthenticationService.kt

Next we need to setup metadata for the AuthenticationService. We do this by defining an xml resource file named authenticator.xml. Here is the code for it:

authenticator.xml
Manifest snippet:
So we have all the stub code we needed, now we can get to account management. Android allows applications to create their account on the device. Every application that wants to access the device for syncing must first create an account on the device. All the features like contacts, calendars etc. are synced and managed under the same account this way. Here is the MainActivity class that creates the app account on the device:

MainActivity.kt

As you can observe, the addAccountExplicitly() method adds the app account to the device. If you go to settings and select Accounts on your device, you will find that many famous applications you know are registered with your device. Your app will be registered as well after the above method runs. setSyncAutomatically() method enables sync for Contacts Provider when the account is created. addPeriodicSync() sets up a periodic sync for the account, which calls the SyncAdapter after the SYNC_INTERVAL specified (minimum is about 15 minutes). You can also perform a manual sync by calling the requestSync() method.

For those of you who want to use OAuth in your app, you can make use of an AuthenticatorActivity which will be launched once the user tries to add an account. You can find out more about this here.

Sync Service

Sync service is responsible for handling data sync between the device and the server. All the action happens here whenever someone tries to refresh contact data. The class that handles this logic is the SyncAdapter. Here is the code:

SyncAdapter.kt

As you can observe, onPerformSync() method does the sync with the server. This method is run on a background thread by the sync framework. In the method, I just fetch the data from the server and cross check with the device contacts to look for numbers that might not be in sync. Then I use the ContactsManager classes' registerNumber() and deleteNumber() to do the syncing. Sync service binds to the sync adapter and listens for the SyncAdapter broadcast. Also there is a metadata xml file named syncadapter.xml. This file handles contains meta-data like app's account type etc. Here is the code for these:

SyncService.kt
sync_adapter.xml
Manifest snippet:

It is all mostly the same as the authenticator but for an xml file called contacts.xml if you observe the manifest file closely. This contacts,xml file specifies which data is shown to user in the phone book. We previously saw how we added data1, data2 and data3 to the data table. Now which data out of all that is actually visible to the user is decided by this file. Here is the snippet:

contacts.xml

You can see there is a ContactDataKind entry for each mimetype we inserted earlier on. As you can also see that there are also attributes for icon, summaryColumn and detailColumn. Icon is the app launcher icon and the data2 for summaryColumn, data3 for detailColumn.

At the end, you also need to build an activity that will be launched once you click on the app's data in the phone book. You will need one activity each for each mimetype you create. Here is a code snippet:

Manifest snippet:
Here is the code for MessageActivity:
Notice that I have accessed the Data1, Data2 and Data3 in the activity. Data3 is what gets displayed in the phones contact list. You can set Data1 and Data2 to whatever you like and then retrieve these values in your actvity by making a query with the uri you get in the intent.

And that's it! You can checkout the application on github. I also built a very simple node server which I sync contact numbers with. I hope you have a better understanding about the syncing contacts and wish you best of luck. Thank you.

References:

1. Android Developers: Contacts Provider
2. Android Developers: Authenticator
3. Android Developers: SyncAdapter