iOS

Core Data Basics: Preload Data and Use Existing SQLite Database


Editor’s note: This is a sample chapter of our new book, Intermediate iOS 8 Programming with Swift.

When working with Core Data, you may have asked these two questions:

  • How can you preload existing data into the SQLite database?
  • How can you use an existing SQLite database in my Xcode project?

I recently met a friend who is now working on a dictionary app for a particular industry. He got the same questions. He knows how to save data into the database and retrieve them back from the Core Data store. The real question is: how could he preload the existing dictionary data into the database?

I believe some of you may have the same question. This is why I devote this tutorial to talk about data preloading in Core Data. I will answer the above questions and show you how to preload your app with existing data.

So how can you preload existing data into the built-in SQLite database of your app? In general you bundle a data file (in CSV or JSON format or whatever format you like). When the user launches the app for the very first time, it preloads the data from the data file and puts them into the database. At the time when the app is fully launched, it will be able to use the database, which has been pre-filled with data. The data file can be either bundled in the app or hosted on a cloud server. By storing the file in the cloud or other external sources, this would allow you to update the data easily, without rebuilding the app. I will walk you through both approaches by building a simple demo app.

Once you understand how data preloading works, I will show you how to use an existing SQLite database (again pre-filled with data) in your app.

Note that I assume you have a basic understanding of Core Data. You should know how to insert and retrieve data through Core Data. If you have no ideas about these operations, you can refer to our book, Beginning iOS 8 Programming with Swift or refer to this tutorial (in Objective-C).

A Simple Demo App

To keep your focus on learning data preloading, I have created the project template for you. Firstly, download the project and have a trial run.

It’s a very simple app showing a list of food. By default, the starter project comes with an empty database. When you compile and launch the app, your app will end up a blank table view. What we are going to do is to preload the database with existing data.

Core Data Preloading Demo

I have already built the data model and provided the implementation of the table view. You can look into the MenuItemTableViewController class and CoreDataDemo.xcdatamodeld for details. The data model is pretty simple. I have defined a MenuItem entity, which includes three attributes: name, detail, and price.

Once you’re able to preload the database with the food menu items, the app will display them accordingly, with the resulting user interface similar to the screenshot shown on the left.

The CSV File

In this demo I use a CSV file to store the existing data. CSV files are often used to store tabular data and can be easily created using text editor, Numbers or MS Excel. They are sometimes known as comma delimited files. Each record is one line and fields are separated with commas. In the project template, you should find the “menudata.csv” file. It contains all the food items for the demo app in CSV format. Here is a part of the file:

The first field represents the name of the food menu item. The next field is the detail of the food, while the last field is the price. Each food item is one line, separated with a new line separator.

coredata-preload

Parsing CSV Files

It’s not required to use CSV files to store your data. JSON and XML are two common formats for data interchange and flat file storage. As compared to CSV format, they are more readable and suitable for storing structured data. Anyway, CSV has been around for a long time and is supported by most spreadsheet applications. At some point of time, you will have to deal with this type of file. So I pick it as an example. Let’s see how we can parse the data from CSV.

The AppDelegate object is normally used to perform tasks during application startup (and shutdown). To preload data during the app launch, we will add a few methods in the AppDelegate class. First, insert the following method for parsing the CSV file:

The method takes in three parameters: the file’s URL, encoding and an error pointer. It first loads the file content into memory, reads the lines into an array and then performs the parsing line by line. At the end of the method, it returns an array of food menu items in the form of tuples.

A simple CSV file only uses a comma to separate values. Parsing such kind of CSV files shouldn’t be difficult. You can call the componentsSeparatedByString method (highlighted in yellow in the code snippet) to split a comma-delimited string. It’ll then return you an array of strings that have been divided by the separator.

For some CSV files, they are more complicated. Field values containing reserved characters (e.g. comma) are surrounded by double quotes. Here is another example:

In this case, we cannot simply use the componentsSeparatedByString method to separate the field values. Instead, we use NSScanner to go through each character of the string and retrieve the field values. If the field value begins with a double quote, we scan through the string until we find the next double quote character by calling the scanUpToString method. The method is smart enough to extract the value surrounded by the double quotes. Once a field value is retrieved, we then repeat the same procedure for the remainder of the string.

After all the field values are retrieved, we save them into a tuple and then put it into the “items” array.

Preloading the Data and Saving it into database

Now that you’ve created the method for CSV parsing, we now move onto the implementation of data preloading. The preloading will work like this:

  1. First, we will remove all the existing data from the database. This operation is optional if you can ensure the database is empty.
  2. Next, we will call up the parseCSV method to parse menudata.csv.
  3. Once the parsing completes, we insert the food menu items into the database.

Add the following code snippets into the AppDelegate.swift file:

The removeData method is used to remove any existing menu items from the database. I want to ensure the database is empty before populating the data extracted from the menudata.csv file. The implementation of the method is very straightforward if you have a basic understanding of Core Data. We first execute a query to retrieve all the menu items from the database and call the deleteObject method to delete the item one by one.

Okay, now let’s talk about the preloadData method.

In the method we first retrieve the file URL of the menudata.csv file using this line of code:

After calling the removeData method, we execute the parseCSV method to parse the menudata.csv file. With the returned items, we insert them one by one by calling the NSEntityDescription.insertNewObjectForEntityForName method.

Lastly, execute the preloadData() method in the didFinishLaunchingWithOptions method:

Now you’re ready to test your app. Hit the Run button to launch the app. If you’ve followed the implementation correctly, the app should be preloaded with the food items.
But there is an issue with the current implementation. Every time you launch the app, it preloads the data from the CSV file. Apparently, you only want to perform the preloading once. Change the application:didFinishLaunchingWithOptions: method to the following:

To indicate that the app has preloaded the data, we save a setting to the defaults system using a specific key (i.e. isPreloaded). Every time when the app is launched, we will first check if the value of the “isPreloaded” key. If it’s set to true, we will skip the data preloading operation.

Using External Data Files

So far the CSV file is bundled in the app. If your data is static, it is completely fine. But what if you’re going to change the data frequently? In this case, whenever there is a new update for the data file, you will have to rebuild the app and redeploy it to the app store.

There is a better way to handle this.

Instead of embedding the data file in the app, you put it in an external source. For example, you can store it on a cloud server. Every time when a user opens the app, it goes up to the server and download the data file. Then the app parses the file and loads the data into the database as usual.

I have uploaded the sample data file to the Amazon cloud server. You can access it through the URL below:

This is just for demo purpose. If you have your own server, feel free to upload the file to the server and use your own URL. To load the data file from the remote server, all you need to do is make a little tweak to the code. First, update the preloadData method to the following:

The code is very similar to the original one. Instead loading the data file from the bundle, we specify the remote URL and pass it to the parseCSV method. That’s it. The parseCSV method will handle the file download and perform the data parsing accordingly.
Before running the app, you have to update the application:didFinishLaunchingWithOptions: method so that the app will load the data every time it runs:

You’re ready to go. Hit the Run button and test the app again. The menu items should be different from those shown previously.

For your reference, you can download the complete Xcode project here.

Using An Existing Database in Your Project

Now that you should know how to populate a database with external data, you may wonder if you can use an existing SQLite database directly. In some situations, you probably do not want to preload the data during app launch. For example, you need to preload hundreds of thousands of records. This will take some time to load the data and results a poor user experience.

Apparently, you want to pre-filled the database beforehand and bundle it directly in the app.
Suppose you’ve already pre-filled an existing database with data, how can you bundle it in your app?

Before I show you the procedures, please download the starter project again. As a demo, we will copy the existing database created in the previous section to this starter project.

Now open up the Xcode project that you have worked on earlier. If you’ve followed me along, your database should be pre-filled with data. We will now copy it to the starter project that you have just downloaded.

But where is the SQLite database?

The database is not bundled in the Xcode project but automatically created when you run the app in the simulator. To locate the database, you will need to add a line of code to reveal the file path. Update the application:didFinishLaunchingWithOptions: method to the following:

The SQLite database is generated under the application’s document directory. To find the file path, we simply print out the value of applicationDocumentsDirectory.path variable.

Now run the app again. You should see an output in the console window showing the full path of the document directory.

Core Data - Locate SQLite Database

Copy the file path and go to Finder. In the menu select Go > Go to Folder… and then paste the path in the pop-up. Click “Go” to confirm.

Finder - Locate Database

Once you open the document folder in Finder, you will find three files: CoreDataDemo.sqlite, CoreDataDemo.sqlite-wal and CoreDataDemo.sqlite-shm.

Starting from iOS 7, the default journaling mode for Core Data SQLite stores is set to Write-Ahead Logging (WAL). With the WAL mode, Core Data keeps the main .sqlite file untouched and appends transactions to a .sqlite-wal file in the same folder. When running WAL mode, SQLite will also create a shared memory file with .sqlite-shm extension. In order to backup the database or use it to in other projects, you will need copy these three files. If you just copy the CoreDataDemo.sqlite file, you will probably end up with an empty database.

Now, drag these three files to the starter project in Xcode.

Preload sqlite database

When prompted, please ensure the “Copy item if needed” option is checked and the “CoreDataPreloadDemo” option of “Add to Targets” is selected. Then click “Finish” to confirm.

Import Core Data Database into Xcode Project

Now that you’ve bundled an existing database in your Xcode project. When you build the app, this database will be embedded in the app. But you will have to tweak the code a bit before the app is able to use the database.

By default, the app will create an empty SQLite store if there is no database found in the document directory. So all you need to do is copy the database files bundled in the app to that directory. In the AppDelegate class update the declaration of the persistentStoreCoordinator variable like this:

The changes are highlighted in yellow. We first verify if the database exists in the document folder. If not, we copy the SQLite files from the bundle folder to the document folder by calling the copyItemAtURL method of NSFileManager.

That’s it! Before you hit the Run button to test the app, you better delete the CoreDataPreloadDemo app from the simulator or simply reset it (select iOS Simulator > Reset Content and Settings). This is to remove any existing SQLite databases from the simulator.

Okay, now you’re good to go. When the app is launched, it should be able to use the database bundled in the Xcode project.

For reference, you can download the final Xcode project here.

Editor’s note: This is a sample chapter of our new book, Intermediate iOS 8 Programming with Swift. If you like this tutorial, you can check out the book here or get the starter package of our Swift book package.

iOS
A Swift Tutorial for Google Maps SDK
iOS
Working With Background Fetch in iOS 7
iOS
Build a Simple Camera App Using UIImagePickerController
Shares