iOS

Core Data Basics: Preload Data and Use Existing SQLite Database


Editor’s note: This is a sample chapter of our new book, Intermediate iOS 8 Programming with Swift.

When working with Core Data, you may have asked these two questions:

  • How can you preload existing data into the SQLite database?
  • How can you use an existing SQLite database in my Xcode project?

I recently met a friend who is now working on a dictionary app for a particular industry. He got the same questions. He knows how to save data into the database and retrieve them back from the Core Data store. The real question is: how could he preload the existing dictionary data into the database?

I believe some of you may have the same question. This is why I devote this tutorial to talk about data preloading in Core Data. I will answer the above questions and show you how to preload your app with existing data.

So how can you preload existing data into the built-in SQLite database of your app? In general you bundle a data file (in CSV or JSON format or whatever format you like). When the user launches the app for the very first time, it preloads the data from the data file and puts them into the database. At the time when the app is fully launched, it will be able to use the database, which has been pre-filled with data. The data file can be either bundled in the app or hosted on a cloud server. By storing the file in the cloud or other external sources, this would allow you to update the data easily, without rebuilding the app. I will walk you through both approaches by building a simple demo app.

Once you understand how data preloading works, I will show you how to use an existing SQLite database (again pre-filled with data) in your app.

Note that I assume you have a basic understanding of Core Data. You should know how to insert and retrieve data through Core Data. If you have no ideas about these operations, you can refer to our book, Beginning iOS 8 Programming with Swift or refer to this tutorial (in Objective-C).

A Simple Demo App

To keep your focus on learning data preloading, I have created the project template for you. Firstly, download the project and have a trial run.

It’s a very simple app showing a list of food. By default, the starter project comes with an empty database. When you compile and launch the app, your app will end up a blank table view. What we are going to do is to preload the database with existing data.

Core Data Preloading Demo

I have already built the data model and provided the implementation of the table view. You can look into the MenuItemTableViewController class and CoreDataDemo.xcdatamodeld for details. The data model is pretty simple. I have defined a MenuItem entity, which includes three attributes: name, detail, and price.

Once you’re able to preload the database with the food menu items, the app will display them accordingly, with the resulting user interface similar to the screenshot shown on the left.

The CSV File

In this demo I use a CSV file to store the existing data. CSV files are often used to store tabular data and can be easily created using text editor, Numbers or MS Excel. They are sometimes known as comma delimited files. Each record is one line and fields are separated with commas. In the project template, you should find the “menudata.csv” file. It contains all the food items for the demo app in CSV format. Here is a part of the file:

The first field represents the name of the food menu item. The next field is the detail of the food, while the last field is the price. Each food item is one line, separated with a new line separator.

coredata-preload

Parsing CSV Files

It’s not required to use CSV files to store your data. JSON and XML are two common formats for data interchange and flat file storage. As compared to CSV format, they are more readable and suitable for storing structured data. Anyway, CSV has been around for a long time and is supported by most spreadsheet applications. At some point of time, you will have to deal with this type of file. So I pick it as an example. Let’s see how we can parse the data from CSV.

The AppDelegate object is normally used to perform tasks during application startup (and shutdown). To preload data during the app launch, we will add a few methods in the AppDelegate class. First, insert the following method for parsing the CSV file:

The method takes in three parameters: the file’s URL, encoding and an error pointer. It first loads the file content into memory, reads the lines into an array and then performs the parsing line by line. At the end of the method, it returns an array of food menu items in the form of tuples.

A simple CSV file only uses a comma to separate values. Parsing such kind of CSV files shouldn’t be difficult. You can call the componentsSeparatedByString method (highlighted in yellow in the code snippet) to split a comma-delimited string. It’ll then return you an array of strings that have been divided by the separator.

For some CSV files, they are more complicated. Field values containing reserved characters (e.g. comma) are surrounded by double quotes. Here is another example:

In this case, we cannot simply use the componentsSeparatedByString method to separate the field values. Instead, we use NSScanner to go through each character of the string and retrieve the field values. If the field value begins with a double quote, we scan through the string until we find the next double quote character by calling the scanUpToString method. The method is smart enough to extract the value surrounded by the double quotes. Once a field value is retrieved, we then repeat the same procedure for the remainder of the string.

After all the field values are retrieved, we save them into a tuple and then put it into the “items” array.

Preloading the Data and Saving it into database

Now that you’ve created the method for CSV parsing, we now move onto the implementation of data preloading. The preloading will work like this:

  1. First, we will remove all the existing data from the database. This operation is optional if you can ensure the database is empty.
  2. Next, we will call up the parseCSV method to parse menudata.csv.
  3. Once the parsing completes, we insert the food menu items into the database.

Add the following code snippets into the AppDelegate.swift file:

The removeData method is used to remove any existing menu items from the database. I want to ensure the database is empty before populating the data extracted from the menudata.csv file. The implementation of the method is very straightforward if you have a basic understanding of Core Data. We first execute a query to retrieve all the menu items from the database and call the deleteObject method to delete the item one by one.

Okay, now let’s talk about the preloadData method.

In the method we first retrieve the file URL of the menudata.csv file using this line of code:

After calling the removeData method, we execute the parseCSV method to parse the menudata.csv file. With the returned items, we insert them one by one by calling the NSEntityDescription.insertNewObjectForEntityForName method.

Lastly, execute the preloadData() method in the didFinishLaunchingWithOptions method:

Now you’re ready to test your app. Hit the Run button to launch the app. If you’ve followed the implementation correctly, the app should be preloaded with the food items.
But there is an issue with the current implementation. Every time you launch the app, it preloads the data from the CSV file. Apparently, you only want to perform the preloading once. Change the application:didFinishLaunchingWithOptions: method to the following:

To indicate that the app has preloaded the data, we save a setting to the defaults system using a specific key (i.e. isPreloaded). Every time when the app is launched, we will first check if the value of the “isPreloaded” key. If it’s set to true, we will skip the data preloading operation.

Using External Data Files

So far the CSV file is bundled in the app. If your data is static, it is completely fine. But what if you’re going to change the data frequently? In this case, whenever there is a new update for the data file, you will have to rebuild the app and redeploy it to the app store.

There is a better way to handle this.

Instead of embedding the data file in the app, you put it in an external source. For example, you can store it on a cloud server. Every time when a user opens the app, it goes up to the server and download the data file. Then the app parses the file and loads the data into the database as usual.

I have uploaded the sample data file to the Amazon cloud server. You can access it through the URL below:

This is just for demo purpose. If you have your own server, feel free to upload the file to the server and use your own URL. To load the data file from the remote server, all you need to do is make a little tweak to the code. First, update the preloadData method to the following:

The code is very similar to the original one. Instead loading the data file from the bundle, we specify the remote URL and pass it to the parseCSV method. That’s it. The parseCSV method will handle the file download and perform the data parsing accordingly.
Before running the app, you have to update the application:didFinishLaunchingWithOptions: method so that the app will load the data every time it runs:

You’re ready to go. Hit the Run button and test the app again. The menu items should be different from those shown previously.

For your reference, you can download the complete Xcode project here.

Using An Existing Database in Your Project

Now that you should know how to populate a database with external data, you may wonder if you can use an existing SQLite database directly. In some situations, you probably do not want to preload the data during app launch. For example, you need to preload hundreds of thousands of records. This will take some time to load the data and results a poor user experience.

Apparently, you want to pre-filled the database beforehand and bundle it directly in the app.
Suppose you’ve already pre-filled an existing database with data, how can you bundle it in your app?

Before I show you the procedures, please download the starter project again. As a demo, we will copy the existing database created in the previous section to this starter project.

Now open up the Xcode project that you have worked on earlier. If you’ve followed me along, your database should be pre-filled with data. We will now copy it to the starter project that you have just downloaded.

But where is the SQLite database?

The database is not bundled in the Xcode project but automatically created when you run the app in the simulator. To locate the database, you will need to add a line of code to reveal the file path. Update the application:didFinishLaunchingWithOptions: method to the following:

The SQLite database is generated under the application’s document directory. To find the file path, we simply print out the value of applicationDocumentsDirectory.path variable.

Now run the app again. You should see an output in the console window showing the full path of the document directory.

Core Data - Locate SQLite Database

Copy the file path and go to Finder. In the menu select Go > Go to Folder… and then paste the path in the pop-up. Click “Go” to confirm.

Finder - Locate Database

Once you open the document folder in Finder, you will find three files: CoreDataDemo.sqlite, CoreDataDemo.sqlite-wal and CoreDataDemo.sqlite-shm.

Starting from iOS 7, the default journaling mode for Core Data SQLite stores is set to Write-Ahead Logging (WAL). With the WAL mode, Core Data keeps the main .sqlite file untouched and appends transactions to a .sqlite-wal file in the same folder. When running WAL mode, SQLite will also create a shared memory file with .sqlite-shm extension. In order to backup the database or use it to in other projects, you will need copy these three files. If you just copy the CoreDataDemo.sqlite file, you will probably end up with an empty database.

Now, drag these three files to the starter project in Xcode.

Preload sqlite database

When prompted, please ensure the “Copy item if needed” option is checked and the “CoreDataPreloadDemo” option of “Add to Targets” is selected. Then click “Finish” to confirm.

Import Core Data Database into Xcode Project

Now that you’ve bundled an existing database in your Xcode project. When you build the app, this database will be embedded in the app. But you will have to tweak the code a bit before the app is able to use the database.

By default, the app will create an empty SQLite store if there is no database found in the document directory. So all you need to do is copy the database files bundled in the app to that directory. In the AppDelegate class update the declaration of the persistentStoreCoordinator variable like this:

The changes are highlighted in yellow. We first verify if the database exists in the document folder. If not, we copy the SQLite files from the bundle folder to the document folder by calling the copyItemAtURL method of NSFileManager.

That’s it! Before you hit the Run button to test the app, you better delete the CoreDataPreloadDemo app from the simulator or simply reset it (select iOS Simulator > Reset Content and Settings). This is to remove any existing SQLite databases from the simulator.

Okay, now you’re good to go. When the app is launched, it should be able to use the database bundled in the Xcode project.

For reference, you can download the final Xcode project here.

Editor’s note: This is a sample chapter of our new book, Intermediate iOS 8 Programming with Swift. If you like this tutorial, you can check out the book here or get the starter package of our Swift book package.

iOS
Introduction to React Native: Building iOS Apps with JavaScript
iOS
Build a Simple Camera App Using UIImagePickerController
iOS
How to Add Header and Footer View in UICollectionView
  • culture2010

    Thank you for the clear and nice tutorial. This is just what i needed.


  • lakshmi

    lakshmilakshmi

    Author Reply

    Hello sir, can you please provide the same example in Objective-C.


  • Dave

    DaveDave

    Author Reply

    Great tut – thanks 🙂 Does this download the database every time, or can the database be stored on the app until there’s an update to the database? For example – if the database updates only 3 times a year, it seems a little excessive to download it each time the app launches.


    • Ismael Idrissi

      if your database exists , it will not download it anymore.
      however , don’t you think we have a double database stored in the project …. don’t we need to erase the dabatase once it has been imported , making it become as a normal core data ???


  • Axel Gora

    Axel GoraAxel Gora

    Author Reply

    Hi everyone,

    thanks for the tuto it’s awesome.
    I just have a problem with this line
    If textScanner.scanLocation < count (textScanner.string) {
    I have a problem with textScanner.string the compiler say 'String' is not convertible to 'Range

    if you can help me it would be great
    thanks


    • Sumesh Agarwal

      use textScanner.scanLocation < textScanner.string.characters.count instead


  • King Allem

    King AllemKing Allem

    Author Reply

    Hi There is a way to get this tutorial in Objective-C ?


  • Chris

    ChrisChris

    Author Reply

    Hello. I see you mentioned Swift 2, Simon.

    I’m running Swift 2 under Xcode 7 Beta 2. As you know, try-catch is different.

    Will these code snippets be re-written in October, or could we expect this beforehand?


    • jper

      jperjper

      Author Reply

      for var index = 0; index < sourceSqliteURLs.count; index++ {

      try! NSFileManager.defaultManager().copyItemAtURL(sourceSqliteURLs[index], toURL: destSqliteURLs[index])
      }

      /*

      This works just as well as try! but try! keyword communicates your intent clearly: you're aware there's the theoretical possibility of the call failing, but you're certain it won't happen in your use case. For example, if you're trying to load the contents of a file in your app's bundle, any failure effectively means your app bundle is damaged or unavailable, so you should terminate. (Thx, Hacking with Swift .com see Error handing)

      do {

      try NSFileManager.defaultManager().copyItemAtURL(sourceSqliteURLs[index], toURL: destSqliteURLs[index])

      } catch _ {

      }

      }

      */

      }


  • Ethan Humphrey

    Is there a way I could do this with iCloud implemented?


  • Sumesh Agarwal

    in func parseCSV instead of

    let content = String(contentsOfURL: contentsOfURL, encoding: encoding, error: error It should be

    if let data = NSData(contentsOfURL: contentsOfURL) {
    if let content = NSString(data: data, encoding: NSUTF8StringEncoding) {
    }
    }


    • Philippe Badizé

      First, thanks a lot for your tutorial, Simon ng ! It helps a lot !

      Regarding your comment, Sumesh :
      if let content = String(contentsOfURL: contentsOfURL, encoding: encoding, error: error)
      is compliant with this parseCSV prototype :

      func parseCSV(contentsOfURL: NSURL, encoding: UInt, error: NSErrorPointer) -> [(name: String, detail: String, price: String)]?{…}

      I only changed this and it works fine.


  • Harry M

    Harry MHarry M

    Author Reply

    Hello Simon,

    a rather late reply to to the post, but I was curious about Apple’s core data guidelines and your second example. I understand the technical solution of loading an existing database to a project, but from what I gather this will be rejected by Apple, since it doesn’t allow (from what I understand) pre-loaded applications in the Documents folder, but suggests to have it in the cache folder or mark it as ‘not-backup’.

    Could you maybe elaborate on what steps we’d need to do to avoid getting our app rejected by adding the existing database into our projects?


    • Manolo Suarez

      Hi Harry are you sure apple would reject the app? Did you get any reply from your post?


    • Georgios Traskas

      Hello Harry,

      I agree with your concerns loading the pre-loaded database inside the documents folder. Have you found any solution?


    • Khushwant Singh

      I have followed the exact same technique and my app got approved in the first shot. So if coded properly it wont be rejected by Apple.


  • Juan Diego Delgado Vargas

    Hello Simon!

    I actually have a question here, I’ve read in different threads that you shouldn’t be calling API and heave/processing stuff in the didFinishLaunchingWithOptions method because if there is a long wait, It will just kill your app. I love the tutorial, and Im actually doing something similar, but not sure if you are aware of my concern here.

    Regards,
    Juan


  • Ralf Daniel

    Hi Simon,

    great tutorial. I am playing around with it for a few days and got it to work yesterday with xCode 6.4.

    Today i upgraded to xCode 7.0 and tried to get it to work. Most errors could be handled but I am stuccoing around wit following error message:

    initializer for conditional binding must have optional type not ‘nsmanagedobjectcontext’

    This is coming up in the fun preloadData and removeData in the line with

    if let managedObjectContext = self.managedObjectContext {

    Another error I do not get fixed is the new error handling in swift 2 with the error

    extra argument ‘error’ in call

    in line

    menuItems = managedObjectContext.executeFetchRequest(fetchRequest, error: &e) as! [MenuDgItem]

    Somebody any idea or is there an update for xCode 7 available .
    Thanks!


    • Fred Fredson

      Yep. I’m stuck here as well. Teh intarwebnets have revealed nada so far.


    • Alex Sikand

      Remove the “if” from the “if let” statement to fix the first error you described and also make sure to remove the brackets that enclosed that if statement. For the second error you need to delete the second parameter from the function call.

      Just call managedObjectContext.executeFetchRequest(fetchRequest)

      Error handling has changed in Swift 2 so instead of passing in this error as a parameter you use do, try, catch, and throws to handle errors. I recommend searching “do try catch throw swift 2” on Google or something like that and you will have your code up and running in no time.


  • rkj2

    rkj2rkj2

    Author Reply

    Hi

    I was thinking of purchasing the Intermediate iOS8 Book in Swift. However, I held off when I found code in this tutorial breaking in many ways in Swift 2.0 working in XCode 7.0

    Is there any plan to relook at example code to refactor/correct it for the book releases ?

    Thanks. I definitely like the scope of the articles and the lucid manner in which things are explained.


  • HongKongTom

    Hi Simon,
    Thanks for that tutorial it s the only working (in iOS8) tut i found on the net. I would ask the same like rkj2. But my question is how would you approach if the cvs list does have a title row. e.g. Name, Description, Price. or if the list is different like this time only name, description, price and next time it name, description, ingreadians, price and then next time maybe information about allergy compatibility. what i try to ask is if the list grows. (i know that I also have to change the core data base, but lets say i ve created enough attributes in entity)


  • Khushwant Singh

    Hi and thanks for this wonderful tutorial. I have followed your tutorial and shipped pre-loaded Sqlite along with my app. It works like charm and my app got approved. I now need to perform changes in my Sqlite DB by adding new entities. Can you please also write a blog showing how to upgrade sqlite db?
    Thanks in advance


  • Vince

    VinceVince

    Author Reply

    Hi Simon,

    Awesome tutorial. I’m working with the Parse Backend framework and need to be able to parse JSON files to CoreData. Would love to know if you have a tutorial like this that shows how to do the CSV parsing with JSON instead.

    Cheers


  • Subway Korea

    Helpful tutorial! but here is an issue with the last method “using an existing database in your project”.
    What if some data are changed and reuploaded to app store?
    How would I update ~.sqlite, ~.sqlite-wal, ~.sqlite-shm files with new versions if “!NSFileManager.defaultManager().fileExistsAtPath(url.path!)” returns false all the time?


  • Bryan

    BryanBryan

    Author Reply

    This is great but how do I do anything with the data from the table. Right now it’s just a static table view, I want to load another view with a full description or something. I’ve seen didSelectRowAtIndexPath but I’m not sure how to implement it.


  • Nicholas Fantuzzi

    I needed to work a little bit on the code since Swift 2 and Xcode 7 has been recently introduced. But this tutorial is awesome! Clear, simple and easy to follow. Thank you very much!


  • Janbask – Online IT Training

    Very helpful tutorial.


  • Sean Zhang

    Sean ZhangSean Zhang

    Author Reply

    do {
    let menuItems = try managedObjectContext.executeFetchRequest(fetchRequest) as! [MenuItem]
    print (menuItems)
    } catch {
    print(“Failed to retrieve record: (error)”)
    }
    }

    for menuItem in menuItems {
    managedObjectContext.deleteObject(menuItem)
    }

    This is giving me an error about Expected Declaration, do you know what is the problem?


  • Sean Zhang

    Sean ZhangSean Zhang

    Author Reply

    do {
    let menuItems = try managedObjectContext.executeFetchRequest(fetchRequest) as! [MenuItem]
    print (menuItems)
    } catch {
    print(“Failed to retrieve record: (error)”)
    }
    }

    for menuItem in menuItems {
    managedObjectContext.deleteObject(menuItem)
    }

    This is giving me an error about Expected Declaration, do you know what is the problem?


  • Cor Prunus

    Cor PrunusCor Prunus

    Author Reply

    Hi Simon,
    Parsing n gives as result \n

    I an using this cvs parser as you wrote it. In my cvs file there are strings with literals as n or r for linefeed or carriage return. After parsing the result is that the code n has been changed to \n and r to \r. So with a double backslash.
    In a label this is interpreted as text and not as a linefeed.
    What is the solution for this?

    Thanks for your answer.
    Cor


  • Masoud r

    Masoud rMasoud r

    Author Reply

    HI Simon plz help
    i bought your book.
    when i open your zip file and type all code in Appdelegate.swift every thing work fine,
    but when i create new project and copy and paste same appdelegate code to new project.
    in new project i made new entity as you did make nsobject class for it copy menudata.csv to project.

    it give me an error in the line
    let modelURL = NSBundle.mainBundle().URLForResource(“CoreDataDemo”, withExtension: “momd”)!
    Thread :EXC_BAD_INSTRUCTION(code = EXC_i386_INVOP_subcode = 0x0)

    did you modify something in actual project properties?


  • Ujjwal Nadhani

    I would really appreciate an update to Swift 3. Great Content 🙂


  • Yerbol

    YerbolYerbol

    Author Reply

    Could you please update your code to the latest Swift 3? Best regards!


  • Hishou

    HishouHishou

    Author Reply

    I third the Swift 3 request!


  • Ikai Hung

    Ikai HungIkai Hung

    Author Reply

    I changed a little bit, This is my Swift 3 code:

    lazy var persistentStoreCoordinator: NSPersistentStoreCoordinator? = {
    // The persistent store coordinator for the application. This implementation creates and return a coordinator, having added the store for the application to it. This property is optional since there are legitimate error conditions that could cause the creation of the store to fail.
    // Create the coordinator and store
    var coordinator = NSPersistentStoreCoordinator(managedObjectModel: self.managedObjectModel)
    let url = self.applicationDocumentsDirectory.appendingPathComponent(“CoreDB.sqlite”)

    if !FileManager.default.fileExists(atPath: url!.path) {
    let sourceSqliteURLs = [Bundle.main.url(forResource: “CoreDB”, withExtension: “sqlite”)!, Bundle.main.url(forResource: “CoreDB”, withExtension: “sqlite-wal”)!, Bundle.main.url(forResource: “CoreDB”, withExtension: “sqlite-shm”)!]

    let destSqliteURLs = [self.applicationSupportDirectory.appendingPathComponent(“CoreDB.sqlite”),
    self.applicationDocumentsDirectory.appendingPathComponent(“CoreDB.sqlite-wal”),
    self.applicationDocumentsDirectory.appendingPathComponent(“CoreDB.sqlite-shm”)]

    var error:NSError? = nil
    for index in 0…sourceSqliteURLs.count-1 {
    do {
    try FileManager.default.copyItem(at: sourceSqliteURLs[index], to: destSqliteURLs[index]!)
    } catch let error as NSError {
    print(“Error: (error.domain)”)
    }
    }

    }

    do {
    try coordinator.addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: url, options: nil)
    } catch let error as NSError {
    print(“Ops there was an error (error.localizedDescription)”)
    abort()
    }

    return coordinator
    }()


Shares