analysis

Providing Context to iOS App Usage with knowledgeC.db and APOLLO

in , ,

With the APOLLO v1.0 update, I updated many of the Application Activity modules used with the knowledgeC.db database. I mentioned in this article that these were updated to provide more context to specific user application activities. 

One column in particular that was added to all the App Activity modules is Z_DKAPPLICATIONACTIVITYMETADATAKEY__USERACTIVITYREQUIREDSTRING from the ZSTRUCTUREDMETADATA table. I will refer to this as “User Activity Required String” as that is what I’ve named it in the APOLLO modules. 

I will start with some of the application activity parsers for the native iOS apps. The first one, Maps, is particularly difficult but holds some really interesting and useful data.

Maps - knowledge_app_activity_maps

Many of the ‘User Activity Required String’ BLOBs for Maps looks more or less like the following (some are smaller, some larger.) Many investigators will recognize the base64. Whenever you are looking at an Apple device and see base64, you will want to decode it – there is usually great data in there!

You may also notice some readable text at the top of these BLOBs such as in this one. This can help provide quick context, so you can tell if you need to take the time to decode it. This example shows I searched for “Mike Serio’s Po-boys in New Orleans” while at BSidesNoLA this past October. (They were delicious.)

v1.0/com.apple.Maps/t='Mike%20Serio%27s%20Po-boys%20in%20New%20Orleans'&u=

….and others including searches, dropped pins, and directions.

v1.0/com.apple.Maps/t='Show%20DoubleTree%20by%20Hilton%20Hotel%20New%20Orleans'&u=
v1.0/com.apple.Maps/t='Dropped%20Pin%20on%20Nebovidsk%C3%A1%20459%2F3,%20Prague'&u=
v1.0/com.apple.Maps/t='Get%20directions%20to%20The%20Hotel%20at%20Arundel%20Preserve'&u=
v1.0/com.apple.Maps/t='The%20Remedy%20Room%0ADirection%20from%20My%20Location'&u=

I extracted just the base64 string (between the two $ signs) and decoded it on the command line with base64 and then piped it to xxd so I could see what it was. The string ‘BZh11AY&SY’ tells me we are working with a bzip archive file. 

This BLOB has four extra bytes at the beginning that we need to remove before we are able to interpret the bzip archive. We can use ‘tail -c +5’ to easily remove these so we get the bzip header right at the beginning.

Next we will pipe this to bzcat to see what is in this archive, again using xxd last so I can see what it might be. bzcat will extract the contents of a bzip archive to standard out.

Would you look at that, we have a protobuf! I’m not surprised to see this data format; it shows up everywhere! We can parse protobufs with protoc. I will let the reader have fun determining what all is in this protobuf. Might I recommend starting with my article on Apple protobufs.

There we have it, the hoops we have to jump through to get the actual maps data. Thanks Apple, keeping it fun! Here is a quick and dirty recap of my decoding process:

Take the SQLite BLOB ➡️ Extract & Decode base64 ➡️ Remove first 4 bytes using tail ➡️ Unarchive bzip with bzcat ➡️ Decode protobuf with protoc

echo "<base64>” | base64 -D | tail -c +5 | bzcat | protoc --decode_raw

Fortunately, while fun, not all of these apps have crazy base64-bzip-protobuf blobs. Mail, Notes, Safari, Photos, Clock, Weather, and Calendar are relatively easy to visually parse and can provide context to what specifically is happening with the application. Some examples below.

Mail - knowledge_app_activity_mail

Shows access to different email accounts and boxes.

Notes - knowledge_app_activity_notes 

Shows which notes are being edited. This one shows my note detailing what I need for my last vacation.

Safari - knowledge_app_activity_safari

Shows specific Safari browsing data - I’m looking at fancy hotels for my trip!

Photos - knowledge_app_activity_photos

Which photo albums am I viewing, certainly not the one with all my cat pictures. 🤫

Clock - knowledge_app_activity_clock

Did I just cancel my early morning alarm or am I timing how long this meeting is going?

Weather - knowledge_app_activity_weather

Weather may be particularly interesting as it as geo location coordinates embedded. However, they might be someplace I am or am looking to go! You may want to correlate with location data extracted with APOLLO.

Calendar - knowledge_app_activity_calendar

I might be checking meetings or checking how long until the next Objective by the Sea conference is. You should go, it’s in Maui!

maps_requiredstring_calendar.png

3rd Party Apps - knowledge_app_activity

Now we go to the generic App Activity module to review some activities from 3rd party applications. I won’t create modules for every 3rd party application because there are so many. Some applications will have data in knowledgeC.db, some will not.

First up we have my favorite time sink, Twitter. I can see access to specific tweets as well as whichever profile pages you are Twitterstalking.

Dropbox shows access to documents with the Dropbox application.

Venmo shows me paying my debts. The User Activity Required Strings shows interesting information about the contact like their Venmo ID, how many friends they have, friend status, and their Venmo join date. 

And finally, Waze shows me going “Home”.

There is some fantastic and useful information in these activities if you dive into them! 

New Year New APOLLO – Officially out of Beta iOS 13 Module Updates!

in , ,

I spent this weekend updating and sprucing up APOLLO for its v1.0 release. It took far longer than anticipated, mostly because I’ve added quite a few new modules. It also takes a while to go through every SQL query module updating it to iOS 13. Database schemas change all the time, new tables, new columns, new everything!

I’ve done my best to test the new modules and a few of the updated older modules with iOS 13.1 and iOS 12.1.1. I have no doubts that some may work with iOS 11 as well but in order to keep on keepin’ on, I’ve decided to only test two major versions back. If you find one works for iOS 11 or older please let me know via Contact or submit a PR. If you happen to be working on an iOS 12 device I would definitely re-process for many new and updated modules! This brings the total modules to 165!

I will talk about a some of these in my upcoming Webinar for Blackbag on January 30 at 2pm EST.

Apple Health Modules

I have added imperial measurements to modules so us yanks can actually understand the measurements 😉:

New modules include:

Location Modules

Routined Databases changed a bit in iOS 13. Cloud.sqlite is now Cloud-V2.sqlite. Most modules did not need to be updated; however a new table was introduced ‘ZRTMAPITEMMO’. Modules were updated to include this data as well as a new separate Map Item module - routined_cloud_mapitem.

In the Cache.sqlite database, a new table was introduced, ‘ZRTVISITMO’ so a new module was created for it - routined_cache_zrvisitmo.

KnowledgeC Modules

The biggest update was with my favorite database knowledgeC.db.

Update modules include more context to the Application Activity modules. Some of the iOS native applications were split out into their own (they should also be parse in the generic knowledge_app_activity module.) Note: Calendar - was its own previously, however it was renamed to knowledge_app_activity_calendar to be consistent with the new modules below.

New modules include:

New(ish) Presentation: Poking the Bear - Teasing out Apple's Secrets through Dynamic Forensic Testing and Analysis

in , ,

I had the wonderful opportunity to present this presentation at two great conferences in October; Jailbreak Security Summit and BSides NoLA. Unfortunately I was going on an extended vacation almost immediately after so I forgot to post this to the site. I had a strict self imposed no-laptop policy for this vacation so it would just have to wait. FWIW: Everyone should take a vacation that is [mostly] offline, very refreshing!

The presentation is here, and the video of the presentation from the Jailbreak summit is here. Both of these links are also available in the Resources section of this website.

While I was off exploring Southeast Asia (see my twitter feed for those updates), there has been some major updates to iOS Jailbreaking that are worth a mention with this posting. The Checkm8 exploit was one of the major points of discussion during this presentation, as it was going to be a game changer for this type of analysis.

While I was drinking fruity drinks with umbrellas in them, the public jailbreak came out - Checkra1n. I almost broke my no-laptop policy when this happened, but I held back - someone else would write about it. Fortunately my good friend Mattia Epifani has written some fantastic blogs about using this in the forensic realm. I highly recommend reading through these.

iOS Device Acquisition with checkra1n Jailbreak [Elcomsoft & Mattia]

Checkm8, Checkra1n and the new "golden age" for iOS Forensics [Mattia]

Checkra1n Era - Ep 1 - Before First Unlock (aka "I lost my iPhone! And now?") [Mattia]

Checkra1n Era - Ep 2 - Extracting data "Before First Unlock" (aka "I found a locked iPhone! And now?") [Mattia]

Checkra1n Era - Ep 3 - Automating extraction "Before First Unlock" (aka "Give me a stupid bash script!") [Mattia]

Just Call Me Buffy the Proto Slayer – An Initial Look into Protobuf Data in Mac and iOS Forensics

in , ,

I was first introduced to the protobuf data format years ago accidentally when I was doing some MITM network analysis from an Android device. The data I was looking at was being transferred in this odd format, I could tell there were some known strings and some patterns to it - but I did not recognize the format. It had no magic number or file header. I started looking at it more in depth, it did have a structure. I spent an embarrassingly amount of time trying to reverse engineer this structure before I realized it was in fact a protobuf data blob. I think it was a moment of frustration in reversing this structure that lead me to searching something like “weird data format Android network traffic” that finally lead me to protobufs. 

 Ok, so what the heck is a protobuf? It actually stands for Protocol Buffer, but everyone calls them protobufs. It is a “language-neutral, platform-neutral extensible mechanism for serializing structured data” created by Google. It is a super-efficient way of storing and transferring data.

Since I was looking at an Android device, a protobuf made perfect sense. This was a Google thing afterall. I started noticing them more and more on Android devices, not just in the network traffic but also storing data on disk as well. It took me a long time to also notice that they were being stored on Apple devices! Native applications, 3rdparty applications, they are used EVERYWHERE! A great example was found by my friend Phill Moore in the iOS Spotify application to keep track of items listened to. 

In this article I’ll introduce you to some of the Apple-specific protobufs that I’ve come across. Some are fairly straight forward, others are less so. The kicker with protobufs is that there is an accompanying *.proto file that contains the definition to what is contained in these buffers. Unfortunately, we likely do not have this file as it is most likely server-side or inaccessible therefore we need to reverse engineer the contents and meaning of the items stored in this blob.

To parse these protobufs, I use protoc from Google to get a raw output. If you have the .proto file you can use this as well, but I have yet to give that a go. On a Mac, I would do a ‘brew install protobuf’ to get protoc installed. To parse a given buffer I will use the following command:

protoc --decode_raw < [protobuf_blob_file]

I will parse out some protobufs from different applications to give you an idea of what is stored in them - Maps, Locations, Health, and Notes.

Maps Data

The Maps application on both macOS and iOS use many protobufs to store location data. These can be found in quite a few different Maps related plist files. I will focus on GeoHistory.mapsdata plist file from iOS which stores historical locations that were mapped. This plist has GUID keys that contain a “contents” subkey. This “contents” blob contains the protobuf of the mapped location. I chose a small example to begin with as some of these can be very large. 

I’ve extracted this blob and put into a hex editor. You can see some visible strings in this blob but really no context. This is where parsing with protoc can be helpful.

I saved this protobuf to a file I named geohistory.pb. Using protoc I decoded it to the output below. I can see the same GUID and location name but now I get some other hex based values. This is where you will have to determine what these values mean and what they are used for, it may not be obvious initially.

At this time, I believe the highlighted pairs of hex in the screenshot above are coordinates. To read these, I copy them into a hex editor to read their values highlighted below. On the left is latitude, on the right is longitude. Plotting these two pairs above I get one location in Manassas, VA and another in Bethesda, MD. Clearly, neither of these are in Fairfax, VA. This is the tricky part, what do these values mean? More testing needs to be done here.

Location Data

The next example shows a protobuf blob being stored inside of a SQLite database. The example below is from the Local.sqlite routine locations database from iOS 11. (In iOS 12, the same data exists but has been placed into separate columns – which IMHO, is far easier to interpret.) The ZRTLEARNEDLOCATIONOFINTERESTMO table contains “learned” locations of interest or significant locations. This example is Heathrow Airport. Two columns contain protobuf data -  ZPLACEMAPITEMGEOMAPITEMHANDLE (highlighted), and ZPLACEMAPITEMGEOMAPITEM. To decode these with protoc, I will need to extract them to separate files. In DB Browser for SQLite, I can use the export feature.

The ZPLACEMAPITEMGEOMAPITEMHANDLE protobuf blob parsed with protoc contains much of the same location data as seen before. You will find that most of the location blobs will look similar. 

Taking the highlighted hex coordinates and plotting them using Google Maps, they make more sense than the ones highlighted above. These coordinates are right at Heathrow Airport in London.

Health Data 

Protobufs are not just used for location data, but also for other data storage such as split times when I have a running or walking workout. The healthdb_secure.sqlite database on iOS contains this information. Shown below is the “workout_events” table. I have selected the BLOB data for one “event” or one split time (a workout may consist of multiple split times for multiple associated “events”).

I exported and decoded with protoc using the same methods described above. The example below is from a walk I took in Monaco. Presumably the labels “s” is for seconds and “m” is for meters, however I’m still verifying this assumption.

Notes Data

One last example brings us to the Notes application. Anyone who has looked into this database likely knows that it is a complicated one. I created a sample note to show how it may look in the protobuf produced. This note has some text, a link, and a cat meme that I’ve been enjoying recently.

This example comes from MacOS in the NotesStore.sqlite database (iOS is similar). The “ZICNOTEDATA” table contains the contents of each note. Some eagle-eyed readers may notice the highlighted bytes in the binary output. It stores note contents in gzip archives embedded in the SQLite database.

I exported this gzip archive and used gzcat and xxd to preview the contents of it. I can see the text and link in the note along with some media information. What we have here is another protobuf!

The protoc output is below, many of the same strings are visible but there are some odd ones in there.

One in strange one in particular is the line "\033\'w\213\256?@Q\251IHEn\221\242\260". This is escaped octal. You will find this being used in a variety of different protobuf data. Some are smaller, some are much larger. I recall the one that Heather Mahalik and I looked at for Android Auto. That one was just full of octal nastiness, it was awful.

This one is small and converts well using echo. What does it represent? I have no idea…yet.

The media itself is stored in the file system (you can use information in the database to find the proper path). We can use the GUID in the protoc output to find the preview thumbnail.

Summing Up 

I will be very honest, I have been looking at these weird data blobs for years without knowing what they were. I am at the point now where I’m a little obsessed with protobufs. I know…its weird – but now every time I see rando-blob-data with a bunch of 0x0A hex bytes – I think protobuf, and nine out of ten times, I am correct! 

If you happen to know where I can find more information on the .proto files, please let me know!