iOS Crash Reports from Unity3D Games

We noticed customers complaining about crashes at launch of our Unity iOS game Hit Tennis 3. This was a problem we could not reproduce, and it lead us down a rabbit hole dealing with both ObjC and C# crashes. Read on to find out how to use iOS crash reports with Unity games, and how to get crash logs from your C#/Unityscript code too. For years now iOS and iTunes Connect have conspired to collect crash reports from real users, gather them up, let us download them from iTunes Connect, and view them in Xcode. However, this system has been very broken in Unity games. Maybe Unity didn’t think it mattered because Unity games aren’t written in ObjC, so who cares about iOS crash reports? Well we care. Our games have lots of native iOS code we’ve written, and we use a bunch of third party iOS libraries too. You know, for useful stuff like UI, analytics, configuration management, advertising. And anyway, for free Apple will gather crash logs and tell us which kinds of crashes are the most common among real customers. Who doesn't want to know that? So here is how to fix it and get your Unity iOS crash reports back…

Crash Symbolication and Dsym Files Explained

When an app crashes, iOS makes a crash log that shows the execution stack of the crashed thread so you can see what code was running when it crashed. A running program’s execution stack is a bunch of memory offset address of code for each method call along with parameter data. Programmers don’t know their code by memory addresses; we know it by source code line numbers. When the code is compiled, there are Xcode settings to include debug symbols into the app. These debug symbols enable iOS to generate a useful crash report with line numbers of the source code, instead of just a bunch of impossible to understand memory addresses. At least that’s what happens for debug builds. Stuffing the debug symbols into the build makes it bigger than it needs to be, so debug symbols get stripped from app store builds.  That means that crash logs from real users don’t have source code line numbers, they only have memory addresses, which makes them incomprehensible. Apple planned ahead for all this though. Xcode can compile with the ‘DWARF with dsym file’ setting. The dsym file is a lookup table that converts from the memory offsets to source code line numbers. Xcode stores the dysm file in the build archive, and it will automatically find the correct dysm and ‘symbolicate’ crash reports so that it shows method names and source code line numbers:

symbolication1

 

 

 

 

 

 

 

To do this you download a crash report from iTunes Connect, and then you drag and drop it into the ‘device logs’ area of the Organizer’s devices tab. It’s brilliant when it works, but it relies on your Xcode project build settings be just right.  When Unity generates the Xcode project it uses different settings that result in no useful dysm file (either no dysm file at all or a dsym file with no symbols). This means crash reports can not be symbolicated, making them useless. (As far as I can tell this is for no good reason, I think its just Unity’s mistake.) However it’s your Xcode project not Unity’s, just fix the settings right and you’ll have good builds.

symbolication2

Correct Xcode Settings for Dsym Files

Here are the Build Settings you need, courtesy of capyvara on Unity Answers. (This isn’t something you can do after the fact. To make this work you will need to submit a new build to the app store and wait for crash reports from that new build.)

  • Build Options -> Debug Information Format: "DWARF with dSYM file"
  • Deployment -> Deployment Postprocessing: Yes (hmmm, I have 'No', but it works...?)
  • Deployment -> Strip Symbols During Copy: Yes,
  • Deployment -> Strip Linked Product: Yes
  • Deployment -> Use Separate Strip: Yes (again, I actually have no).
  • Linking -> Other Linker Flags:   remove these: "-Wl,-S,-x"
  • Apple LLVM compiler - Code Generation ->  Generate Debug Symbols: Yes

symbolcation3

 

iOS Crash Reports for C# or Unityscript crashes.

For an app store build, Unity recommends using the player setting ‘Script Call Optimization: Fast but no exceptions’. With ‘fast but no exceptions’ the normal C# error handling is removed from the compiled code for performance reasons. With ‘fast but no exceptions’ if you try to dereference a null object reference in C#, instead of getting a null reference exception the game will just crash. While some would argue that you could run in ‘Slow and safe mode’ for an app store build and hence get null reference exceptions rather than crashes, that won’t really help most cases. Throwing exceptions in the middle of game logic will mean that Update loops or other parts of your game code won't be running all the way though. That just results in a broken game that limps along. Better to get crash reports you can find out about and fix. But Unity 4.1 and earlier, when the C# code crashes, Unity would shut down the app cleanly without generating a crash report. So iOS would not know there was a crash, and no crashes would be reported via iTunes Connect. This is a big problem if you are paying attention to crash reports in iTunes Connect -  what if the vast majority of game crashes are in your C# / unityscript code? By looking at iTunes Connect you would never know. You’d only know when users complaining in emails and app reviews. The problems we were dealing with were crashes during startup, so it was even harder for users to even email us… they can’t tap the ‘Contact Us’ button in the app if the app never even starts up! The good news is that Unity 4.2 fixes this, and by default if the C# code crashes then Unity will actually crash the whole app. iOS will see it and report to iTunes Connect.

So this is easy: update to Unity 4.2, and submit an update to the App Store. Note that these are still iOS crash reports, so you will see a C stack, not a C# stack. First off, its good to know you are getting crashes, iTunes Connect will tell you which kinds of crashes are most common. Second, your C# code is compiled to a native executable, so you will still see symbols you recognize from your C# or Unityscript.

Unity 4.2 has additional crash reporting features that you can choose to enable. Its possible to query from code for a previous crash the next time the game starts up, and crash logs can be sent to a third party server. However these are still iOS crash logs, not C#/unityscript crash logs. So I’m not sure what the point of that is… iOS and iTunes Connect do this anyway. I guess using a third party crash reporting service will collect more crash logs and collect them faster than Apple does, but the basic service from Apple has always be fine for us.

In Part 2 I’ll describe how we get logs from real players to figure out crashes in our C# or Unityscript code.