I've been running CyanogenMod on my Samsung Galaxy S4 since pretty much the day I bought it. As far as I can tell it's pretty close to the stock Android Open Source Project (AOSP) with a minimum of bloat and a few Google things (like the Play Store) removed1. You can reinstall the Google Apps but many, my self included just stick with F-Droid2.
Mostly I had just been running the stable release but a few months ago I decided I'd like to upgrade to Android 5 and was going to move across to nightly. As this was a big step I though I'd take the opportunity to completely format my phone and start again rather than upgrade and I would also enable full disk encryption.
Now with any "Full Disk Encryption" solution that you want to boot you still need a small unencrypted partition to boot from in order to get to the point where you can display a password prompt decrypt the rest of the disk. For example when setting up LUKS you need to have
/boot/ unencrypted and you can encrypt the rest of the partitions.
/boot/ doesn't need to be on the same physical disk as the one that's getting encrypted, in fact you could burn it to a CD so it's read only and boot from that. Then your hard disk really would be fully encrypted but you still need
/boot/ unencrypted somewhere.
Obviously some things have to be stored unencrypted but I was expecting that I would need to decrypt my disk somehow before I could upgrade to newer nightly builds of CyanogenMod. I assumed only a small part of the OS would be unencrypted but I found that if I booted into recovery mode I could reflash my device without decrypting it. By sideloading a zip file
adb sideload cm-12.1-20151010-NIGHTLY-jfltexx.zip so it looks like the whole OS (that is the contence of the zip file) is unencrypted but presumably all the apps and user data are encrypted, which leads me to the headline What exactly is (and is not) encrypted with Android full disk encryption?
From my digging it looks like everything in
/data is encrypted and everything else is unencrypted. Apps and all their associated data seems to be in
/data/data user data (for example my downloads, photos I've taken, music, etc...) are stored in
I think that's a fairly reasonable decision to just encrypt
/data because the android OS itself is not really what you want to protect.
As this is a security blog I feel I should point out that I use F-Droid for philosophical open source reasons, not for security reasons. Moxie Marlinspike made a good post where he points out that enabling "unknown sources" or "allow 3rd party APKs" is one of the most harmful things the average android user can do the the security of their system. ↩