Disclaimer:THIS TUTORIAL IS PROVIDED ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL I
BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS TUTORIAL, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.Motivation and ideaStarting with Android 3.x, the system theoretically supports encrypting the personal data on your device. But this has several caveats:
- it turns out that AOSP only supports encrypting /data, leaving other personal data (e.g. your pictures or storage from varios messengers) completely unencrypted on internal (emulated) or external SD storage.
- ever since, the partitioning of /data to a small, limited area leads to "no space left" issues (that's why many people go for app2sd etc.)
- in case your device suffers from physical damage, you usually loose all data you didn't backup previously.
- in case you want to sell or dispose your broken device you never know who will get access to your data anywhere in the future.
- securely wiping flash memory is complicated due to the fact that many controllers do not let you access their trim areas. Having built-in flash makes the situation even worse, especially because almost no manufacturer provides even a "manufacturer-declared-secure" wiping method - unlike some SSD drive vendors.
In this tutorial I will describe how I managed to move all personal data (probably excluding cached data, but this could be enhanced) to my external microSD card, overcoming these issues at the expense of a few drawbacks:
- (micro)SD cards are usually much slower than internal flash: boot time is about 2x slower and some apps feels a little slower, though not sluggish
- the mod requires root
- on most devices the mod also requires a modified boot image (due to change in initial RAM disk)
- it's pretty complicated to set it up (but following this guide can save you many efforts I already invested)
Although this is more a proof-of-concept, I have been running this setup for a couple of days now as my daily driver without significant issues. I had some force-closes in Firefox, but I don't relate them to this mod and other browsers seem to work fine.
Enough said, let's get to work and move all personal data to your external SD card!
CURRENTLY, THIS GUIDE IS TARGETED AT ADVANCED USERS / DEVELOPERS / EXPERTS AND THUS ONLY COVERS THE ESSENTIAL BASICS, NOT THE DETAILS!FOLLOWING THE PROCESS, YOU WILL LOOSE ALL YOUR PERSONAL DATA ON YOUR DEVICE! MAKE BACKUPS FIRST!!!CompatibilityThe tutorial is based on OmniROM 4.4 and TWRP. With not too much effort, it could be adopted to earlier Android Versions based on AOSP 4.0, probably even 3.0. (most likely, the location and internal layout of some configuration files will vary). Please note that I tried it only on one device, your milage may vary!
Step 1: Find out your partition layoutFirst off, you need to find out which devices are used for holding which data partitions. You can play around with "mount" or simply look in your fstab files which are located in /fstab.vendor in KK. While in some cases the devices are listed i.e. "by-name", the fstab files of custom recoveries often list the "real" device names. I.e. on my Xperia V they are as follows:
Code:
Internal Data partition: /dev/block/platform/msm_sdcc.1/by-name/Userdata
Internal, emulated SD card: /dev/block/mmcblk0p15
First partition of external SD card: /dev/block/mmcblk1p1
Step 2: Encrypt internal dataAfter backing up everything, run the usual Android encryption process so your data partition (in this case: /dev/block/platform/msm_sdcc.1/by-name/Userdata) will be encrypted by means of standard 128bit AES Android encryption.
Step 3: Extract the cryptofooterBoot into TWRP recovery (don't unlock your /data if prompted!) and connect via ADB as root. Usually, the crypto footer is stored in the last 16384 bytes of your partition (but check your fstab to be sure). Now determine the size of your partition with:
Code:
fdisk -l /dev/block/platform/msm_sdcc.1/by-name/Userdata
In my case:
Code:
Disk Userdata: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
...
In my case, the size is 2147483648 bytes. Subtracting 16384 bytes, you know the starting offset for the extraction. Now extract the footer using dd:
Code:
dd if=/dev/block/platform/msm_sdcc.1/by-name/Userdata of=/cryptofooter bs=1 seek=2147467264 count=16384
Please note, that due to a limitation in dd which supports only up to 2GB this may file - you will need a modified busybox dd command in this case - or just download the file to your UNIX machine and use the normal dd from there.
Step 4: Prepare the external SD card:We de not want to partition the SDcard, thus we will not put any partition table on it. Nevertheless, we need to determine its exact size using fdisk:
Code:
fdisk -l /dev/block/mmcblk1p1
Disk mmcblk1: 31.9 GB, 31914983424 bytes
255 heads, 63 sectors/track, 3880 cylinders, total 62333952 sectors
Units = sectors of 1 * 512 = 512 bytes
...
Again, subtract the 16384 bytes from 31914983424 and use dd to write the cryptofooter on the end of SD:
Code:
dd if=/cryptofooter of=/dev/block/mmcblk1= bs=1 skip=31914967040 count=16384
Again, this might fail due to dd limitations.
For later testing, we copy over the existing (encrypted) data partition to the SD card:
Code:
dd if=/dev/block/platform/msm_sdcc.1/by-name/Userdata of=/dev/block/mmcblk1p1
Finally, we can wipe the internal data partition:
Code:
dd if=/dev/zero of=/dev/block/platform/msm_sdcc.1/by-name/Userdata
Step 5: Modifying fstab and init files:Now we will need to modify the fstab files and init so they will reflect the new layout. On most devices, this requires compiling a new rootfs. This can be either done by uncompressing your boot.img, making the changes and recompressing it or recompiling it from sources. The modified boot.img has to be flashed back to the device, usually via fastboot.
In particular, we need to tell Android not to use the internal userdata partition at all, to mount the emulated sd card as usual and to use the external storage for /data using an encrypted ext4 filesystem. In my case, the modified fstab.qcom looks like this:
Code:
/dev/block/mmcblk1 /data ext4 noatime,nosuid,nodev,barrier=1,data=ordered,noauto_da_alloc wait,check,encryptable=footer,length=-16384
/devices/platform/msm_sdcc.1/mmc_host/mmc0 auto ext4 defaults voldmanaged=sdcard0:15,nonremovable,noemulatedsd
#/devices/platform/msm_sdcc.3/mmc_host/mmc1 auto auto defaults voldmanaged=sdcard1:auto
/devices/platform/msm_hsusb_host auto auto defaults voldmanaged=usbdisk:auto
You will also need to modify the twrp.fstab file accordingly.
We also need to modify init (in my case through sony.init.rc) to avoid getting the external sd card being treated as sdcard during boot by uncommenting the relevant lines:
Code:
chmod 0701 /mnt/media_rw
mkdir /storage 0775 system system
mkdir /mnt/media_rw/sdcard0 0700 media_rw media_rw
# mkdir /mnt/media_rw/sdcard1 0700 media_rw media_rw
mkdir /mnt/media_rw/usbdisk 0700 media_rw media_rw
mkdir /storage/sdcard0 0775 system system
# mkdir /storage/sdcard1 0775 system system
mkdir /storage/usbdisk 0775 system system
export EXTERNAL_STORAGE /storage/sdcard0
# export SECONDARY_STORAGE /storage/sdcard1
# for backwards compatibility
symlink /storage/sdcard0 /sdcard
symlink /storage/sdcard0 /mnt/sdcard
# symlink /storage/sdcard1 /ext_card
# symlink /storage/sdcard1 /mnt/ext_card
symlink /storage/usbdisk /usbdisk
symlink /storage/usbdisk /mnt/usbdisk
Step 6: First booting testWith all this set up, we can now boot again into our system. It should ask you for your encryption password (it's the same as you've set it in step 2) and you should notice that it runs a little slower - because SD cards are usually slower than internal flash. You will notice that the size of the internal space and external storage has not changed - we will fix this in the next step.
Step 7: Tweak the cryptofooter and reformat the SD card:Now comes the fun part: Android determines the size of encrypted filesystems not by information from the filesystem, but by the size as it is stored in the cryptofooter. Therefore, we need to modify the information there. The cryptofooter is a complicated mess. Luckily, there is a great guide which describes its structure much better than the source code itself:
http://nelenkov.blogspot.de/2014/10/revisiting-android-disk-encryption.html
You will need to use a hex editor to edit the from bytes between Offset 0000:0018 and 0000:001B to match the size of your SD card. But please leave enough bytes at the end, at least for the cryptofooter!
Write back your modified cryptofooter as outlined in step 4 and boot into TWRP. Unlock your /data (which is now served from SD), connect via ADB and check the size of your encrypted device using fdisk:
Code:
fdisk -l /dev/block/dm-0
It should reflect the size as you modded it in your cryptofooter. Now, reformat the partition so fills the whole space:
Code:
make_ext4fs /dev/block/dm-0
Step 8: Adjust internal SD.Now, you are ready to reboot into your system serving its freshly wiped, encrypted data from your external SD! Be amazed by the space you now have for your internal Userdata! Unfortunately, it will still serve "internal SD" from internal flash. First, create a directory on the sdcard to hold it using a root shell:
Code:
# mkdir -p /data/sdcard0
I tried various things to fix this already in the boot process, but somehow vold always got in the way or the device was mounted too early. Also, the media server indexes the internal flash instead of the stuff on the external SD. Therefore, I bind-mount the sdcard0 after every boot and rerun the media server by running the following commands in a root shell:
Code:
# busybox mount -bind /data/sdcard0 /mnt/media_rw/sdcard0
# am broadcast -a android.intent.action.MEDIA_MOUNTED -d file:///storage/sdcard0
Now, you can copy back all your data from the backup to the device and enjoy your new setup!
Final notes:- Suggestions for correctly mounting the sdcard0 inside data at boot time without the bind-mounting is warmly welcome.
- Some (Stock) ROMs allow you to encrypt both SD "cards", but the implementations I have seen so far seem to be vendor-specific and closed source. Although I noticed that encrypting internal SD seems to work with AOSP out-of-the-box on some devices without external SD (e.g. Nexus S) I didn't find any Custom ROM for my Xperia V which worked with encrypted any of the SDs when putting the "encryptable=sdcard" flag in the fstab file: The storage seems to have been encrypted, but Android simply fails to mount it.
XDA:DevDB InformationAll data 2 encrypted, removable SD, Tool/Utility for the Android GeneralContributorsDefier525,
Defier525Version InformationStatus: Alpha
Created 2015-01-08
Last Updated 2015-01-07
