Do I need to keep my photo receipts on my phone?

No. Once you attach a photo receipt to an Expense that photo will be stored on a server.