Remove some auto created indices from Entity Framework?

I'm using Microsoft's Entity Framework. It automatically creates indices for every foreign key, but in many cases, I can see from the index stats that these are never used for reading - they are only ever updated (which doesn't surprise me as I doubt I am ever actually doing a join on many of them). My gut feeling is that I should remove these indexes as they are just slowing down updates (albeit probably not that significantly) and taking up space. I like to try to keep things clean and lean. Is there a good reason to keep them?