Battery draining faster after recharging on Galaxy S4 Mini

I'm struggling with my Samsung Galaxy S4 Mini. Actually the battery is very good and since I don't use my phone a lot, it lasts pretty much 2 days when I let it die and full recharge it. The issue is, everytime I recharge it when the battery is not dead, the battery is draining very fast even though I don't use my phone. As you can see, after I unplugged it, even though everything was off, the curve is plunging whereas when everything was off before the plug, it's almost horizontal. Does anyone have an idea how can I fix it?