Newly installed apps creates multiple app icons

My phone is Panasonic P66 Mega with android lollipop version. When I install a new app from play store or connect to an app with an email, multiple app icons are being created on the screen, which disappears when the phone is restarted. Since the phone only consists of a home screen (No app drawer) it's annoying to see many icons. Please let me know how to get rid of this issue.