Keys behaving differently when copying Xmodmap from Arch to Kali

I have copied my keyboard config(.Xmodmap file) from arch to kali. It is sourced in .xsessions and some keys work fine. However some keys are not good, for example I use the first letter row of the keyboard with AltGr to write numbers. But Now using AltGr with a does not write 1 however using it with e writes 3. Also when I press the alt-space it is intended to swap the languages of input however it takes 3 presses to get to the language and in between is either random characters or the layout that I want when I press AltGr ( the first letter line becomes numbers but without me pressing AltGr) Here for example why is 3 working and 1 not: keycode 24 = a A Arabic_dad Arabic_fatha 1 AE NoSymbol U2066 keycode 25 = z Z Arabic_sad Arabic_fathatan 2 less NoSymbol U2067 keycode 26 = e E Arabic_theh Arabic_damma 3 cent NoSymbol U2068