irb does not use my .irbrc

On my MacOS, I have the default ruby, but for certain development task, I also have installed ruby 2.4.2 via rvm. When I explicitly invoke /usr/bin/irb my ~/.irbrc is processed (I verified it by putting a puts :processing at top of this file). When I use the irb which rvm puts into my PATH (i.e. in my case /Users/MYNAME/.rvm/rubies/ruby-2.4.2/bin/irb, the ~/.irbrc is not processed. I also tried to explicitly do a export IRBRC=$HOME/.irbrc but this also did not help. What could be the reason? BTW, in case it matters: I'm using zsh as interactive shell.