SSH VScode to ec2 instance AWS Linux

I am trying to connect to an AWS EC2 instance from my MacBook Pro using VSCode. I am able to connect to the EC2 instance through the terminal by navigating to the Downloads folder (where my key pair .pem file is) using the command:

-shell
ssh -i "Key-Pair.pem" ec2-user@ec2-**-***-**-***.ap-northeast-1.compute.amazonaws.com

My terminal shows me

-shellsession
[ec2-user@ip-***-**-**-*** ~]$

However, when trying to connect to the AWS instance from VScode I get the error

Load key "/Users/****/Downloads/Key-Pair.pem": Operation not permitted
ec2-user@**.***.**.***: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
local-server-1> ssh child died, shutting down

It seems as though my key pair .pem file is not able to authenticate with the AWS instance. For reference this is what my config file looks like for this instance

Host PM.ByBit.SMM
    Hostname **.***.**.***
    User ec2-user
    IdentityFile /Users/****/Downloads/Key-Pair.pem