How do I do the required route(s) for getting internet traffic to/from my client assigned public IPv4s on my L2TP server

I have a server running Ubuntu Server 20.04 which has two ethernet interfaces and is hosting the L2TP server (using accel-ppp). 'eno1' has a single public IPv4 address assigned. 'eno2' has access to a /26 public IPv4 block which I'd like to make use of from another location via an L2TP server. Details further down. Now what I'm trying to do is make it so my router, at another location, can connect to the L2TP server and have a public facing IPv4 as well as a /27 public IPv4 routed to it from splitting the /26 public IPv4 mentioned earlier. For example, x.x.161.64/27. While I can ping the IP of the router connected to the L2TP server, from the L2TP server, as well as any /27 IPv4 I assigned via the router's LAN, I can't figure out how to get a route to the internet or beyond presumably the L2TP server's own gateway IP (x.x.161.122). eno1

IP address:  x.x.176.62 (public IPv4)
Subnet mask: 255.255.255.0
Gateway IP:  x.x.176.254

eno2

IP address:  x.x.161.125 (public IPv4)
Subnet mask: 255.255.255.252 (split from what is actually a /26)
Gateway IP:  x.x.161.126

My router assigned IP addresses, that's connecting to the L2TP server but aren't currently able to access the internet or go beyond x.x.161.122 (the L2TP server's gateway IP address - I believe) it seems.

x.x.161.121/30
x.x.161.64/27

On this Ubuntu server I have accel-ppp installed and configured as an L2TP server. In

/etc/accel-ppp.conf

I have the following:

[modules]
log_file

pptp
l2tp

auth_mschap_v2
auth_mschap_v1
auth_pap

chap-secrets

ippool

pppd_compat

[core]
log-error=/var/log/accel-ppp/core.log
thread-count=4

[common]
single-session=replace

[ppp]
verbose=1
min-mtu=1280
mtu=1400
mru=1400
ipv4=require
ipv6=deny
ipv6-intf-id=0:0:0:1
ipv6-peer-intf-id=0:0:0:2
ipv6-accept-peer-intf-id=1
lcp-echo-interval=1
lcp-echo-failure=5
lcp-echo-timeout=120
unit-cache=1

[pptp]
verbose=1
#echo-interval=30
#ip-pool=pptp
#ipv6-pool=pptp
#ipv6-pool-delegate=pptp
ifname=pptp%d

[l2tp]
verbose=1
ifname=l2tp%d

[dns]
dns1=8.8.8.8
dns2=8.8.4.4

[client-ip-range]
disable

[ip-pool]
gw-ip-address=x.x.161.122
attr=Framed-Pool
x.x.161.121/30

[log]
log-file=/var/log/accel-ppp/accel-ppp.log
log-emerg=/var/log/accel-ppp/emerg.log
log-fail-file=/var/log/accel-ppp/auth-fail.log
copy=1
level=3

[pppd-compat]
verbose=1

[chap-secrets]
chap-secrets=/etc/ppp/chap-secrets

Current ip route:

default via x.x.161.126 dev eno2 proto static
default via x.x.176.254 dev eno1 proto dhcp src x.x.176.62 metric 100
x.x.176.0/24 dev eno1 proto kernel scope link src x.x.176.62
x.x.176.254 dev eno1 proto dhcp scope link src x.x.176.62 metric 100
x.x.161.64/27 via x.x.161.121 dev l2tp0
x.x.161.121 dev l2tp0 proto kernel scope link src x.x.161.122
x.x.161.124/30 dev eno2 proto kernel scope link src x.x.161.125

Current route:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         x.x.161.126     0.0.0.0         UG    0      0        0 eno2
default         x.x.176.254     0.0.0.0         UG    100    0        0 eno1
x.x.176.0       0.0.0.0         255.255.255.0   U     0      0        0 eno1
x.x.176.254     0.0.0.0         255.255.255.255 UH    100    0        0 eno1
x.x.161.64      x.x.161.121     255.255.255.224 UG    0      0        0 l2tp0
x.x.161.121     0.0.0.0         255.255.255.255 UH    0      0        0 l2tp0
x.x.161.124     0.0.0.0         255.255.255.252 U     0      0        0 eno2

Current ifconfig:

eno1: flags=4163  mtu 1500
        inet x.x.176.62  netmask 255.255.255.0  broadcast x.x.176.255
        inet6 x:x:x:x::  prefixlen 56  scopeid 0x0
        inet6 fe80::d250:99ff:feda:91b6  prefixlen 64  scopeid 0x20
        ether d0:50:99:da:91:b6  txqueuelen 1000  (Ethernet)

eno2: flags=4163  mtu 1500
        inet x.x.161.125  netmask 255.255.255.252  broadcast x.x.161.127
        inet6 fe80::d250:99ff:feda:91b5  prefixlen 64  scopeid 0x20
        ether d0:50:99:da:91:b5  txqueuelen 1000  (Ethernet)

l2tp0: flags=4305  mtu 1400
        inet 198.244.161.122  netmask 255.255.255.255  destination x.x.161.121
        ppp  txqueuelen 3  (Point-to-Point Protocol)

How would I go about making it so, for example, the router IP address x.x.161.121 would be able to reach the internet and be reachable from the internet? Presumably it would need to somehow have a route to x.x.161.126, the gateway IP address of the entire original /26 IPv4 block. If there's a simpler or different approach I should be taking please say. I don't want to do NAT as that I imagine defeats what I'm trying to do. Hopefully I've been reasonably clear and provided plenty of details, if there's more details you need please ask. I've been trying to get my head round this for nearly two days. Playing around with changing routes is a little new to me. Thanks in advance for any assistance! EDIT: It doesn't look hopeful that I'll get an answer here so I might just have to see about finding an expert to hire for this task, assuming quotes aren't ridiculously pricey. If someone does read this question and knows the answer then I'd be really grateful to hear your solution! Thanks.