VyOS Network and Routing Configuration

How-To: VyOS

Karl William

Mar 31, 2025

Monday 31st March 2025

In this post I’m going to cover:

VRF Configuration
Physical and VLAN interface configuration
Basic BGP and OSPF routing configuration

Note: This is post is not going to explain what all these things are. I assume the readers knows that already.

Network Design

Layer 3 Design

This is the intended design:

Click on the picture to get a bigger view

Explanation:

Ignore everything inside AS 65000 and AS 65001. Those are out of scope for this post.
This post is focusing on:
- the eBGP peering between VyOS and pfSense FW;
- and the OSPFv3 peering between VyOS and the Ubuntu Server
I have written about BGP configuration on pfSense before so i won’t cover it here.
Ubuntu Network configuration will be covered in the next post.
I will only be advertising IPv6 scopes but Docker insists on having to configure dual-stack, unless I disable IPv4 completely from the Linux Server.
IPv6 NPT will be configured in the future to translate for example 2001:DB8:1234:A000::/52 → 2001:DB8:1234:1000::/52. The reason for that will be covered in a later post.
There are 3 VRFs defined:
- NMS = Network Management Services
- INT = Internal Services
- PUB = Public Services
The Global RIB will be used for an MPLSoDMVPN tunnel that will be covered in a later post.

Layer 2 Design

Explanation:

Configuration for VLAN 10, 20 and 30 won’t be covered here. That’s for the upcoming post.
This post will cover:
- VLAN configuration for the uplink FW peering interfaces (3001 - 3004)
- VLAN configuration for the downlionk Linux Server peering interfaces (4001 - 4003)
Only configuration for bastuklubben will be covered. “LBS” is just included in the drawing to show that the setup can scale to multiple servers.
The vmbr1 is a OVS bridge interface defined in Proxmox VE. To learn how to setup the network on the Proxmox side, view Proxmox VE Setup Part 2: Network

VyOS Configuration

VRF Configuration

Define VRFs:

set vrf name NMS table '110'
set vrf name INT table '120'
set vrf name PUB table '130'

Note: This configuration will be, in Cisco terms, a “VRF Lite” setup. Full blown MP-BGP support with MPLS underlay will be added in a later post.

VLAN Interface Configuration

Uplink Interface

VLAN interface configuration, as you can imagine, is not much more complex than physical interface configuration. You just add “vif <vlan ID>” to the commands:

set interfaces ethernet eth1 vif 3001 address '2001:db8:1234:1:1::2/64'
set interfaces ethernet eth1 vif 3001 address 'fe80::1:2/64'
set interfaces ethernet eth1 vif 3001 address '10.1.255.1/31'
set interfaces ethernet eth1 vif 3001 description 'VLAN 3001 Management'
set interfaces ethernet eth1 vif 3001 vrf 'NMS'

set interfaces ethernet eth1 vif 3004 address '2001:db8:1234:4::2/64'
set interfaces ethernet eth1 vif 3004 address 'fe80::4:2/64'
set interfaces ethernet eth1 vif 3004 address '10.4.255.1/31'
set interfaces ethernet eth1 vif 3004 description 'VLAN 3004 Global RIB'

Explanation:

VLAN 3001 - 3004 are point-to-point links. I still use /64 prefixes for IPv6 because it makes sense to me.
VLAN 3004 is in the global RIB so it doesn’t have a VRF linked to it.
The IPv4 address is only added so I can define a router ID for OSPFv3 and BGP. It’s not used to propagate any routes.

Downlink Interface

set interfaces ethernet eth1 vif 4001 address 'FE80::4001:1/64'
set interfaces ethernet eth1 vif 4001 description 'Downlink MGMT'
set interfaces ethernet eth1 vif 4001 vrf 'NMS'

Explanation:

The downlink interface doesn’t require an IPv6 global unicast address because OSPFv3 uses only link-local addresses as next-hop. Technically it’s not required on the uplink either but I configured it anyway for easier ICMP testing.
There is no downlink VLAN for the global RID at the moment. This will be configured later when it’s time to setup the DMVPN tunnel.
These interfaces will be a shared segment since more than one Linux server can peer with the VyOS router.

Note: There is no use including configuration for the other VRF’s, since the configuration is practically identical.

BGP Configuration

Global RIB Configuration

# Set system parameters:
set protocols bgp parameters log-neighbor-changes
set protocols bgp system-as '65001'
set protocols bgp parameters router-id '192.168.0.1'

# Neighbor Parameters:
set protocols bgp neighbor 2001:DB8:1234:4::1 local-as 65002
set protocols bgp neighbor 2001:DB8:1234:4::1 remote-as '65000'
set protocols bgp neighbor 2001:DB8:1234:4::1 timers holdtime '45'
set protocols bgp neighbor 2001:DB8:1234:4::1 timers keepalive '15'
set protocols bgp neighbor 2001:DB8:1234:4::1 address-family ipv6-unicast soft-reconfiguration inbound
set protocols bgp neighbor 2001:464f:6f83:4::1 address-family ipv4-unicast soft-reconfiguration inbound

# IPv6 address-family parameters:
set protocols bgp address-family ipv6-unicast aggregate-address  2001:DB8:1234:4000::/52 summary-only
set protocols bgp address-family ipv6-unicast redistribute ospfv3

# IPv4 address-family parameters:
set protocols bgp address-family ipv4-unicast aggregate-address 10.4.0.0/16 summary-only
set protocols bgp address-family ipv4-unicast redistribute ospf

Explanation:

IPv4 is not so much in use but this demonstrates how you are using only an IPv6 neighbor to carry updates for both IPv6 and IPv4 address families.
For an experienced network engineer, the commands should be fairly self-evident.

Note: When I wrote about BGP configuration on pfSense, I used IPv4 addresses for the neighbor configuration. To use IPv6 addresses instead, you need to toggle this setting from IPv4 to IPv6:

VRF NMS Configuration

BGP configuration inside VRFs is the same, but “vrf name NMS” has been pre-pended to all commands:

# Set the AS number and router ID:
set vrf name NMS protocols bgp system-as '65002'
set vrf name NMS protocols bgp parameters router-id 10.1.255.1

# Neighbor Parameters:
set vrf name NMS protocols bgp neighbor 2001:DB8:1234:1::1 local-as 65002
set vrf name NMS protocols bgp neighbor 2001:DB8:1234:1::1 remote-as '65000'
set vrf name NMS protocols bgp neighbor 2001:DB8:1234:1::1 timers holdtime '45'
set vrf name NMS protocols bgp neighbor 2001:DB8:1234:1::1 timers keepalive '15'
set vrf name NMS protocols bgp neighbor 2001:DB8:1234:1::1 address-family ipv6-unicast soft-reconfiguration inbound
set vrf name NMS protocols bgp neighbor 2001:DB8:1234:1::1 address-family ipv4-unicast soft-reconfiguration inbound

# IPv6 address-family parameters:
set vrf name NMS protocols bgp address-family ipv6-unicast aggregate-address 2001:DB8:1234:1000::/52 summary-only
set vrf name NMS protocols bgp address-family ipv6-unicast redistribute ospfv3

# IPv4 address-family parameters:
set vrf name NMS protocols bgp address-family ipv4-unicast aggregate-address 10.1.0.0/16
set vrf name NMS protocols bgp address-family ipv4-unicast redistribute ospf

Explanation:

On Cisco IOS the router-id would normally be the same for all VRFs. InVyOS however, you can configure different router ID per VRF.

OSPFv3 Configuration

Only OSPF configuration under VRF NMS will be shown as there currently is no downlink interface in the global RIB to activate OSPFv3 on.

OSPFv3 parameters:
set vrf name NMS protocols ospfv3 default-information originate
set vrf name NMS protocols ospfv3 interface eth1.4001 area '0'
set vrf name NMS protocols ospfv3 interface eth1.4001 cost '10'
set vrf name NMS protocols ospfv3 interface eth1.4001 network 'broadcast'
set vrf name NMS protocols ospfv3 log-adjacency-changes
set vrf name NMS protocols ospfv3 parameters router-id '10.1.255.1'

That’s about it I think. Note that the OSPFv3 configuration for the Linux server will be covered in the next post.

Verification

Some verification commands you can use:

# Show routing table:
show ipv6 route
show ipv6 route vrf NMS

# Show BGP:
show bgp
show bgp vrf NMS

show bgp summary
show bgp vrf NMS summary

show bgp neighbors
show bgp vrf NMS neighbors

# Show OSPFv3
show ipv6 ospfv3 neighbor
show ipv6 ospfv3 vrf NMS neighbor

Explanation:

There are more BGP and OSPFv3 commands available but these are the ones you use the most

And of course, I have to ping from the VyOS router to the Internet to verify connectivity on all VRFs:

vyos@LBS-RO1:~$ ping 2620:FE::FE
PING 2620:FE::FE(2620:fe::fe) 56 data bytes
^C
--- 2620:FE::FE ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5103ms

vyos@LBS-RO1:~$ ping 2620:FE::FE vrf NMS
PING 2620:FE::FE(2620:fe::fe) 56 data bytes
64 bytes from 2620:fe::fe: icmp_seq=1 ttl=60 time=10.8 ms
64 bytes from 2620:fe::fe: icmp_seq=2 ttl=60 time=10.0 ms
...

vyos@LBS-RO1:~$ ping 2620:FE::FE vrf INT
PING 2620:FE::FE(2620:fe::fe) 56 data bytes
64 bytes from 2620:fe::fe: icmp_seq=1 ttl=60 time=4.47 ms
64 bytes from 2620:fe::fe: icmp_seq=2 ttl=60 time=3.76 ms
...

vyos@LBS-RO1:~$ ping 2620:FE::FE vrf PUB
PING 2620:FE::FE(2620:fe::fe) 56 data bytes
64 bytes from 2620:fe::fe: icmp_seq=1 ttl=60 time=50.1 ms
64 bytes from 2620:fe::fe: icmp_seq=2 ttl=60 time=7.81 ms
...

Note: The global RIB does not have Internet connectivity on purpose.

Appendix

Caveats with OSPFv3

Point-To-Multipoint network type not supported on OSPFv3

On my version of VyOS, 1.5-rolling-202502110508, The supported network types for OSPFv3 are:

Point-to-Point
Broadcast

The supported network types for OSPFv2 are:

Point-to-Point
Broadcast
point-to-multipoint
non-broadcast

This is not ideal for my setup as I’m only using IPv6 and I would like to have a point-to-multipoint network segment to control communication between Virtual Servers.

Workaround

VyOS is based on FRR and in the latest version of FRR you can actually configure point-to-multipoint and static neighbors, but you can’t configure it from VyOS CLI. You can enter shell and the FRR daemon and configure it from there:

vyos@LBS-RO1:~$ /bin/sh
sh-5.2$ vtysh
LBS-RO1# conf t
LBS-RO1(config)# interface eth1.4003
LBS-RO1(config-if)# ipv6 ospf6 network point-to-multipoint

Even though you can’t configure it from VyOS CLI, verification commands displays the correct output:

vyos@LBS-RO1:~$ show ipv6 ospfv3 vrf PUB neighbor 
Neighbor ID  Pri  DeadTime  State/IfState      Duration  I/F[State]
10.9.0.1       0  00:00:31  Full/PtMultipoint  00:00:18  eth1.4003[Pt...

The problem however is that if the configuration changes, or the router reboots, the changes made through vtysh will be removed.

Feature request created

I have opened a request for these missing features: https://vyos.dev/T7265

OSPFv3 neighbor relationships doesn’t form if using identical link-local addresses on different interfaces

I use to hardcode the IPv6 link-local addresses on my VLAN interfaces for easy identification. One day I decided that I'm going to change the link-local addresses on the uplink router (VyOS) for all VLAN interfaces to be FE80::1/64. This should not be any problem since they are only local to the link.

However, OSPFv3 neighborship doesn't form after change, or it forms only on one of the interfaces. The others get stuck in INIT state on one side.

Workaround

Continue using different Link-local addresses per interface

Bug Report created

https://vyos.dev/T7298

VyOS/FRR does not support OSPFv3 IPv4 embedded IPv6

Cisco can transport IPv4 prefixes through IPv6 OSPF networks according to RFC 6992, similar the way BGP does it. FRR doesn’t seem to support this. AFAIK it seems like only Cisco have managed to program OSPFv3 the correct way.

Open Source Is Fun

Discussion about this post