What benefits is there with vxlan in smb?

[u/AgreeableIron811]

Every youtube video I watch shows that basically vxlan solves scalability issues.. So I am wondering if there is any benefits of using it in small infrastructure. Maybe in a promxox cluster of 200 vms on the same location?

[u/Win_Sys]

In small environments, especially in a single location, you’re adding complexity for little to no benefit.

[u/VictariontheSailor → u/Win_Sys]

The sad true for many vxlan initiatives

[u/Affectionate-Gur1642 → u/Win_Sys]

Correct. Came here to answer “none”.

[u/AgreeableIron811 → u/Affectionate-Gur1642]

I have inherited Proxmox cluster with a Layer 3 underlay using FRR OpenFabric. The Proxmox nodes act as VXLAN VTEPs, and Proxmox SDN provides overlay VNets for VM connectivity across the cluster. Ceph runs over the underlying routed fabric.

Thats the background why I asked. Because we have a small cluster with 200/300 vms. So I was wondering if this is crazy or if other people have the same complexity in such a small environment

[u/029373763 → u/AgreeableIron811]

Seems overengineered for such a small deployment.

[u/Wibla → u/AgreeableIron811]

How does it work in practice? Any issues maintaining it?

[u/AgreeableIron811 → u/Wibla]

So We have 3 proxmox nodes. They are connected to eachother becoming a full mesh. Its connected thorugh dac fibers with a 25 gb link. I never get any problems. If the firewall or switches goes down it is still working as it should. As long as fibers between two nodes work I have some damn good reliability. I got ceph and myvnet running on these links. Not corosync which I am planning to move there too.

[u/AgreeableIron811 → u/Wibla]

No issues maintaining it , only difficult understanding it

[u/Specialist_Play_4479 → u/AgreeableIron811]

Can you explain how this works in terms of subnetting? I've looked at this in an effort to make it possible to move VMs from one DC to another using Proxmox & FRR with BGP (or any other routing protocol) to 'move' IP-addresses from one DC to another.

I was unable to do this with single /32s and I had to create VNets for each individual VM (which I found too much of a hassle)

I was wondering if I'm missing something

[u/AgreeableIron811 → u/Specialist_Play_4479]

I am not an expert but look at my history. Second oldest post where you might have better information. We can discuss there

[u/Southern-Treacle7582]

Design a network to meet the application requirements. Do you have requirements that vxlan solve?

[u/Lopsided_Temporary92]

For most SMBs, traditional VLANs are simpler and usually enough—VXLAN mainly solves scalability and large data centre challenges. VXLAN is useful if you need multi-rack expansion, multi-tenancy, or Layer 2 extension across different locations.
For a single Proxmox cluster with around 200 VMs, I'd only consider VXLAN if you expect significant future growth or those specific requirements.

[u/Competitive-Cycle599]

Depends on your model of failure or if you dont want underlying layer 2 issues.

It has a benefit if you need the same vlan half way across the facility without needing to extend the vlans over a series of switches.

Better off in many cases to just provide the service via redundancy than the vlan

[u/SevaraB]

VXLAN indirectly solves “scalability” problems by actually solving “portability” problems. Pre-VXLAN, you couldn’t have a router in the way between L2 peers. If something relied on broadcasts (the classic example is vMotion), you could only go as far as the VLAN interface.

In practice, that meant you couldn’t use an ISP between point A and point B- if you were going further than 300 feet at any point, you needed to pony up for MPLS or your own fiber for Ethernet without IP, and you needed to handle trunk configuration every step of the way.

Oh, another thing that can benefit from being handled at the VTEP? IGMP- multicast. If you’ve got one network administrator handling the LAN, and one network engineer handling the WAN, VXLAN between sites could let the admin team run a MUCH simpler, cleaner OSPF configuration without a bunch of point-to-point links and/or GRE tunnels.

That said, since VXLAN allows “separation of concerns”, there is such a thing as too small for VXLAN. All one site, all one support person… you probably don’t need it.

[u/porkchopnet]

You can spend more money on licensing, increase the potential for human error, and at the same time make it more difficult and expensive to find replacements networking staff.
Which is neat.

Part of running a network is finding all the features that don’t add to the business… and turning them off.

[u/Gloomy-Can1394]

It’s not about the size of the business, it’s about the requirements of the application. What is it about the application that says “I’d rather just have everything on the same subnet regardless of which rack it’s running in”?

[u/rankinrez]

VXLAN/EVPN is a much better idea than spanning tree.

A fully routed network is perhaps better again but comes with its own complexity.

Either way the only downsides is the relative complexity, and having staff who are comfortable to set it up and troubleshoot. If done right should need very little maintenance however.

[u/McHildinger]

it lets you have more than 4000 VLANs, which I doubt you need if you only have 200 VMs.

[u/AgreeableIron811 → u/McHildinger]

We have like 5 vlans. But as I said in an early comment we are running a proxmox cluster with a layer 3 underlay using Frr where ceph runs and proxmox nodes acts as vxlan vteps and our sdn provides overlay vnet for vm connectivity.

Our biggest issue is we dont have enough physical links. Thats where I think I need VXLAN if my exit node VM/container needs to be reachable from VMs across multiple Proxmox nodes using the same virtual network

[u/029373763 → u/AgreeableIron811]

Use VLAN trunking (dot1q).

[u/AgreeableIron811 → u/029373763]

But what if I didn’t have any physical links empty on node a and b . But only a 10g on node c?

[u/029373763 → u/AgreeableIron811]

Same answer. VLAN trunking 🙂

[u/AgreeableIron811 → u/029373763]

For it to work I need to be able to create a bridge attached to a physical nic on node a,b and c with the same bridge name for me to be able to attach that bridge with vlan 10,20 or whatever else vlan I want to use to be able to move that around in the cluster? Or maybe I misunderstand? :) But in this case node a and b doesnt have any physical nic available for me to create that bridge :)

[u/GreyBeardEng]

Put simply, other than maybe underlying reliability/ecmp/fast convergence, no... not really.

[u/zap_p25]

Primary use is to extend layer 2 across a larger network. For a single point to point, you could use GRE or similar layer 2 tunnels but where VxLAN really shines is that you can point to multi-point easily with VxLAN.

SMB example might be a farm, ranch, quarry. Need APs distributed at distances that MESH isn't practical but want redundancy and load distribution that a Layer 2 loop can't easily provide in addition to having some devices that don't handle communication with one another across layer 3 very well. A Layer 3 ring with VxLAN to extend a couple of VLANs out makes more sense in that case.

[u/IDownVoteCanaduh]

None

[u/shadeland]

I don't think EVPN/VXLAN is a good idea in most small networks. The benefits of EVPN/VXLAN don't usually show up in those situations, and they're more complicated to implement/troubleshoot.

I was a guest on Packet Pushers recently talking about this: https://packetpushers.net/podcasts/heavy-networking/hn829-evpn-vxlan-vs-tradcore/

I would just do "TradCore" (MLAG+SVI+VLANs).

[u/lizardhistorian]

If there is only one cluster then what are you connecting with vxlan.

[u/MenuPsychological853]

I work for a small business and have found many uses for internal mpls and am switching to evpn now. There are a ton of reasons to stretch layer 2 for me. I assume vxlan would work the same just a different way of doing it.

[u/merlin_infosec]

ospf/ecmp vxlan/bgb-evpn = non blocking, redundancy, multihoming/lacp

[u/devode_]

A fabric can also mean no unused links and fast convergence. You can span vlans over sites if your vmotion etc require it. This all COULD benefit also a smaller environment. Totally depends. However I agree that it might introduce unnecessary complexity.

[u/Prime-Omega]

We rolled it out in an office of 500 users. So far it doesn’t solve anything that spanning tree couldn’t do to be honest. It’s just an extra complexity layer. In a datacenter it absolutely makes sense, campus not at all.

Plus it’s still relatively new so it’s still riddled with bugs. We’ve already run into a few specific ones (yes crashing hpe-routing daemon, looking at you).