Skip navigation

Category Archives: Mikrotik

Still no good news! I have decided to setup 2 x86 Mikrotik test boxes with 2 Atheros antennae installed in each. I am then  going to configure the same Mangle (Nth) via packet marking and will see the results.

The only difference from my live setup is that the antennae are separate boxes (RB333) over a quad NIC in the Mikrotik Load Balancer / Mangle box.  I am still waiting for my fourth WiFi card to arrive since i seem to have taken all their available stock..

….. More weeks passed and i can officially say that Mikrotik Load Balancing DOES NOT WORK ! The issue is always the same. I have used Wireshark and confirmed the issue. It can be Bonding, Mangle or whatever but Mikrotik devices do not reorder packets at the destination, which results in a very high loss of bandwidth. Bonding is slower since there is packet encapsulation, Mangle will give a slightly higher bandwidth but will not give you redundancy once a link fails (you manually have to change the rules or else packet loss is experienced). Over all it is, was and will remain a disaster for load balancing. Whoever makes use of any form of load balancing, make sure you keep an eye on the end systems since there are taking on the load to reorder or re-request packets .

I have to point out that this issue is present when load balancing is performed on wireless links (since each wireless link’s is not as stable as a wired link –  which creates different delays on each wireless link –  which makes packets arrive faster on some link when compared to others) hence why when i tested this setup on wired links i had no problems. Also note, that with UDP packets are lighter and that UDP is connection-less so performance on UDP will look good. In reality, no connection is UDP only, so this means that you will never really use the bandwidth you see when you are testing under UDP. TCP bandwidth tests are a totally different story.

PS: A number of months passed and the issue still seems to be present. Another setup such as bonding over WDS was tested but the same results were obtained –  time to move away from Mikrotik for this setup.

Case Closed.

The past three weeks were no easy, I had a large setup of Mikrotik devices to setup for work, and i am going to briefly point out the issues /  glitches i had with these *damn* boxes.(RouterOS 3.13)

1) Bonding and Eoip –  Does not work. You can bond any number of antennae via EoIP but simply because one pair antennae achieved 17Mbps (TCP), bonding 4 pairs of Antennae and adding them to a bonding  /  Eoip will only give you the bandwidth of a crawling snail. The more antennae and bonding interfaces you add in the equation the worse the situation gets. I have tested it out with TTCP bandwidth tester and results were bad – oh my… so bad. The issue is, the bandwidth (of all four antennae) will be there, but to benefit from this bandwidth you will have to open several connections from a different source to a different destination all the time. (connection tracking and all other crap was disabled – these mikrotik boxes were properly configured)

So, what could  be used instead of bonding or Eoip?  OSPF you may ask? –  Nope!

2)  OSPF load balancing works properly when all interfaces are “wired” (wireless is another story)

I check the cost of the link along with the Administrative Distance settings. (load balancing depends on those), but once again –  no! load balancing over OSPF Mikrotik does not work. The boxes simply decided to pick a link from the four available and send all traffic on that one. Sometimes it even changes link to one of the available ones randomly.

So what is left? All that is required here is simply load balance over 4 wireless links, and yet it seems so difficult to achieve.

The last option is Mangle. There are different flavours of Mangle out here, It seems to be mostly utilised when once wishes to load balance over multiple ISP gateways  –  using NAT. I came across this blog which helped me…

http://mustnofee.com/story/load-balancing-mikrotik-3x

….but did not work since my situation is different. I do not need NAT since all my antennae are using static routing, and Mangle / connection / connection marking did not work since i am not using the above link . This is a LAN, and each end of the setup only sees one IP address. The source or destination.

The Mangle required here is via packet / routing marking:

ip/firewall/mangle > print detail

0   chain=prerouting action=mark-packet new-packet-mark=pktlink1
passthrough=yes in-interface=Eth-Local 1 nth=2,1

1   chain=prerouting action=mark-routing new-routing-mark=mk-link1
passthrough=no in-interface=Eth-Local packet-mark=pkttolink1

2   chain=prerouting action=mark-packet new-packet-mark=pktlink2
passthrough=yes in-interface=Eth-Local

3   chain=prerouting action=mark-routing new-routing-mark=mk-link1
passthrough=no in-interface=Eth-Local packet-mark=pktlink2

IP Route

1 A S  dst-address=192.168.10.1/28 gateway=192.168.1.1 interface=Eth2
gateway-state=reachable distance=1 scope=255 target-scope=10
routing-mark=mk-link2

2 A S  dst-address=192.168.10.1/28 gateway=192.168.2.1 interface=Eth1
gateway-state=reachable distance=1 scope=255 target-scope=10
routing-mark=mk-link1

Although the above worked, for some odd reason the antennas still seem to be giving me issues. The ptp antennae are running at 50db signals, with a registration of 54mbps/54mbps. So if the wireless peers are working correctly, why is it still necessary to open multiple connections to utilize the full capacity of my 4 links?

(The reason is that by opening multiple links, the bandwidth seems to increase for the simple reason that there are more packets bouncing around for retransmissions but the total bandwidth which can be actually utilised is minimal. Seeing large numbers does not mean that the packets are actually arriving without errors and that the total usable bandwidth is that “large figured number”.

Continued on next thead  ” Mikrotik –  Load Balancing Over Multiple Interfaces (Part 2) “

Since i am a fan of Mikrotik devices and i can say that i have now setup quite a number of topologies, i decided to provide links to the most common configurations that i have setup.

WDS Bridging mode: http://wiki.mikrotik.com/wiki/Transparently_Bridge_two_Networks

Using netinstall: http://www.mikrotik.com/testdocs/ros/2.9/guide/netinstall.pdf

Interface bonding: http://www.mikrotik.com/testdocs/ros/2.9/guide/netinstall.pdf

VRRP: http://www.mikrotik.com/testdocs/ros/2.9/ip/vrrp.php

Firewall: http://wiki.mikrotik.com/wiki/Firewall

Supported Hardware: http://www.mikrotik.com/testdocs/ros/2.9/guide/driverlist.php

Personally i am running a 7 interface Mikrotik box (PCI Nics). It is running as a firewall/router. RouterOS version is currently the latest (3.11 at the time of this post). This Mikrotik is running on an AMD Athlon XP 2200 (1.8Ghz with 1Gigi Ram)

I have another Mikrotik router which is not active since i usually use it as a test machine.

The non active Mikrotik has 2 Realtek PCI NiCs and 1 PCI Wfi Card running on an old 128MB Ram Pentium Pro. The Wifi PCI card is 100% compatible with the Mikrotik Kernel and supports all features that run at 2.4Ghz.  – The card is a  Trendnet TEW-443PI (Atheros Chipset)

I have experimented OSPF and BGP but i can say that i never come accross a setup where i really required such protocols. Interface Bonding over multiple wireless links and static routing is still my favourite 🙂