If you ever wonder how I test networking devices, including Wi-Fi broadcasters (routers and access points) and switches, for reviews on this website, you’re reading the right post.
I’ll reveal here my testing methods and the equipment I used. If you pay close attention, you might be able to build similar testing “labs” yourself. But it would be best if you took my word for it.
This post is mainly for your information in case you want to know how I come up with the performance numbers and the rating for each review. Spoiler: It’s easier, though more time-consuming than you think.
Dong’s note: The content of this post was originally part of the post on Wi-Fi and Internet speed testing and was moved out as a separate post on November 9, 2022, to clarify the subject and make it easier for future updates.
Networking speed testing: Super-fast network car is a must
Wi-Fi Speed tests in Dong Knows Tech’s reviews
It usually takes me a week to finish evaluating a Wi-Fi device and about three days for a network switch — I only review Multi-Gig switches.
During this time, I generally use the hardware as my personal device for an extended amount of time. This testing portion is mostly about throughput speeds, which are only part of what I do during a review.
In any case, I put all reviewed hardware through the same process. And I use the same testing method mentioned in the speed testing post.
Specifically:
- I use a “server” unit, a Windows 10 computer, that connects to the test hardware in question via a wired connection, which is always Gigabit or faster (Multi-Gig) when applicable.
- For the sustained throughput rates, I use various high-end (Wi-Fi) clients to copy particular test files from the server to the client and back. I time the process to figure out the speed in megabits per second (Mbps.)
Below is the general specs of my equipment, which I upgrade pretty frequently — I will update this post when the changes are significant enough.
Server specs for networking speed tests
My server is a custom-built computer with the following specs:
- CPU: Intel Core i5-11400 Processor
- RAM: 32GB DDR4
- Storage: 2TB Samsung 990 Pro.
- LAN connection: ASUS XG-C100C 10GBASE-T network adapter card.
This server hosts the test files, which I copy to clients via the test router’s wireless connection to determine the Wi-Fi speeds.
This server connects to the test device via a wired connection, which varies from Gigabit to 10Gbps, depending on the hardware. I don’t test any device that has sub-Gigabit ports.
Clients’ specs for networking speed tests
The test client is a computer that connects to the test hardware — a Wi-Fi router, an access point, or a switch — using the fastest connection method possible, wireless or wired.
Generally, I use three clients for official throughput testing. They all use a relatively high-end Intel CPU, 16GB or more RAM, and a top-tier NVMe SSD as the primary storage.
- Client #1 (wired and Wi-Fi): This is a desktop computer, sharing the same specs as the server above, plus an Asus PCE-AC88 4×4 Wi-Fi 5 adapter — the fastest Wi-Fi 5 client on the market. I use this mainly for a close-range (10 feet or shorter) throughput test of a router’s Wi-Fi 5 band. This client also has a 10GbE BASE-T card and a built-in Gigabit port that I use for testing wired connection speeds.
- Client #2 (Wi-Fi only): A Dell XPS 15-inch laptop. This machine has top-of-the-line specs, built-in Wi-Fi 6E, and a Netgear A700 3×3 Wi-Fi 5 USB adapter.
- Client #3 (Wi-Fi only): An Intel-based Apple MacBook Pro 16 with a built-in Wi-Fi 5 adapter. This laptop runs both macOS and Windows (via Bootcamp) and works for compatibility tests.
For additional Wi-Fi tests, I also use over half a dozen other laptops, tablets, USB Wi-Fi adapters, and phones of different Wi-Fi standards, including some extra Wi-Fi 6 and Wi-Fi 6E devices.
Networking speed test (Wi-Fi broadcaster and switches): Data and configurations
The following are the data and how I conduct tests on standalone networking devices, including Wi-Fi broadcasters (routers, access points, extenders) and switches.
Depending on a particular device, there might be more testing. For example, a Wi-Fi 6E router will also be tested as though it were a Wi-Fi 6 or Wi-Fi 5 router. But all of them have to go through these:
- I upgrade the hardware to the latest firmware available.
- I use single large files for test data, which generally take less time to copy than multiple small files. Depending on the tests and how fast a router is, I use a 2GB, 6GB, 10GB, 20GB, or 60GB test file. Generally, I use the smaller test file when a bigger one would make the test take too long.
- Wi-Fi broadcasters are first tested with settings that favor speed (and not compatibility, which is generally the default) using each available Wi-Fi channel separately. After that, I also try them using the default (Auto) settings.
- All switches are tested in unmanaged mode (applicable only to managed switches.)
- For official performance scores, I test the hardware with just one connection at a time.
- For Wi-Fi broadcasters: I place this client at two specific locations that are (a) less than 10 feet (3 m) and (b) 40 feet (12.2 m) away from the router — within the line of sight. I do multiple tests at different times during the day and on different days of the week and use the highest consistent numbers as the final scores.
- For switches: The client is connected to the switch via a 7-foot CAT6a or CAT5e cable. Also, the best throughput rates are recorded. I might also do additional tests using Gigabit client or CAT5e cables might.
- If a router has two or more Multi-Gig LAN ports, these ports are also tested, like the case of a Multi-Gig switch.
Besides the performance, I use the hardware for an extended period in real-world anecdotal testing, from a few days to even a few weeks. During this time, Wi-Fi broadcasters are used with clients of different standards and tiers, placed at various locations.
I report network performance in megabits per second (Mbps).
Wi-Fi mesh speed test: Hardware placement
Generally, I test the multi-unit Wi-Fi system in the wireless setup. That’s because, in the wired backhaul configuration, the extender unit (satellite) performs similarly to the router unit — you can use the router’s throughput numbers as that of the satellite.
For standard testing, I place the satellite 40 feet away from the router unit. After that, the test client is set at 10 feet and 40 feet away from the satellite. All are within lines of sight with no objects or walls in between them.
Also, I use the star topology for the testing, meaning the satellite units (if more than one) are placed around the router unit. In other words, a 2-pack mesh will generally deliver the same test results as a 3-pack one.
When the mesh hardware is Tri-band — where it has an additional 5GHz band — I will test it so that one of the 5GHz bands works as the backhaul and the other works as the front haul. The idea is to get the best possible performance.
In exceptional cases, I also do additional testing on the hardware via wired backhauling.
Backhaul vs fronthaul
A Wi-Fi connection between two direct devices occurs in a single band, using a fixed channel, at any given time.
Generally, when you use multiple Wi-Fi broadcasters, like in the case of a mesh network, there are two types of connections: fronthaul and backhaul.
Fronthaul is the Wi-Fi signal a mesh hub broadcasts outward for clients or its network ports for wired devices. It’s what we generally expect from a Wi-Fi broadcaster.
On the other hand, backhaul, a.k.a backbone, is the link between one broadcasting hub and another, be it the main router, a switch, or another satellite hub.
This link works behind the scene to keep the hardware units together as a system. It also determines the ceiling bandwidth (and speed) of all devices connected to the particular satellite hub.
When a Wi-Fi band handles backhaul and fronthaul simultaneously, only half of its bandwidth is available to either end. From the perspective of a connected client, that phenomenon is called signal loss.
When a Wi-Fi band functions solely for backhauling, it’s called the dedicated backhaul.
In a mesh system, only traditional Tri-band hardware — those with an additional 5GHz band — can have a dedicated backhaul band without ostracizing clients of the same band.
Generally, it’s best to use a network cable for backhauling — wired backhauling. And that’s an advantage of mesh hardware with network ports. In this case, a hub can use its entire Wi-Fi bandwidth for front-hauling.
In networking, network cables are always much better than wireless in speed and reliability.
Similar to the case of standalone broadcasters, I use the mesh in different real-world scenarios, with anecdotal tests, to understand how it works and use that experience for the review.
Important note on Wi-Fi speed testing
Again, I measure the official test score using a single high-speed Wi-Fi client using a single band at a time. This client pulls data from the test server connected to the router (or access point) via a network cable.
Two things to note:
- Using a single client is the only way to consistently figure out all broadcasters’ speeds. If I use multiple clients, due to different speed tiers, standards, and the fact Wi-Fi bandwidth is shared, it’s impossible to come up with relevant throughput numbers to say if this router is faster than the other, etc.
- A router (or AP) with a Multi-Gig LAN port has a significant advantage in test scores over those with only Gigabit ports. That’s because, without this super-fast port, the hardware’s performance is limited at Gigabit.
In any case, the scores reported in my reviews are likely those of the best-case scenario. They show the router’s Wi-Fi band’s total real-world bandwidth.
You can turn most Wi-Fi routers with a USB port into a mini NAS server.
Router network-attached storage (NAS) speed test
If a router has a USB or eSATA port that can host a storage device, I test the performance of its NAS feature, too.
In this case, for consistence’s sake, the following is the standard way I go about it:
- I use at least two random portable SSDs, out of this list, for the test and pick the one with higher consistent scores as the official numbers. So far, any portable SSD has proved to have way higher speeds than the router’s USB port. In other words, regardless of the drive I used, all routers’ NAS performances remained the same.
- The drive is formatted in NTFS. If the router doesn’t support this file system, I’ll use the applicable one and note that in the review. So far, all routers I’ve reviewed support NTFS.
- I perform the test using Windows’ File Explorer (a.ka. Windows Explorer). It’s a simple drag and drops copy test via the standard Server Message Block (SMB, a.k.a. Samba) protocol with a 20GB single file as the data.
- The test computer has the old and insecure SMBv1 disabled by default, meaning the router must support SMBv2 for the test to work. (If SMBv1 must be enabled, I’ll note that in the review.)
- I do the test using a wired Gigabit connection. If the router has a Multi-Gig port, I’ll use that port, too.
- I perform each test (write and read) at least three times and pick the highest consistent numbers (within 5 percent) as the router’s scores.
- I report NAS performance in megabytes per second (MB/s). (You can find the scores of all routers I’ve tested in this post of the best routers with NAS feature.)
Note: The test for the router’s NAS feature is just for reference in terms of raw speeds. I use a real NAS server (and I’d always recommend that you do, too.) That said, I only use the router’s USB port for a short time and do not try all available applications or functions, nor do I check for conflicts or issues.
Power consumption measurement
Since mid-2021, I’ve started adding real-world power consumption on reviewed hardware.
I measure this by connecting the hardware via a simple power meter and reporting the result after 24 hours.
It’s worth noting that the measurement is intended to give a rounded ballpark number of how much energy the device uses in watt-hour (Wh.)
A device consumes more or less energy depending on its loads. In my case, the reviewed hardware is likely being used more intensively than average.
An Asus ZenWiFi Pro XT12 router is being tested.
There are a lot of things going on behind that fancy lighting.
Networking speed test: You can’t put your finger on it.
After years of working with hundreds of networking devices, I have to admit that the process can get repetitive and tedious. It’s also not 100% accurate. Also, no one can try every scenario, considering a router can have many features and settings.
It’s impossible to say how fast a router’s Wi-Fi is because so many factors and elements can affect a test’s outcome. That’s not to mention the fact a router can change dramatically via firmware updates.
For this reason, I try to keep my testing as consistent as possible. Ultimately, I aim to show how a particular device does against others.
In other words, my testing doesn’t mean to represent your experience of a device, such as a Wi-Fi router, in terms of throughputs. Instead, it describes how better, or worse, a choice it is, among others, at the time of the review.
Again, speed testing is just one of many things I do to evaluate networking devices. I always take the time to have real-life experiences with them before publishing my reviews. You can count on that.
Wanna be in touch? Sign up for DKT newsletter!
Source by dongknows.com
