Speed check.

A transfer goes only as fast as the slowest part of its path. The speed check measures each part of your server, tells you in one sentence which part is the limit, and says what to do about it. It is included on every plan, Free mode too.

Before you run it

The check is a real test. It is safe on a live server, and it is not free:

• It writes a test file of a few gigabytes to each location it tests: the staging area, each storage location and the data volume. The size is 2 GB by default, and an administrator can change it.
• It works your disks and processor hard while it runs.
• It takes about 2 minutes on a normal server. Slow disks and many storage locations take longer. Each card on the screen fills in as its test finishes, so you can watch it work.
• Active transfers slow down while it runs. If transfers are running, Farwing asks first: A check slows active transfers for about 2 minutes. Run it anyway?
• A scheduled check is skipped when the server is busy and tries again later, so it never slows your busiest hours.
• Test files are always deleted. They live in a hidden .farwing/speedcheck folder inside each location. If the server stops during a check, it removes the leftovers the next time it starts.
• A location without enough free space is skipped, and the result says why. Free some space or lower the test size to include it.
• A cloud bucket is tested only if you include it. When you run the check, tick Include next to each bucket to test. The test uploads one temporary file, reads it back and deletes it, and your provider may charge for that data.

Run it

• In the portal: Admin → Transfers → Speed check, then Run the speed check.
• From the dashboard: a small card shows the last result's headline and date, with a Run check button.
• In the setup wizard: the last step offers Check this server's speed (2 minutes).
• From the command line: farwing doctor runs the same local checks and prints the same result as text. farwing doctor user@host also checks the far end over SSH. It works on every plan.
• From the API: POST /api/v1/speedcheck starts a check, GET /api/v1/speedcheck/{id} returns its result, and GET /api/v1/speedcheck lists past results. These need a license with the API.

farwing doctor
farwing doctor [email protected]

Running a check by hand is on every plan. Scheduled checks need a Team license or higher. See Scheduled checks.

What it measures

Receiving goes network, then processor, then the staging disk, then the destination disk or S3. Sending goes the source disk or S3, then the processor, then the network. The check measures each part on its own and compares them with your license.

Each speed is shown twice. Disk tools use MB/s and network people use Gbit/s. 1 Gbit/s is 125 MB/s, so a disk that writes 400 MB/s can keep up with 3.2 Gbit/s.

Farwing runs the checksum and the encryption together, the way a transfer does, so the processor appears as one combined number. The check measures it for one core and for the number of threads Farwing will use. The details panel also shows the processor model, the core count and the current load, and the memory: total, free and what Farwing is using now.

For S3 storage that you include, the check uploads and downloads one test object and then deletes it. It shows the bucket's region and the server's region when it knows them. If the bucket refuses or cannot be reached, the result passes on the storage service's own words. See A location was not tested.

How to read the verdict

The top of the screen is one sentence in large type. It names the part that limits your server and its speed:

This server can receive at about 3.2 Gbit/s. The limit is the staging disk (/data), which writes at 400 MB/s.
Put staging on a faster disk to reach your 10 Gbit/s license.

The line under it is the advice for that part. If nothing limits the server below your license, you see a different sentence instead: This server can use your full license. Nothing is holding it back. That is the good result. Do not look for a fault.

• The pipeline below shows one card for each part, left to right, in the order a transfer passes through them. A switch changes between Receiving and Sending. Each card shows its speed in Gbit/s, with MB/s underneath.
• The slowest card is amber and carries a Your limit tag. The others are grey with a check mark. The line joining the cards narrows at the slowest one, like a pipe.
• Your license is the last card. When every part is at least as fast as the license, the license is the limit.
• The details are folding panels, one for each part. They hold every number behind the verdict.

The line under the verdict

What to change

Under the verdict, the advice list holds plain sentences, most serious first. Each has a Why link to the matching section below. Advice marked as a limit holds your speed back. Advice marked as a warning is worth fixing. A note is information.

Staging and a storage location are on different disks

Staging and Projects are on different disks, so every file is copied a second time after it arrives.

Every upload lands in staging first. When staging and the destination are on the same disk, the file moves into place without a copy. On different disks, Farwing must copy every byte again, which costs time and disk speed.

How to fix it: put the storage location and the staging area on the same disk. If they must be on different disks, make staging the faster one.

The UDP buffer limits are small

The system limits a UDP buffer to 208 KB, which is too small to pass 1 Gbit/s.

Farwing moves data over UDP. Linux limits how much memory one UDP connection may use, and the usual default is small. On a fast link, a small buffer overflows and packets are lost. The check shows this advice when your license or link is faster than 1 Gbit/s.

How to fix it: run this on the server itself, as an administrator. Do not run it inside the container. Farwing uses the host's network, so the host's setting applies.

sysctl -w net.core.rmem_max=16777216 && sysctl -w net.core.wmem_max=16777216

It raises both limits to 16 MB. The change lasts until the next restart. To check the current values:

sysctl net.core.rmem_max net.core.wmem_max

To keep the change after a reboot, put the same two settings in the system's sysctl configuration. This creates a file and applies it:

printf 'net.core.rmem_max=16777216\nnet.core.wmem_max=16777216\n' | sudo tee /etc/sysctl.d/99-farwing.conf
sudo sysctl --system

The file /etc/sysctl.d/99-farwing.conf then holds:

net.core.rmem_max=16777216
net.core.wmem_max=16777216

A finished check, and its downloaded report, keep the limit it measured. Open the speed check page again and it adds a line starting Current setting: with the limit the server has now. Run the check again to measure with the new limit.

The server runs on Docker's bridge network

This server is running on Docker's bridge network, which adds work for every packet. Using the host network would give you better performance.

How to fix it: run the container with host networking. The compose file from the Server admin guide already does:

services:
  farwing:
    network_mode: host

With docker run, add --network host.

An S3 bucket is in another region

This S3 bucket is in eu-west-1 but the server is in us-west-2. Reading from it is slower and costs extra.

The distance adds delay to every request, and cloud providers charge for data that crosses regions. Farwing reports this only when it knows where the server runs.

How to fix it: use a bucket in the server's region, or move the server to the bucket's region.

A folder is a network share

The /files folder is a network share. Its speed depends on your storage system, not on Farwing.

The check can measure the share, but it cannot change it. Its speed comes from the storage system behind it and the network between that system and your server.

How to fix it: look at the storage system and its connection. Check the link speed, the load on the storage and the mount options. For staging, use a local disk, because every upload is written there first.

A disk is slower than your license

This disk is slower than your license: it writes at 400 MB/s, and the license allows 10 Gbit/s.

Your license allows more speed than the disk can supply. Convert to compare: 10 Gbit/s needs 1,250 MB/s.

How to fix it: move the location to a faster disk. NVMe disks are fastest, SSDs are next, and spinning disks are slowest. Several disks working together as one volume can also be faster.

The processor is slower than your license

The processor can encrypt and check about 4 Gbit/s, which is below your license's 10 Gbit/s.

Every transfer is encrypted and checked, and that work runs on the processor.

How to fix it: use a server with more cores or faster cores. If the server also runs other work, move that work elsewhere. On a virtual machine, give it more virtual cores.

The network link is slower than your license

The network connection runs at 1 Gbit/s, which is below your license's 10 Gbit/s.

A 1 Gbit/s link cannot deliver more than 1 Gbit/s, whatever the license allows.

How to fix it: connect the server to a faster port or use a faster network card. To see the speed the link negotiated, run this, replacing eth0 with the name of your network interface:

ethtool eth0 | grep Speed

A read speed may include the cache

This system would not let the check skip its memory cache, so the read speed may be higher than the disk really is.

What to do: nothing is broken. Read the number as the best the disk can do, and give the write speed more weight. The report marks these results as possibly including the cache.

A location was not tested

Projects was not tested, followed by the reason. When only part of the test finished, it says was not fully measured instead, and the numbers that were measured are still shown.

The reason depends on whether the location is a folder on this server or a cloud bucket. Its Why link opens the matching entry below. A check saved by an older version of Farwing links here instead; run the check again to get the specific reason.

A folder does not exist

/mnt/projects does not exist inside the Farwing container. On a server that does not run in a container, it says does not exist on this server.

In a container, Farwing can only see the folders that are mounted into it. A folder that exists on the host but is not listed as a volume looks missing to Farwing.

How to fix it: in a container, add the folder as a volume in your Compose file, with the same path inside the container as the location uses, then restart Farwing. On a server without a container, check that the disk is connected and mounted. If the message says the path is a file, point the location at a folder. Then run the check again.

A folder's disk has no room for the test file

1.2 GB free on /mnt/projects, and the test needs 6.0 GB.

The check asks for several times the size of its test file, so it never fills a disk that is in use. If the free space could not be read, the check writes nothing there and says so.

How to fix it: free some space on that disk, or lower the test size, and run the check again.

Farwing cannot write in a folder

This server cannot write in /mnt/projects (permission denied).

How to fix it: check that the folder is not mounted read-only, in the Compose file or on the host, and that the user Farwing runs as may read and write there. Then run the check again.

A folder's disk failed during the test

The disk failed while /mnt/projects was being timed, followed by the error from the system.

How to fix it: look at the disk's health and the system log for errors at the same time, then run the check again. The test file is deleted even when the test fails.

A bucket's cloud test was not chosen

Not tested: the cloud test was not chosen.

A bucket is tested only when you ask for it, because the test uploads a temporary file and your provider may charge for that data. This is information, not a fault.

How to test it: run the check, and in the window that asks to confirm, tick Include next to the bucket. The test file is deleted when the test ends.

Farwing may not write to a bucket

Not tested: this bucket is set to read only in Farwing, or Free mode lets Farwing read cloud buckets but not write to them.

The test has to write a temporary file, so it cannot run on a bucket Farwing may only read.

How to fix it: turn off read only in the bucket's settings under Admin → Storage → Locations. In Free mode, a license that includes S3 storage lets the check test the bucket.

A bucket's settings cannot be used

This bucket's saved settings could not be used, followed by what is wrong with them.

How to fix it: open the bucket under Admin → Storage → Locations, check the endpoint, region and access key, and press Test connection. Then run the check again.

A bucket refused access

The bucket refused access while writing the test file. The sentence ends with the storage service's own words, such as Access Denied or The request signature we calculated does not match the signature you provided.

The access key is wrong, has expired, or is not allowed to do everything the test does.

How to fix it: check the access key and secret in the bucket's settings. Make sure the key may write, read and delete objects in that bucket: on AWS, that is s3:PutObject, s3:GetObject and s3:DeleteObject. Then run the check again.

A bucket was not found

The bucket was not found while writing the test file. The storage service's words follow, such as The specified bucket does not exist.

The bucket name is wrong, the bucket was deleted, or it is in a different region from the one in its settings.

How to fix it: compare the bucket name and region in its settings with your provider's console, press Test connection, and run the check again.

The server could not reach a bucket's storage service

This server could not reach the storage service while writing the test file, followed by the network error, such as a timeout or a name that could not be found.

How to fix it: check the endpoint address in the bucket's settings, that the server can resolve it in DNS, and that no firewall or proxy blocks outgoing HTTPS from the server. In a container, check that the container itself can reach the internet. Then run the check again.

A bucket returned another error

The bucket returned an error while writing the test file, followed by the storage service's words.

How to fix it: the service's words say what went wrong. If they say the service is busy or unavailable, wait a few minutes and run the check again. Otherwise, press Test connection in the bucket's settings, and look up the error with your provider.

Scheduled checks

Scheduled checks need a Team license or higher. Free mode runs checks by hand only.

• Choose Every week, a day and a time on the Speed check screen. Times use the server's time zone. Pick a quiet hour.
• A scheduled check waits when the server is busy with transfers, and runs later.
• Farwing emails an alert when a location becomes more than 30% slower than its own average, or when the limit drops below your license.
• The alert is also an event, speedcheck.degraded, which event rules can use to post a message or open a ticket.

If the server drops to Free mode, scheduled checks pause and manual checks keep working.

History

Past checks are on their own tab, so you can see whether a disk got slower over time. Page through them, filter by date or delete one. Pick two to compare them side by side.

Seeing the limit live

In the live usage view, each active transfer shows what limits it right now: the disk, the processor, the network, your license cap or a traffic rule. You can see limited by: disk (Projects) while it happens, not only in a test.

The report for support

Copy report and Download report give a plain-text summary of every number on the screen. Paste it into an email to support, and the first reply can be the fix instead of a list of questions. The report opens with the Farwing version, your plan and the time the check started, then the result. Here is its beginning, shortened:

Farwing speed check
License plan: Business

Result
This server can receive at about 3.2 Gbit/s. The limit is the staging disk (/data), which writes at 400 MB/s.
Put staging on a faster disk to reach your 10 Gbit/s license.
Limiting part: Staging disk

It continues with the speed of each part for receiving and sending, each disk (write, read, small files, free space, type and whether it shares staging's disk), the processor, the memory, the network settings and the advice. A part that was not measured says not measured and why. Paths appear only for the locations that were tested.

Related

• Server admin for ports, storage and the traffic limits
• CLI reference for farwing bench
• Event rules
• Plans and licenses

Farwing docs · farwing.io