Bad network performance after suspend/resume

I am observing several issues with suspend/resume on my Lenovo Thinkpad L15 G2 (Intel) running Ubuntu 21.04 (currently with kernel 5.11.0-25). One of these issues is that, after the machine wakes up from suspend, the wired network speed was very slow. I used the Vodafone speed check (speedtest.vodafone.de) which indeed showed only a download rate between 2 and 4 MBit/sec. (which really should be > 300 MBit/sec). In this article I describe some of the steps I took to further track this down.

First of all, I made sure that the issue is really reproducible in a simple scenario. And indeed, after reboot the network speed was fine until I did a suspend/resume cycle. After wakeup, the network speed was bad, and the only immediate solution was to reboot, which fixed the network speed until after the next wakeup.

In the next step, I tried bringing the network down and up again. For that, we first need to find out the interface name of the wired network device:

$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 38:f3:ab:48:8e:2b brd ff:ff:ff:ff:ff:ff
3: wlp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000
    link/ether c0:3c:59:29:40:5f brd ff:ff:ff:ff:ff:ff

Here enp0s31f6 is the wired LAN interface. Now we can bounce the network interface (note that this requires root access):

# ip link set dev enp0s31f6 down ; ip link set dev enp0s31f6 up

However, this did not fix the issue. The network speed was still slow.

The next try was to reload the module which contains the device driver for the network device. In order to do this, I first had to check which network controller is actually used in the laptop – a simply way is to search the list of PCI devices:

# lspci | grep -i eth
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (13) I219-V (rev 20)

So, the network controller is an Intel I219-V. A quick google search (“Intel I219-V linux driver”) revealed that the kernel module is called e1000e. Alternatively, the hwinfo command can be used which directly shows the driver which is used for a particular device:

# hwinfo
...
88: None 00.0: 10701 Ethernet
  [Created at net.126]
  Unique ID: 23b5.ndpeucax6V1
  Parent ID: AhzA.zr6+Yth08U0
  SysFS ID: /class/net/enp0s31f6
  SysFS Device Link: /devices/pci0000:00/0000:00:1f.6
  Hardware Class: network interface
  Model: "Ethernet network interface"
  Driver: "e1000e"
  Driver Modules: "e1000e"
  Device File: enp0s31f6
  HW Address: 38:f3:ab:48:8e:2b
  Permanent HW Address: 38:f3:ab:48:8e:2b
  Link detected: yes
  Config Status: cfg=new, avail=yes, need=no, active=unknown
  Attached to: #34 (Ethernet controller)

Now I was able to reload the module:

# lsmod | grep e1000e
e1000e                270336  0
# rmmod e1000e
# lsmod | grep e1000e
# modprobe e1000e

It is always a good idea to double check if a particular command has the expected result – in this case that the module has really been unloaded.

Unfortunately, this also did not fix the speed issue.

Now, I decided to dig further into the network infrastructure. Since the Vodafone speed test is not always reliable (in fact they had another outage exactly at that day when I was investigating this speed issue), I first have set up a local environment where I could reproduce the issue without relying on any Internet connection. I have another computer available in my local network, so I used iperf to do some performance checks. I started an iperf server on the other node:

otherNode$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------

and used the iperf client on the laptop to start the measurement:

thinkpad$ iperf -c otherNode
------------------------------------------------------------
Client connecting to otherNode, TCP port 5001
TCP window size:  620 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.56 port 35608 connected with 192.168.1.57 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.09 GBytes   936 Mbits/sec

This looks really good – and, after suspend/resume, I still got the same values!! This certainly means that the issue is in another layer of the network stack, right? Probably it is related to http only, or probably it is even a browser related issue …

But, no – the explanation was much simpler: by default, iperf -s (the server) is the data sink, and the client (iperf -c) is the side which generates and sends data. Hence, what I was testing with the above commands, was the upload speed and not the download speed (and, the upload speed was still fine even in the aforementioned speed test). So, in order to check the download speed I had to switch the roles of client and server. After starting iperf -s on the laptop and using iperf -c on the other node, I could immediately reproduce the issue. It now became clear that this is most likely related to the network interface / network controller which obviously fails to properly wakeup after the resume.

With that information, i now did another google search and finally found this issue: Massive network problems with I219-V on Thinkpad T14 Gen2 (e1000e). Besides some other issues (which I did NOT see on my machine) there were also mentioned some hibernation related issues. Luckily there was also a workaround posted which indicated that this might be related to the Intel Management Engine Interface, and they suggested to change a particular setting in the MEI:

$ lspci -vvnns 16                      # Check that config 16 is the number of MEI device
00:16.0 Communication controller [0780]: Intel Corporation Tiger Lake-LP Management Engine Interface [8086:a0e0] (rev 20)
	Subsystem: Lenovo Tiger Lake-LP Management Engine Interface [17aa:508f]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 147
	IOMMU group: 8
	Region 0: Memory at 601d189000 (64-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: mei_me
	Kernel modules: mei_me

# echo on > /sys/bus/pci/devices/0000\:00\:16.0/power/control

And, indeed – this setting fixed the issue! So by setting the “power/control” property to “on” in the MEI, the issue was gone – the network speed is now still fine after suspend and wakeup.

Screen capturing in Wayland on Ubuntu 21.04

Ubuntu 21.04 has changed the graphics system from the Xorg server to the Wayland server. At first glance, this seems to work well – I had not even recognized this until I wanted to do a screen recording using SimpleScreenRecorder:

When I tried to create a screenshot of the above dialog using Shutter, the whole desktop was filled with this funny pattern:

So, it seems that especially screenshot and sreen recording applications still have their issues with the Wayland server. Before going back to Xorg as suggested by SimpleScreenSaver, I tried some alternative applications which reportedly already work with Wayland:

Kazam

Kazam is available in the universe repository. Hence installation with apt install kazam is straightforward. It looks less feature rich as SimpleScreenSaver though:

Unfortunately the application crashed several times while creating a screenshot from an area. Other options like “Fullscreen” did not work either, so I was not able to test it further.

Kooha

Kooha is relatively new, and there is no package available in the Ubuntu repository yet. So it needs to be built from source. meson and ninja are required for the build, and also a couple of build dependencies need to be satisfied, in particular gettext, libglib2.0-dev and appstream-util. Then, the application can be built using

$ git clone https://github.com/SeaDve/Kooha.git
$ cd Kooha
$ meson builddir --prefix=/usr/local
$ ninja -C builddir install

Unfortunately, running the application then terminates with an error which indicates that it requires Gtk 4.0 (Ubuntu is still on 3.38):

Traceback (most recent call last):
  File "/usr/local/bin/kooha", line 44, in <module>
    from kooha import main
  File "/usr/local/share/kooha/kooha/main.py", line 22, in <module>
    gi.require_version('Gtk', '4.0')
  File "/usr/lib/python3/dist-packages/gi/__init__.py", line 129, in require_version
    raise ValueError('Namespace %s not available for version %s' %
ValueError: Namespace Gtk not available for version 4.0

I decided not to track this down further.

OBS Studio

OBS Studio is a very feature rich application aimed especially towards screen recording and streaming. It is available in the universe repository, however this version (26.12) only records a black screen when using the Wayland server. Version 27.0 is available as Snap, and this seems to work well. After installing it with snap install obs-studio, I was able to create a screen recording and play it back with VLC.

However, the application crashes with a segmentation fault when starting it a second time. Removing the configuration files in the user’s snap directory solves this, but then requires to go through the complete setup process again when restarting the application.

Conclusion

When doing a lot of screen capturing and/or screen recording, it is probably best to stay on Xorg for the time being – this can simply be configured with a button in the lower right corner on the login screen, after selecting the login user:

This selection is persisted across logout/login.
With the Xorg server, both Shutter and SimpleScreenRecorder are working again. Also, OBS Studio does not segfault when relaunching.

Run time data structure of a Java Virtual Machine

The following diagram summarizes the internal structure of a Java Virtual Machine’s memory structure as specified in chapter “2.5. Run-Time Data Areas” in The Java┬« Virtual Machine Specification, Java SE 8 Edition. In this example, three classes A, B, and C have been loaded into the JVM, and for each of those classes, a varying number of instances have been created. Two threads have also been created.

The Frames on each Thread’s stack are associated with a particular method. Each time a method is invoked, a new Frame is created on the Stack. When the method returns, the frame is discarded from the stack. The Frame which is associated to the current method is the Current Frame. Each Frame has an Operand Stack which is used by the bytecode instructions to perform operations. Local variables are referenced by their index, starting at 0. The Runtime Constant Pool reference in each Frame is a reference to the Runtime Constant Pool of the class which contains the method which is currently executing.

Separating color channels in JavaFX

Suppose you have an Image and want to extract the color channels for red, green and blue into three new images. A straight forward way would be to create a new empty image with the same size as the source image, iterate the source image pixel by pixel, convert the pixel color of each pixel and set the converted color for each pixel in the destination image. In JavaFX, this is possible through the PixelReader and PixelWriter interfaces:

public Image getBlueChannel(Image sourceImage) {
    PixelReader pr = sourceImage.getPixelReader();
    int width = (int) sourceImage.getWidth();
    int height = (int) sourceImage.getHeight();

    WritableImage result = new WritableImage(width, height);
    PixelWriter pw = result.getPixelWriter();
    for (int x = 0;  x < width;  x++) {
        for (int y = 0; y < height;  y++) {
            Color col = pr.getColor(x, y);
            pw.setColor(x, y, new Color(0, 0, col.getBlue(), 1.0));    
        }
    }

    return result;
}

This example creates a new WritableImage with the same size as the source image, and then iterates the source image pixel by pixel and sets the corresponding pixel in the writable image, using the blue component of the source pixel only.

While this works well (and shows how to access individual pixels of an Image, which can be useful in various situations) it is definitely not a good approach, especially due to performance reasons. It might work well enough for a smaller image (like 200×100 pixels) but for larger images it will most likely get too slow.

A much better way is to use the JavaFX Blend effect which takes two input sources and combines them into an output. The two input sources are called topSource and bottomSource, and they can be combined by a couple of operations which define how each individual pixel of the two input sources are combined to create the result pixel. When the effect is applied to a node in a scene graph, the node itself is taken as top source and/or bottom source if either of them has not been set explicitly. Thus, if the top source is explicitly set for the Blend effect and then the effect is applied to a node, the result is the topSource combined with the node itself. The following example creates a ColorInput where all pixels are set to blue (rgb(0, 0, 255)) as the top source, and an ImageView node is used as the bottom source. The two sources are combined by multiplying the color values of each pixel to generate the resulting pixel:

public Image getBlueChannel(Image sourceImage) {
    ColorInput colorInput = new ColorInput();
    colorInput.setPaint(Color.Blue);
    colorInput.setX(0);
    colorInput.setY(0);
    colorInput.setWidth(sourceImage.getWidth());
    colorInput.setHeight(sourceImage.getHeight());

    Blend blend = new Blend();
    blend.setMode(BlendMode.MULTIPLY);
    blend.setTopInput(mask);

    Node img = new ImageView(sourceImage);
    img.setEffect(blend);

    SnapshotParameters params = new SnapshotParameters();
    Image result = img.snapshot(params, null);

    return result;
}

Why does the example use the MULTIPLY operation for each pixel? In the earlier days, a 24 bit RGB pixel was made up of three eight-bit-values, one for each color channel:

   Red         Green        Blue
xxxx xxxx    xxxx xxxx    xxxx xxxx

So, to get one of the color components only (say, blue) it sufficient to create a bit mask where only the bits from the blue channel were set to 1. By AND-combining the color with the mask, the blue channel was then extracted:

       Red         Green        Blue
    xxxx xxxx    xxxx xxxx    xxxx xxxx
AND 0000 0000    0000 0000    1111 1111
----------------------------------------
  = 0000 0000    0000 0000    xxxx xxxx

In JavaFX, the color channel values are floting point values between 0.0 and 1.0. In order to keep the values for one color channel and set the values for all other color channels to 0.0, we can multiply the color with (0.0, 0.0, 1.0):

    Red   Green  Blue
    n.n    n.n    n.n
X   0.0    0.0    1.0
----------------------
=   0.0    0.0    n.n

https://github.com/afester/JavaFXsamples/tree/master/src/afester/javafx/examples/colorchannels contains a complete example in the ColorChannelsExample.java file, which extracts the red, green and blue color channels from an image:

The lower blue image uses the initial example where each individual pixel is processed through a PixelReader and PixelWriter.

Alternative approach

If we only want to display the color channel of the image, and if there is no need to further process the images which contain a specific color channel, we do not need to generate real Image instances for each color. We can simply use different ImageView nodes which all use the same source image, and apply the Blend effect for red, green and blue to the ImageView nodes:

    ...
    // Create Blend effect for red channel
    ColorInput colorInput = new ColorInput();
    colorInput.setPaint(Color.RED);
    colorInput.setX(0);
    colorInput.setY(0);
    colorInput.setWidth(sampleImage.getWidth());
    colorInput.setHeight(sampleImage.getHeight());
    Blend redBlend = new Blend();
    blend.setMode(BlendMode.MULTIPLY);
    blend.setTopInput(mask);

    // Create Blend effect for green channel
    Blend greenBlend = ...

    // Create Blend effect for blue channel
    Blend blueBlend = ...

    ImageView sourceView = new ImageView(sampleImage);

    ImageView redView =   new ImageView(sampleImage);
    redView.setEffect(redBlend);

    ImageView greenView =   new ImageView(sampleImage);
    greenView.setEffect(greenBlend);

    ImageView blueView =   new ImageView(sampleImage);
    blueView.setEffect(blueBlend);
    ...

The result is the same as above, but we did not have to create all the Image instances:

The complete example is available at https://github.com/afester/JavaFXsamples/tree/master/src/afester/javafx/examples/colorchannels in the ColorChannelsExample2.java file.