I have off and on been working on getting a device tree files submitted for a Solid Run board that I own. I would like to have an easier time running OpenWrt on the board. Uptreaming for the win.
I had a well short attempt to add the device tree file as it came from Solid Run. So after working out the small naming problems I decided I want to try to use the dtbs_check functionality in the kernel makefile. In case I don’t get this merged upstream I want to document how to get this check running:
make CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 KBUILD_OUTPUT=out/ defconfigmake CHECK_DTBS=y marvell/cn9130-cf-pro.dtbWhen a linux system has many interfaces and you want to identify the interface with the highest packet or byte counters the -s and the -j flags combined with the jq utility will be of assitance.
ip -s -j link | jq 'max_by(.stats64.rx.bytes)' this will show the pretty print of the jq output.
{
"ifindex": 2,
"ifname": "wlp1s0",
"flags": [
"BROADCAST",
"MULTICAST",
"UP",
"LOWER_UP"
],
"mtu": 1500,
"qdisc": "noqueue",
"operstate": "UP",
"linkmode": "DORMANT",
"group": "default",
"txqlen": 1000,
"link_type": "ether",
"address": "ba:ba:ba:ba:ba:ba",
"broadcast": "ff:ff:ff:ff:ff:ff",
"stats64": {
"rx": {
"bytes": 1049995463,
"packets": 819829,
"errors": 0,
"dropped": 10963,
"over_errors": 0,
"multicast": 0
},
"tx": {
"bytes": 42628714,
"packets": 200952,
"errors": 0,
"dropped": 0,
"carrier_errors": 0,
"collisions": 0
}
}
}
Instead of stats64.rx.bytes there is also stats64.tx.packets or a combination of the two.
A nice option to add to the jq output is ip -s -j link | jq 'max_by(.stats64.rx.bytes).ifname' which will slot it in nicely to a script.
Also notable is the sort_by function of jq, ip -s -j link | jq 'sort_by(.stats64.rx.bytes)[-3:] which will give us the NICS with the top 3 highest counters.
A colleague pointed out to me that this is a static view of the counters and there could be an active NIC with incrementing counters that hasn’t surpassed the highest NIC on the system.
]]>I recently made a Debian preseed file so that no questions would have to be answered during the installation. The configuration is simple, dhcp, sshd, all the files in one partition, and that partion uses all of the available disk space. This is useful if a pc comes from a manufacturer without an operating system. This iso would be useful for a pxe boot install as well, though it was a net install iso so the packages are pulled from the mirrors.
The Debian project has a great wiki on the preseed process and I will reproduce the commands I used here. The preseed file I created is based off their sample. The comments in this file do a good job describing what the options do, and offering guidance.
#### Contents of the preconfiguration file (for stretch)
### Localization
# Preseeding only locale sets language, country and locale.
d-i debian-installer/locale string en_US
# The values can also be preseeded individually for greater flexibility.
#d-i debian-installer/language string en
#d-i debian-installer/country string NL
#d-i debian-installer/locale string en_GB.UTF-8
# Optionally specify additional locales to be generated.
#d-i localechooser/supported-locales multiselect en_US.UTF-8, nl_NL.UTF-8
# Keyboard selection.
d-i keyboard-configuration/xkb-keymap select us
# d-i keyboard-configuration/toggle select No toggling
### Network configuration
# Disable network configuration entirely. This is useful for cdrom
# installations on non-networked devices where the network questions,
# warning and long timeouts are a nuisance.
#d-i netcfg/enable boolean false
# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select auto
# To pick a particular interface instead:
#d-i netcfg/choose_interface select eth1
# To set a different link detection timeout (default is 3 seconds).
# Values are interpreted as seconds.
#d-i netcfg/link_wait_timeout string 10
# If you have a slow dhcp server and the installer times out waiting for
# it, this might be useful.
d-i netcfg/dhcp_timeout string 30
#d-i netcfg/dhcpv6_timeout string 60
# If you prefer to configure the network manually, uncomment this line and
# the static network configuration below.
#d-i netcfg/disable_autoconfig boolean true
# If you want the preconfiguration file to work on systems both with and
# without a dhcp server, uncomment these lines and the static network
# configuration below.
#d-i netcfg/dhcp_failed note
#d-i netcfg/dhcp_options select Configure network manually
# Static network configuration.
#
# IPv4 example
#d-i netcfg/get_ipaddress string 192.168.1.42
#d-i netcfg/get_netmask string 255.255.255.0
#d-i netcfg/get_gateway string 192.168.1.1
#d-i netcfg/get_nameservers string 192.168.1.1
#d-i netcfg/confirm_static boolean true
#
# IPv6 example
#d-i netcfg/get_ipaddress string fc00::2
#d-i netcfg/get_netmask string ffff:ffff:ffff:ffff::
#d-i netcfg/get_gateway string fc00::1
#d-i netcfg/get_nameservers string fc00::1
#d-i netcfg/confirm_static boolean true
# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
# If you want to force a hostname, regardless of what either the DHCP
# server returns or what the reverse DNS entry for the IP is, uncomment
# and adjust the following line.
#d-i netcfg/hostname string somehost
# Disable that annoying WEP key dialog.
#d-i netcfg/wireless_wep string
# The wacky dhcp hostname that some ISPs use as a password of sorts.
#d-i netcfg/dhcp_hostname string radish
# If non-free firmware is needed for the network or other hardware, you can
# configure the installer to always try to load it, without prompting. Or
# change to false to disable asking.
d-i hw-detect/load_firmware boolean true
### Network console
# Use the following settings if you wish to make use of the network-console
# component for remote installation over SSH. This only makes sense if you
# intend to perform the remainder of the installation manually.
#d-i anna/choose_modules string network-console
#d-i network-console/authorized_keys_url string http://10.0.0.1/openssh-key
#d-i network-console/password password r00tme
#d-i network-console/password-again password r00tme
### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string http.us.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string
# Suite to install.
#d-i mirror/suite string testing
# Suite to use for loading installer components (optional).
#d-i mirror/udeb/suite string testing
### Account setup
# Skip creation of a root account (normal user account will be able to
# use sudo).
d-i passwd/root-login boolean false
# Alternatively, to skip creation of a normal user account.
#d-i passwd/make-user boolean false
# Root password, either in clear text
#d-i passwd/root-password password r00tme
#d-i passwd/root-password-again password r00tme
# or encrypted using a crypt(3) hash.
#d-i passwd/root-password-crypted password [crypt(3) hash]
# To create a normal user account.
d-i passwd/user-fullname string Debian User
d-i passwd/username string user
# Normal user's password, either in clear text
#d-i passwd/user-password password insecure
#d-i passwd/user-password-again password insecure
# or encrypted using a crypt(3) hash.
d-i passwd/user-password-crypted password $6$Pbb.p.dmhJ4t$/u5DLdgM9jiSqHh.cHj5ZalEzKO9CUfsu50669GNqPnrHwzpO5auEdb2i93JcsqCj/rU6BLG1T5znG8Artm6C.
# Create the first user with the specified UID instead of the default.
#d-i passwd/user-uid string 1010
# The user account will be added to some standard initial groups. To
# override that, use this.
#d-i passwd/user-default-groups string audio cdrom video
### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true
# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string US/Central
# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true
# NTP server to use. The default is almost always fine here.
#d-i clock-setup/ntp-server string ntp.example.com
### Partitioning
## Partitioning example
# If the system has free space you can choose to only partition that space.
# This is only honoured if partman-auto/method (below) is not set.
#d-i partman-auto/init_automatically_partition select biggest_free
# Alternatively, you may specify a disk to partition. If the system has only
# one disk the installer will default to using that, but otherwise the device
# name must be given in traditional, non-devfs format (so e.g. /dev/sda
# and not e.g. /dev/discs/disc0/disc).
# For example, to use the first SCSI/SATA hard disk:
#d-i partman-auto/disk string /dev/sda
# In addition, you'll need to specify the method to use.
# The presently available methods are:
# - regular: use the usual partition types for your architecture
# - lvm: use LVM to partition the disk
# - crypto: use LVM within an encrypted partition
d-i partman-auto/method string regular
# If one of the disks that are going to be automatically partitioned
# contains an old LVM configuration, the user will normally receive a
# warning. This can be preseeded away...
d-i partman-lvm/device_remove_lvm boolean true
# The same applies to pre-existing software RAID array:
d-i partman-md/device_remove_md boolean true
# And the same goes for the confirmation to write the lvm partitions.
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
# You can choose one of the three predefined partitioning recipes:
# - atomic: all files in one partition
# - home: separate /home partition
# - multi: separate /home, /var, and /tmp partitions
d-i partman-auto/choose_recipe select atomic
# Or provide a recipe of your own...
# If you have a way to get a recipe file into the d-i environment, you can
# just point at it.
#d-i partman-auto/expert_recipe_file string /hd-media/recipe
# If not, you can put an entire recipe into the preconfiguration file in one
# (logical) line. This example creates a small /boot partition, suitable
# swap, and uses the rest of the space for the root partition:
#d-i partman-auto/expert_recipe string \
# boot-root :: \
# 40 50 100 ext3 \
# $primary{ } $bootable{ } \
# method{ format } format{ } \
# use_filesystem{ } filesystem{ ext3 } \
# mountpoint{ /boot } \
# . \
# 500 10000 1000000000 ext3 \
# method{ format } format{ } \
# use_filesystem{ } filesystem{ ext3 } \
# mountpoint{ / } \
# . \
# 64 512 300% linux-swap \
# method{ swap } format{ } \
# .
# The full recipe format is documented in the file partman-auto-recipe.txt
# included in the 'debian-installer' package or available from D-I source
# repository. This also documents how to specify settings such as file
# system labels, volume group names and which physical devices to include
# in a volume group.
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
# When disk encryption is enabled, skip wiping the partitions beforehand.
#d-i partman-auto-crypto/erase_disks boolean false
## Partitioning using RAID
# The method should be set to "raid".
#d-i partman-auto/method string raid
# Specify the disks to be partitioned. They will all get the same layout,
# so this will only work if the disks are the same size.
#d-i partman-auto/disk string /dev/sda /dev/sdb
# Next you need to specify the physical partitions that will be used.
#d-i partman-auto/expert_recipe string \
# multiraid :: \
# 1000 5000 4000 raid \
# $primary{ } method{ raid } \
# . \
# 64 512 300% raid \
# method{ raid } \
# . \
# 500 10000 1000000000 raid \
# method{ raid } \
# .
# Last you need to specify how the previously defined partitions will be
# used in the RAID setup. Remember to use the correct partition numbers
# for logical partitions. RAID levels 0, 1, 5, 6 and 10 are supported;
# devices are separated using "#".
# Parameters are:
# <raidtype> <devcount> <sparecount> <fstype> <mountpoint> \
# <devices> <sparedevices>
#d-i partman-auto-raid/recipe string \
# 1 2 0 ext3 / \
# /dev/sda1#/dev/sdb1 \
# . \
# 1 2 0 swap - \
# /dev/sda5#/dev/sdb5 \
# . \
# 0 2 0 ext3 /home \
# /dev/sda6#/dev/sdb6 \
# .
# For additional information see the file partman-auto-raid-recipe.txt
# included in the 'debian-installer' package or available from D-I source
# repository.
# This makes partman automatically partition without confirmation.
d-i partman-md/confirm boolean true
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
## Controlling how partitions are mounted
# The default is to mount by UUID, but you can also choose "traditional" to
# use traditional device names, or "label" to try filesystem labels before
# falling back to UUIDs.
d-i partman/mount_style select uuid
### Base system installation
# Configure APT to not install recommended packages by default. Use of this
# option can result in an incomplete system and should only be used by very
# experienced users.
#d-i base-installer/install-recommends boolean false
# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
#d-i base-installer/kernel/image string linux-image-686
### Apt setup
# You can choose to install non-free and contrib software.
#d-i apt-setup/non-free boolean true
#d-i apt-setup/contrib boolean true
# Uncomment this if you don't want to use a network mirror.
#d-i apt-setup/use_mirror boolean true
# Select which update services to use; define the mirrors to be used.
# Values shown below are the normal defaults.
d-i apt-setup/services-select multiselect security, updates
d-i apt-setup/security_host string security.debian.org
# Don't Scan DVD/netinst image
d-i apt-setup/cdrom/set-first boolean false
d-i apt-setup/cdrom/set-next boolean false
d-i apt-setup/cdrom/set-failed boolean false
# Additional repositories, local[0-9] available
#d-i apt-setup/local0/repository string \
# http://local.server/debian stable main
#d-i apt-setup/local0/comment string local server
# Enable deb-src lines
#d-i apt-setup/local0/source boolean true
# URL to the public key of the local repository; you must provide a key or
# apt will complain about the unauthenticated repository and so the
# sources.list line will be left commented out
#d-i apt-setup/local0/key string http://local.server/key
# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
#d-i debian-installer/allow_unauthenticated boolean true
# Uncomment this to add multiarch configuration for i386
#d-i apt-setup/multiarch string i386
### Package selection
tasksel tasksel/first multiselect standard, ssh-server
# Individual additional packages to install
#d-i pkgsel/include string openssh-server
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
d-i pkgsel/upgrade select safe-upgrade
# Some versions of the installer can report back on what software you have
# installed, and what software you use. The default is not to report back,
# but sending reports helps the project determine what software is most
# popular and include it on CDs.
popularity-contest popularity-contest/participate boolean false
### Boot loader installation
# Grub is the default boot loader (for x86). If you want lilo installed
# instead, uncomment this:
#d-i grub-installer/skip boolean true
# To also skip installing lilo, and install no bootloader, uncomment this
# too:
#d-i lilo-installer/skip boolean true
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
# This one makes grub-installer install to the MBR if it also finds some other
# OS, which is less safe as it might not be able to boot that other OS.
d-i grub-installer/with_other_os boolean true
# Due notably to potential USB sticks, the location of the MBR can not be
# determined safely in general, so this needs to be specified:
#d-i grub-installer/bootdev string /dev/sda
# To install to the first device (assuming it is not a USB stick):
d-i grub-installer/bootdev string default
# Alternatively, if you want to install to a location other than the mbr,
# uncomment and edit these lines:
#d-i grub-installer/only_debian boolean false
#d-i grub-installer/with_other_os boolean false
#d-i grub-installer/bootdev string (hd0,1)
# To install grub to multiple disks:
#d-i grub-installer/bootdev string (hd0,1) (hd1,1) (hd2,1)
# Optional password for grub, either in clear text
#d-i grub-installer/password password r00tme
#d-i grub-installer/password-again password r00tme
# or encrypted using an MD5 hash, see grub-md5-crypt(8).
#d-i grub-installer/password-crypted password [MD5 hash]
# Use the following option to add additional boot parameters for the
# installed system (if supported by the bootloader installer).
# Note: options passed to the installer will be added automatically.
#d-i debian-installer/add-kernel-opts string nousb
### Finishing up the installation
# During installations from serial console, the regular virtual consoles
# (VT1-VT6) are normally disabled in /etc/inittab. Uncomment the next
# line to prevent this.
#d-i finish-install/keep-consoles boolean true
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note
# This will prevent the installer from ejecting the CD during the reboot,
# which is useful in some situations.
#d-i cdrom-detect/eject boolean false
# This is how to make the installer shutdown when finished, but not
# reboot into the installed system.
#d-i debian-installer/exit/halt boolean true
# This will power off the machine instead of just halting it.
#d-i debian-installer/exit/poweroff boolean true
### Preseeding other packages
# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
# debconf-get-selections --installer > file
# debconf-get-selections >> file
#### Advanced options
### Running custom commands during the installation
# d-i preseeding is inherently not secure. Nothing in the installer checks
# for attempts at buffer overflows or other exploits of the values of a
# preconfiguration file like this one. Only use preconfiguration files from
# trusted locations! To drive that home, and because it's generally useful,
# here's a way to run any shell command you'd like inside the installer,
# automatically.
# This first command is run as early as possible, just after
# preseeding is read.
#d-i preseed/early_command string anna-install some-udeb
# This command is run immediately before the partitioner starts. It may be
# useful to apply dynamic partitioner preseeding that depends on the state
# of the disks (which may not be visible when preseed/early_command runs).
#d-i partman/early_command \
# string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"
# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.
#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh
Most of this is straight forward, though there are a couple of gotchas that I had to deal with.
The Debian installer would ask if there was another CD/DVD to scan. Turning this off was not in the Debian docs, but thankfully stackoverflow had the answer.
d-i apt-setup/cdrom/set-first boolean false
d-i apt-setup/cdrom/set-next boolean false
d-i apt-setup/cdrom/set-failed boolean false
Since I didn’t want to push any buttons after I booted from the usb media, I had to figure out how to launch the installer, thankfully the Debian docs had the answer Section B.2.1 says to set timeout to 1 in syslinux.cfg which will automatically select the GUI install. The gui install wouldn’t continue on for me, I think the reason for that was that I needed to supply boot time parameters to select the localization. I didn’t want to fiddle with that. I tried the text based installer and it worked without having to supply any additional boot time arguments. So I set the default target in isolinux.cfg to be install. install is the label that is given to the text based installer in the file txt.cfg. My isolinux.cfg is below, the default is to launch the vesamenu.
# D-I config version 2.0
# search path for the c32 support libraries (libcom32, libutil etc.)
path
include menu.cfg
#default vesamenu.c32
default install
prompt 0
timeout 1
There are a few ways to point the installer towards the preseed.cfg file. The best option for my use case was to re-spin the installer iso with the preseed.cfg inside the initrd. Here are the instructions. I installed the udevil utility and followed the instructions in the wiki page to mount the iso and extract the files to my working directory. My dev/test cycle was:
preseed.cfgchmod +w -R isofiles/install.386/
gunzip isofiles/install.386/initrd.gz
echo preseed.cfg | cpio -H newc -o -A -F isofiles/install.386/initrd
gzip isofiles/install.386/initrd
chmod -w -R isofiles/install.386/
# cd isofiles
# md5sum `find -follow -type f` > md5sum.txt
# cd ..
genisoimage -r -J -b isolinux/isolinux.bin -c isolinux/boot.cat \
-no-emul-boot -boot-load-size 4 -boot-info-table \
-o preseed-debian-9.3.0-i386-netinst.iso isofiles
You can leave the isofiles directory mounted while you are making changes, the genisoimage command will copy the files. Ultimately I only needed to add the preseed.cfg and change /isolinux/isolinux.cfg and I am now able to have an unattended isntall.
This was a good exercies in some system administration tasks. The next step from here would be to make this preseed iso available via PXE so a blank system would automatically install without being booted from USB. I haven’t sorted out PXE with the new uefi based systems. So that is a topic for another time. There is a lot of discussion about using containers and serverless architectures, but there are still needs that are solved by a Linux appliance.
]]>I have been reading a lot of articles about the eXpress Data Path(XDP) and I would like to learn my about it. I decided to write a program that will track the size of packets inbound to my NIC, and increment a counter for the corresponding “bin” size. I have my MTU set to 1500 and I not worried about packets that are smaller than 64 bytes.
eBPF is kind of like an in-kernel virtual machine. It allows a programmer to write code that the kernel will execute in kernel space at various hook-points in the kernel. There is a good FAQ of what eBPF is in the kernel docs
I adapted my code from the xdp1 example given in the kernel source located in the samples/bpf. That directory has C files named similarly, with the difference being the suffix, _kern.c and _user.c. The _kern code is the code that will be translated to the eBPF instructions then compiled to machine native code and executed in the kernel, the _user is code that is executed in user-space.
The code examples below utilize the API for XDP programs. From the FAQ above, we know that not all kernel code is available to XDP programs. The example kernel code does some simple math for the packet size and will atomically increment the appropriate counter. The counters are values in the eBPF maps. The user space program will read those counters from the map via file descriptors. Links to documentation are provided as comments to the code. The userspace code include code from bpf_load.h which assists in in communicating with the eBPF map.
/* Copyright (c) 2016 PLUMgrid
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of version 2 of the GNU General Public
* License as published by the Free Software Foundation.
*/
#define KBUILD_MODNAME "foo"
#include <uapi/linux/bpf.h>
#include <linux/in.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <linux/if_vlan.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
#include "bpf_helpers.h"
struct bpf_map_def SEC("maps") size_hist_map = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(u32),
.value_size = sizeof(u64),
.max_entries = 5,
};
// Still looking for where this call is attached to in the networking stack
SEC("xdp1")
// xdp_md is documented in the kernel
// https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/bpf.h#L2416
int xdp_prog1(struct xdp_md *ctx)
{
long* value;
u32 len = ctx->data_end - ctx->data;
u32 key = 0;
if(len >= 64 && len < 128){
key = 0;
}else if(len >= 128 && len < 256){
key = 1;
}else if(len >= 256 && len < 512){
key = 2;
}else if(len >= 512 && len < 1024){
key = 3;
}else if(len >= 1024 && len <=1500){
key = 4;
}
value = bpf_map_lookup_elem(&size_hist_map, &key);
if(value){
// LLVM maps this to built-in bpf atomic add instructions,
// BPF_STX | BPF_XADD | BPF_W for word sizes
// according to https://cilium.readthedocs.io/en/latest/bpf/#llvm
__sync_fetch_and_add(value,1);
}
return XDP_PASS;
}
char _license[] SEC("license") = "GPL";
/* Copyright (c) 2016 PLUMgrid
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of version 2 of the GNU General Public
* License as published by the Free Software Foundation.
*/
#include <linux/bpf.h>
#include <linux/if_link.h>
#include <assert.h>
#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <libgen.h>
#include <sys/resource.h>
#include "bpf_util.h"
#include "bpf/bpf.h"
#include "bpf/libbpf.h"
static int ifindex;
static __u32 xdp_flags;
static void int_exit(int sig)
{
bpf_set_link_xdp_fd(ifindex, -1, xdp_flags);
exit(0);
}
char* str_arr[5] = {
"64 - 127",
"128 - 255",
"256 - 511",
"512 - 1023",
"1024 - 1500"};
/* simple size counter
*/
static void poll_stats(int map_fd, int interval)
{
const unsigned int nr_keys = 5;
__u64 values[nr_keys];
__u32 key;
while (1) {
sleep(interval);
for (key = 0; key < nr_keys; key++) {
assert(bpf_map_lookup_elem(map_fd, &key, (&values[key])) == 0);
printf("range %s: %10llu\n",str_arr[key],values[key]);
}
}
}
static void usage(const char *prog)
{
fprintf(stderr,
"usage: %s [OPTS] IFINDEX\n\n"
"OPTS:\n"
" -S use skb-mode\n"
" -N enforce native mode\n",
prog);
}
int main(int argc, char **argv)
{
struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};
// This is an XDP program
struct bpf_prog_load_attr prog_load_attr = {
.prog_type = BPF_PROG_TYPE_XDP,
};
const char *optstr = "SN";
int prog_fd, map_fd, opt;
struct bpf_object *obj;
struct bpf_map *map;
char filename[256];
while ((opt = getopt(argc, argv, optstr)) != -1) {
switch (opt) {
case 'S':
xdp_flags |= XDP_FLAGS_SKB_MODE;
break;
case 'N':
xdp_flags |= XDP_FLAGS_DRV_MODE;
break;
default:
usage(basename(argv[0]));
return 1;
}
}
if (optind == argc) {
usage(basename(argv[0]));
return 1;
}
if (setrlimit(RLIMIT_MEMLOCK, &r)) {
perror("setrlimit(RLIMIT_MEMLOCK)");
return 1;
}
ifindex = strtoul(argv[optind], NULL, 0);
/* argv[0] is the name of the executable, good way to find the object file */
snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
prog_load_attr.file = filename;
if (bpf_prog_load_xattr(&prog_load_attr, &obj, &prog_fd))
return 1;
map = bpf_map__next(NULL, obj);
if (!map) {
printf("finding a map in obj file failed\n");
return 1;
}
map_fd = bpf_map__fd(map);
if (!prog_fd) {
printf("load_bpf_file: %s\n", strerror(errno));
return 1;
}
signal(SIGINT, int_exit);
signal(SIGTERM, int_exit);
// this function is defined in bpf_load.c like all the user-space helpers
if (bpf_set_link_xdp_fd(ifindex, prog_fd, xdp_flags) < 0) {
printf("link set xdp fd failed\n");
return 1;
}
poll_stats(map_fd, 2);
return 0;
}
To compile the code, modify the Makefile in samples/bpf to include your _kern.c and your _user.c code. Grepping the Makefile shows the lines I had to add:
hostprogs-y += xdp_count
xdp_count-objs := xdp_count_user.o
always += xdp_count_kern.o
Ensure that you follow the directions in the readme of the samples/bpf directory and run make headers_install at the top level of your kernel source directory before compile the samples directory.
I did have some compile errors. The errors came from my _kernel code, googling did help, but the answers weren’t as easy to find. I ended up seeing the errors as github issues. It is possible that the bpftool could have been of help. My errors came from not using the correct map access function. The other xdp examples and the cillium docs helped.
To run: sudo ./xdp_count -S 3. The -S 3 tells the xdp program to use “skb” mode as opposed to a mode that is hardware accelerated. The ifindex of the NIC can be found using the ip command.
This is the output when I am streaming music, the user-space program polls the map every two seconds:
range 64 - 127: 506
range 128 - 255: 17
range 256 - 511: 17
range 512 - 1023: 0
range 1024 - 1500: 1
range 64 - 127: 511
range 128 - 255: 17
range 256 - 511: 20
range 512 - 1023: 0
range 1024 - 1500: 1
range 64 - 127: 516
range 128 - 255: 17
range 256 - 511: 20
range 512 - 1023: 0
range 1024 - 1500: 1
This is the output when I am watching an XDP related youtube video:
range 64 - 127: 10
range 128 - 255: 6
range 256 - 511: 1
range 512 - 1023: 1
range 1024 - 1500: 79
range 64 - 127: 37
range 128 - 255: 11
range 256 - 511: 1
range 512 - 1023: 4
range 1024 - 1500: 1488
range 64 - 127: 57
range 128 - 255: 17
range 256 - 511: 2
range 512 - 1023: 6
range 1024 - 1500: 2454
It makes sense that the youtube video would have bigger packets than music streaming.
I am still searching for the attachment point of the XDP program. I read that the program executes when the packet “comes off the NIC” but I am trying to find where specifically.
There are a lot of resources out there I will link to several that have helped me:
]]>This article is going to discuss the libradosstriping API. Striping across objects is desireable as it spreads the IO load across different objects. Each object is mapped to a placement group, and each placement group maps to a set of OSDs. The end result is that each OSD is responsible for a smaller portion of the IO load. This is in contrast to writing 4MiB to a single object, which is mapped to a single placement group which is mapped to K + M OSDs. The Ceph docs have a good illustration of this, so I won’t go over it here.
The libradosstriping API can be seen here. For this article I am going to assume that you already have access to a cluster, and it has an erasure coded pool created. If you need steps on created an erasure coded pool, see here.
To run the example code below you need to have the rados libraries installed, specifically libradosstriper1-devel. This article is using a Ceph Jewel (10.2.5) cluster, with 5 OSDs, and a pool with 256 placement groups.
The makefile:
The source file:
The Rados CLI has a --striper option. It can be used to cleanup the files you made with a simple rados -p pool_name --striper rm obj_name. Also, you can use the ls option to see the object you have put into Ceph. If the --striper option is not suppiled you can see all of the Ceph objects that the stripping API created. I have a feeling this will be usefule in the future when I am looking into the alignment details.
I am still wrapping my head around the idea of alignment I know that it is realted to the erasure coding that is done on the pool, but I don’t know the factors that influence it. I will be digging into this in a future blog post. In my examples my alignment is 4128. It seems that is not ideal as my object size is 4MiB, so the values don’t divide evenly.
When I ran the example code and used ceph -w, I saw both read and write traffic reported, when I was only doing a write. I will be digging into the details of why libradosstriper does this.
I was happy to discover the bl.read_file method. That made this example code much cleaner, as I didn’t have to worry about reading the file, Ceph takes care of it for me! I have no idea how stable the buffer api is, but you can find it here
Since I am writing to an Erasure Coded pool and don’t have the EC overwrite turned on, I use the write_full call inside of libradosstriper.
Thanks to James Norman for writing the post at the Storage Made Easy blog. The Alignment tip is excellent.
I would also like to thank CERN for their source file that also uses the striping API, which helped me write this post.
]]>Configuration is as simple as they say:
git clone https://github.com/cjdelisle/cjdns.git
cd cjdns
./cjdroute --genconf > cjdroute.conf
# find a peer, I used a public one
# add a peer to the cjdroute.conf
sudo ./cjdroute cjdroute.conf
To simply join the network you have to add a peer to the configuration file and launch the daemon. If you have made an error in the configuration file, it won’t be immediately obvious from the daemon, but the startup output will throw a critical message. A handy debugging step is to generate a blank config and try to launch the daemon with that. You should see a tun device show up in the “ip addr show”
To become a part of then network (from your house) you would need to forward a port from your router (assuming you have one) to your machine on port given in the cjdroute.conf file. The network is built around anonymity, so all packets are encrpyted.
The first thing I noticed is that there isn’t a DNS server per se. There is a way to not have to type an entire IPv6 address into the url bar, but it can be summarized as register with the guy who keeps track of this stuff.
Cjdns creates a tun device for its network traffic. This is useful as code can be written in user-space and not the kernel. The Tun device is given the ipv6 address.
routing is complicated, I am still working on it, but it is similar to the old ATM style of routing
Every instance of cjdns has an admin interface that provides statistics for your node
Sadly, it doesn’t look like the organizers posted any results from their expirements. They have a github repository and youtube playlist of some of the talks given. UPC donated 20 WRTnodes for the event. There were about 60 confirmed attendees to the event.
As best I can tell they downloaded and ran the wibed on their hardware. The stated goal was to “run different routing protocols…perform some tests and see which routing protocol behaves better”. Several of the participants were from the creators of the mesh protocols, I can only assume that they took the knowledge gained applied it to their codebases.
Looks like more of the same, the venu will give options for indoor and outdoor testing. There are 3 participants registered affiliated with cjdns It is taking place in Maribor, Slovenia.
A deep dive into cjdns. I will be running it on my wrtnode, laptop, and my old router that is openwrt comaptible. Another possibility would be to use containers on a local system.
]]>This post will serve as an overview of all the mesh networking technologies, projects, and efforts that I will be reasearching for my gradschool project:
Bit-Torrent style peer to peer
Block chain
I will add other projects when I come across them.
I will be looking into the results of battlemeshv7. It is a group of people who get together to test out the wireless mesh protocols.
]]>