Hercules: TCP/IP Networking

# Hercules Version 4: TCP/IP networking with Hercules

This page describes how to set up TCP/IP connectivity between a Hercules machine and the outside world.

Since Hercules runs as a user process under the control of a driving system (usually Linux/x86 or Windows), it does not have direct access to the driving system's network adapter. This presents a problem in establishing connectivity between the network and the TCP/IP stack of an operating system running under Hercules.

But thanks to a technique originally demonstrated by Willem Konynenberg, it is possible to establish a virtual point-to-point link between the TCP/IP stack running under Hercules and the TCP/IP stack of the driving system. The driving system is then used as a router to pass IP frames between the Hercules TCP/IP stack and the rest of the network, as shown in the below diagram.

``

            +--------------------------------+
            |     Linux/x86 Driving System   |
            |                                |
            +-------------+                  |
            |  Hercules   |                  +--------+
            |-------------|                  |  eth0  |
            |   OS/390    |      TCP/IP ------------------> Network
            |   TCP/IP    |                  |10.1.2.1|
            |-------------|         |        +--------+
            |    CTCA     |         |        |
            |192.168.200.1|         |        |
            +------|------+   192.168.200.2  |
            |  /dev/tun           tun0       |
            |      |                |        |
            |      +----------------+        |
            |       Virtual CTC link         |
            |                                |
            +--------------------------------+

The virtual CTC link is provided by the Universal TUN/TAP driver developed by Maxim Krasnyansky. This driver creates a tunnel which appears to Hercules as a character device (/dev/tun0 or /dev/net/tun) and appears to the driving system as a virtual network interface (tun0). The Hercules 3088 driver makes the tun device appear as a CTCA (Channel to Channel Adapter) to the S/390 operating system running under Hercules. Each end of the link has its own IP address which is distinct from the IP address of the driving system's real network adapter.

Windows users should refer to Fish's CTCI-WIN web page which provides similar "TunTap" functionality on Windows platforms.

## Installing the TUN/TAP Driver (Linux 2.4)

The TUN/TAP driver is delivered as part of the Linux 2.4 kernel, and if you are using one of the popular Linux distributions you will find that the TUN/TAP driver is already installed. If not, then you must rebuild the kernel with the configuration option CONFIG_TUN=m specified.

Note that the version of TUN/TAP in Linux 2.4 differs from the earlier version in that it allows access to all TUN interfaces (tun0, tun1, etc) through a single character device /dev/net/tun, instead of defining multiple devices /dev/tun0, /dev/tun1, etc.

The procedure for completing the TUN/TAP setup for Linux 2.4 is shown below.

1. Use these commands to create the TUN device: `
   su` (enter the root password when prompted)`
   mkdir /dev/net
   mknod /dev/net/tun c 10 200
   chgrp xxxxx /dev/net/tun` (where xxxxx is the group under which you run Hercules)`
   chmod g+rw /dev/net/tun
   chmod o-rw /dev/net/tun`
2. Edit the file /etc/modules.conf (it is called /etc/conf.modules in some distributions) and add the following line:
   `alias char-major-10-200 tun`
   This causes the TUN/TAP driver to be loaded automatically when the /dev/net/tun device is opened by Hercules.

## Installing the TUN/TAP Driver (Linux 2.6, FreeBSD, OS X)

For distributions based on the Linux 2.6 kernel you will probably find that the TUN/TAP driver is already installed and the /dev/net/tun device is already defined. If not, then follow the procedure for Linux 2.4 as described above.

For FreeBSD, refer to `man tun`. You will likely want to enable `net.link.tun.devfs_cloning` to create `/dev/tun`, but you will be in uncharted waters.

If you are using a current distribution you will need to alter the permissions on the /dev/net/tun device to allow Hercules to open it. Issue `ls -l /dev/net/tun` (Linux) `ls -l /dev/tun0` (OSX) to determine whether Hercules can open the device.

    [root@servus ~]# ls -l /dev/net/tun
    crw-rw---- 1 root staff 10, 200 Nov 24 15:01 /dev/net/tun

    [enrico@enrico-mbp ref.obj]$ls -l /dev/tap0
    crw-rw-rw-  1 root  wheel   21,   0 Nov 22 18:31 /dev/tap0

    [enrico@enrico-mbp ref.obj]$ls -l /dev/tun0
    crw-rw-rw-  1 root  wheel   19,   0 Nov 22 18:31 /dev/tun0

You have two ways to make the `tun` device usable. Use the second only with nonstandard and/or old kernels (that is, if the first udev rule does not work for you.)

1. The preferred way to change properties for the Linux `tun` device is via a udev rule in `/etc/udev/rules.d`

       [root@servus rules.d]# cat 99-jph.rules
       KERNEL=="tun",                  GROUP="staff", MODE="0660"

2. Alternatively, use the following commands to set the necessary permissions: `

   su

   `(enter the root password when prompted)`
   chgrp xxxxx /dev/net/tun` (where xxxxx is the group under which you run Hercules)`
   chmod 0660 /dev/net/tun`

   or make the device file writable by world; this does not consist an integrity exposure, as the world at large cannot do anything with the file descriptor it gets. ``

   chmod 0666 /dev/net/tun

   Additional notes from Greg Smith:

   1. I find on my Fedora Core 6 system that I have to add the above 2 commands to /etc/rc.local and update /etc/udev/rules.d/50-udev.rules replacing
      `KERNEL=="tun", NAME="net/%k"`
      by
      `KERNEL=="tun", NAME="net/%k", GROUP="xxxxx", MODE="0660"`
   2. In the hercules log you should see /dev/net/tun0 opened. I get a couple of error messages about SIOCDIFADDR and SIOCSIFHWADDR ioctl's failing but these can be ignored.

In Linux 2.6 the file /etc/modules.conf no longer exists, instead there is a file called /etc/modprobe.conf. TUN/TAP will usually work, however, without any change to the modprobe configuration.

## Configuring a TUN interface

You may be able to configure the tunnel interfaces you will need at system boot; but if you cannot or will not, Hercules supplies a utility (`hercifc`) to do this.

### At Linux boot

`openvpn` (http://openvpn.net/) is a utility that can open a tunnel device permanently. In addition you are likely to need a udev rule as described above. Openvpn is usually installed as a package (`yum install openvpn` on Fedora); if not refer to http://openvpn.net/index.php/open-source/documentation/howto.html. Note, however, that `openvpn` is just one way to create a permanent tunnel device; alternative methods may be possible.

To create permanent a tunnel interface, add this to rc.local (or as appropriate for the distribution you are using):

        openvpn --mktun --dev tun0 --group hercules
        ifconfig tun0 192.168.2.65 pointopoint 10.0.0.33
        echo 1 >/proc/sys/net/ipv4/ip_forward

This example assumes that Hercules is in the group `hercules`; it shows the tunnel device in a separate network, which requires routing as described below. `ifconfig` can display the permanent interface:

    [root@servus ~]# ifconfig tun0
    tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
              inet addr:192.168.2.65  P-t-P:10.0.0.33  Mask:255.255.255.255
              UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
              RX packets:71282 errors:0 dropped:0 overruns:0 frame:0
              TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:100
              RX bytes:4289708 (4.0 MiB)  TX bytes:2764 (2.6 KiB)

If the interface is in the same subnet as the Hercules host, you need to enable proxy arp. One way to achieve this is to turn it on dynamically:

        echo 1 >/proc/sys/net/ipv4/conf/eth0/proxy_arp

### Free BSD

On FreeBSD, use the command `ifconfig tun create` to create an interface.

### Using hercifc

The tun0 network interface in the driving system must be configured as a point-to-point link. The original design of the TUN/TAP driver did not allow the interface to be statically configured like a regular network interface — the tun0 interface does not exist until a program opens the TUN device. For this reason, Hercules provides a special program called hercifc to configure the tun0 network interface. This program is launched automatically by Hercules 3088 CTC device initialization. (Later, the ability to make an interface permanent was added; this is why `openvpn` and preconfigured interfaces work.)

To allow the hercifc program to issue the necessary configuration commands, you must ensure that hercifc is installed with setuid root file permissions. When Hercules is built with the configuration option --enable-setuid-hercifc=xxxxx, make install will install hercifc in /usr/local/bin with setuid root permissions. The hercifc program will be executable only by group xxxxx. Note: Unrestricted access to the hercifc program could present a potential security exposure, so you will want to ensure that hercifc can be executed only by the group which is authorized to run Hercules.

The following commands have the same effect as the configuration option --enable-setuid-hercifc=xxxxx: ``

su (enter the root password when prompted)`
chgrp xxxxx /usr/local/bin/hercifc` (where xxxxx is the group under which you run Hercules)`
chmod 4750 /usr/local/bin/hercifc
exit`

Note: you can avoid the need to install hercifc as a setuid root program by creating a permanent tunnel interface at Linux boot time, as described above.

## Enabling IP forwarding

You must ensure that your kernel is enabled for IP forwarding. Popular Linux distributions usually have a configuration option to enable IP forwarding or routing:

- For RedHat, specify `net.ipv4.ip_forward=1` in the /etc/sysctl.conf file.
- For SuSE, specify `IP_FORWARD="yes"` in the /etc/rc.config file.
- If you cannot find this option in your distribution, the following command should work on any Linux 2.x and later kernel:
  `echo "1" > /proc/sys/net/ipv4/ip_forward`

## Defining a route to Hercules TCP/IP

Client systems that connect to TCP/IP applications running in the Hercules machine need to have a routing entry to define the driving system as the gateway into the Hercules system. An example route definition for a Unix client system is shown below: ``

    route add 192.168.200.1 gw 10.1.2.1

For a Windows client, go to Settings -> Control Panel -> Network -> Configuration -> TCP/IP -> Properties -> Gateway and add the driving system's IP address to the list of gateways. Alternatively, enter a route command such as: ``

    route add 192.168.200.0 mask 255.255.255.0 10.1.2.1 metric 1

If you want to avoid having to update client systems, another way is to add an appropriate routing entry to your default gateway router.

## Defining the link in Hercules

You must define a CTC device pair in the Hercules configuration file: The first device number must be even. The second device must bear the same definition as the first instance and be at device number + 1 (for CTCI, the first definition beyond the device type is ignored; you only need to specify the definition with the second device). Devices should preferably be grouped (furthermore, it makes the configuration file easier to read).

For a preconfigured tunnel:

`
0E20.2 CTCI tun0
or
0E20,0E21 CTCI tun0
or
0E20-0E21 CTCI tun0`

Using hercifc:

`
0E20.2 CTCI 192.168.200.1 192.168.200.2
or
0E20,0E21 CTCI 192.168.200.1 192.168.200.2
or
0E20-0E21 CTCI 192.168.200.1 192.168.200.2`

Check Device Definition Statement syntax for an explanation of device grouping.

Two IP addresses must be assigned, one for the driving system's end of the link, and one for the Hercules end of the link. For this example I have chosen 192.168.200.1 for the Hercules IP address, and 192.168.200.2 for the driving system's IP address. Since this is a point-to-point link, any addresses may be chosen, provided that the network part of the address (192.168.200 in this example) does not conflict with any existing network addresses used in your IP network.

## Configuring the Hercules TCP/IP stack

#### TCP/IP for VSE

This is an example of the configuration statements which you need to include in the IPINIT00.L member of PRD1.BASE:

    SET IPADDR   = 192.168.200.001
    SET MASK     = 255.255.255.000
    DEFINE LINK,ID=CTCE20,TYPE=CTCA,DEV=(E20,E21),MTU=1500
    DEFINE ROUTE,ID=LINUX,LINKID=CTCE20,IPADDR=0.0.0.0

The CTC devices should be defined to VSE using the following statements in the $IPLxxx.PROC procedure in IJSYSRS.SYSLIB:

    ADD E20:E21,CTCA,EML

#### TCP/IP for OS/390 or VM/ESA

This is an example of the configuration statements which you need to include in the TCPIP.PROFILE.TCPIP dataset (OS/390), or in the PROFILE TCPIP file on TCPMAINT 198 (VM):

    DEVICE CTCDEV1 CTC E20
    LINK CTCLINK1 CTC 0 CTCDEV1
    HOME 192.168.200.1 CTCLINK1
    GATEWAY
    ; Network      First Hop     Link Name Size   Subnet Mask  Subnet Value
    192.168.200.2  =             CTCLINK1  1500   HOST
    DEFAULTNET     192.168.200.2 CTCLINK1  1500   0
    START CTCDEV1

For OS/390, the CTC devices need to be defined as device type 3088 in the IODF. Use the `D U,CTC` command to find out which 3088 addresses are defined in your IODF.

For VM, the CTC devices must be attached to the TCPIP virtual machine.

Because TCP/IP uses long running channel programs, the missing interrupt handler should be disabled for the CTC devices. For OS/390, add this statement in PARMLIB member IECIOS00:

    MIH TIME=00:00,DEV=(E20-E21)

For VM, add this command to the PROFILE EXEC file of OPERATOR 191:

    'CP SET MITIME 0E20-0E21 OFF'

#### Linux for S/390

This is an example of the network definitions which you need in a Linux/390 system running under Hercules:

    ifconfig ctc0 192.168.200.1 pointopoint 192.168.200.2 mtu 1500
    route add defaultroute gw 192.168.200.2

Linux/390 will autodetect the CTC devices E20 and E21 at startup and will assign the interface name ctc0.

## What to do if TUN/TAP doesn't work

Check the following (thanks to Richard Higson for this checklist):

1. Enter the command `ls -l /dev/tun0 /dev/net/tun`.
   For Linux 2.4 and later kernels, the response should be: `
   ls: /dev/tun0: No such file or directory
   crw-rw---- 1 root xxxxx 10, 200 Sep 13 07:06 /dev/net/tun`
   For Linux 2.2, the response should be: `
   crw-rw---- 1 root xxxxx 90, 0 Feb 3 2001 /dev/tun0
   ls: /dev/net/tun: No such file or directory`
   (xxxxx should be the group under which you run Hercules).
2. `ls -l /usr/local/bin/hercifc` should show `
   -rwsr-x--- 1 root xxxxx 17333 Dec 31 20:55 /usr/local/bin/hercifc`
   (xxxxx should be the group under which you run Hercules).
3. When hercules comes up, and before IPLing your favorite OS, verify that you have your underlying network stuff up and ready to roar: `

       [root]# ifconfig
       eth0      Link encap:Ethernet  HWaddr 00:12:34:56:78:9A
                 inet addr:10.1.2.1  Bcast:10.255.255.255  Mask:255.0.0.0
                 UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
       tun0      Link encap:Point-to-Point Protocol
                 inet addr:192.168.200.2  P-t-P:192.168.200.1  Mask:255.255.255.0
                 UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
       [root]# netstat -in
       Kernel Interface table
       Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR   TX-OK TX-ERR TX-DRP TX-OVR Flg
       eth0   1500   0     201      0      0      0     196      0      0      0 BMRU
       tun0   1500   0       0      0      0      0       0      0      0      0 MOPRU
       [root]# netstat -rn
       Kernel IP routing table
       Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
       10.0.0.0        0.0.0.0         255.0.0.0       U        40 0          0 eth0
       192.168.200.0   0.0.0.0         255.255.255.0   U        40 0          0 tun0
       127.0.0.0       0.0.0.0         255.0.0.0       U        40 0          0 lo
       0.0.0.0         10.1.0.1        0.0.0.0         UG       40 0          0 eth0

   `
4. ``cat /proc/sys/net/ipv4/ip_forward`` should show "1". If it doesn't, your L386 won't forward (==route) packets at all.
5. Is the TUN/TAP driver loaded?
   1. TUN/TAP compiled into the kernel (``make menuconfig``) look for `"CONFIG_TUN=m"` in /usr/src/linux
   2. ``lsmod`` after starting hercules should show `tun 3456 2 (autoclean)`
6. Look for `
   Dec 14 16:47:19 wie kernel: Universal TUN/TAP device driver 1.3 (C)1999-2000
   Maxim Krasnyansky`
   in syslog after starting hercules