/net in Linux: project log

A test script to fetch over HTTP on command-line

Rahul Murmuria — Mon, 20 Apr 2009 16:21:01 +0000

I decided to use HTTP Get request as a good test of the /net filesystem’s first deliverable (once I complete it). I plan to get connect, sendmsg and recvmsg working in order to have this work in Linux. The script just reads and writes to files, and achieves fetching a page over HTTP on Plan 9′s shell. Even so, simple cat and echo commands sent to the netfs in shell do not work, because when we echo connect ! to the ctl file, and exit to terminal again for next command, the file and the connection get closed. Here’s the script, that works on Plan 9:

#!/bin/rc
server=$1
port=80
clonefile=/net/tcp/clone
netdir=()

fn showme {
while (~ `{cat $netdir/status} Established*) {
line=`{read}
if (! ~ $#line 0)
echo $line
}
exit
}

<[5] $clonefile {
netdir=`{basename -d $clonefile} ^ / ^ `{cat /fd/5}
echo connect $server!$port >$netdir/ctl || exit ‘Cannot Connect’
echo connected to tcp!$server!$port on $netdir
cat $netdir/data | tr -d ‘
‘ | showme &

while (cmd=`{read}) {
echo $cmd > $netdir/data
}
}

To use this script, run it with the server ip as an argument. While using the url will work on Plan 9, I have not implemented DNS for /net on Linux yet, so even after I complete the first deliverable, only IPs will work. After you run the command, enter the request as :
GET / HTTP/1.0
Host:

You will see the requested page fetched and displayed on your shell. It will be best to use rc from plan9ports though.

Thank you Anant for helping in finishing the above script!

Where to hook onto the tcp stack?

Rahul Murmuria — Mon, 20 Apr 2009 13:57:57 +0000

I recently discussed with a diagram, exactly where I plan to hook onto the existing structure in Linux. My plan is to bypass the Socket ioctl and the Socket structure, and plug in directly to the sk structure, which is a common interface to the various transport layer protocols. I plan to write my own connection create and sendpkt functions, which work with the sk structure. Initially I had planned to plug these functions into the .connect , .sendmsg etc. of proto_ops structure, but it seems unnecessary to do that. I now think I’ll call those functions directly in my FS code, when relevant commands are written to the ctl and data files.

I was recommended to look at ksocket by a friend. While the package is looking to provide BSD-style network programming in kernel I cannot hook onto the functions they are hooking on to. ksocket is simply a wrapper on top of the socket structure, and does not help me gain insight on what’s underneath. My plan currently is to work at a much lower layer than they are doing, even though we are both in kernel-space. I just find it useless overhead to maintain a socket structure, for what I am working to achieve. Doing it my way might be much harder though!

Current issue that needs solving: If I dont have sock->sk element, because I dont use a socket to represent a connection, but use a folder /net/tcp/n , then how to I associate the sk structure with it? struct file has no sk element like struct socket does…

Basic framework ready!

Rahul Murmuria — Sun, 05 Apr 2009 20:02:37 +0000

Week 9 :-

With this revision I have figured out a way to use tcp.c, cs.c, ether.c etc. to do the file-specific processing, while passing only the file structure *filp from a common “write_file” function in net.c. The filesystem however now occupies some memory – TMPSIZE * no_of_files to be precise. I would say this is desirable. I dont know with respect to code, but Plan 9 also seems to behave in the same fashion as far as usability is concerned.

Next, what needs to be done is implement the functions of /net/tcp/clone and the /net/tcp/n/ctl files. Doing these will be enough to test out a simple connection from command-line. Predicted forthcoming issue –> Plan 9 uses “connect 192.168.1.8!80″ as the command. ! is a symbol not allowed in bash. I dont know if it will work if I used other shells (or use rc from plan9ports?).

“fsnet_create” a good plan?

Rahul Murmuria — Thu, 26 Mar 2009 00:08:05 +0000

Week 8 :-

I think I have found my footing. Here is a design diagram. I am considering creating a fsnet which is more or less equivalent to inet – and perhaps even borrows / wraps some code - except that it provides functions to the libfs based netfs implementation program.

The inet function in question can be found at net/ipv4/af_inet.c

Glendix on Routers?

Rahul Murmuria — Thu, 19 Mar 2009 16:07:15 +0000

Glen-concept #1

Glendix for routers? I was poking around for what it would take to get there. I found this. I am basically looking to have a way to do routing using Glendix. You can already do that on any standard Linux using Quagga (see http://www.quagga.net/docs/quagga.html#SEC3 ). They use a traditional dynamic library approach with architecture specific code.

I plan to expose the UNIX Kernel Routing Table as a filesystem instead. Then do the routing protocols, and other actions like updating the table from information from other routers, and load-balancing etc from userspace scripts. The advantage is that, we will not need any daemons working directly on top of the kernel. Our simple userspace scripts that do the routing can hack on the, say /net/ether0/route file.

One of the factors that will excite people will obviously be a awesome infrastructure to test new routing protocols in real networks, without writing complex architecture-specific daemons for GNU Zebra. Plus, I wonder what network mounts of such directories might lead us to achieve?!

UPDATE (May 03, 2009): Further discussion of this idea led to me applying to GSoC 2009 with this proposal. Although the idea was not selected, I got some valuable feedback and the project is still on (probably with a more relaxed timeline, and aligned towards Glendix rather than Plan 9 from Bell Labs).

DNS: A hard to crack nut!

Rahul Murmuria — Fri, 06 Mar 2009 04:16:35 +0000

Week 6 :-

Apparently, Linux kernel does not seem to have a DNS resolver! The fundtion gethostbyname() first checks /etc/hosts and if no entries are found, it makes a direct socket connection with the ip address given in resolv.conf and fetches the DNS resolved ip address!

I would do a in-kernel DNS implementation from scratch, but apparently, there are many issues with it. Its not so easy as it originally seemed! Read this response by a certain Josh Graessley in the MAC OS X community who understands Darwin probably. So did the Linux Kernel Hackers actually chose not to have DNS in-kernel for some real-world reasons?

More information we need, but a change of track in the project is apparent. We have decided to move on to the /net/tcp implementation and work with IPs for now.

My first /net code commited!

Rahul Murmuria — Tue, 24 Feb 2009 01:14:02 +0000

Week 5 :-

My first commit into Glendix (and incidentally and surprisingly, the first non-Anant Narayanan commit to the project!).

The design for the inode and file creation is much the same as lwnfs. I added the mechanism for read and write of files created in the /net. One of the issues I ran into was using the i_private field of the inode to point to the buffer created to store the input content in slashnet_write_file(). See comment at the bottom of this function to know more about this issue.

UPDATE:- Some memory leaks. I forgot to use kfree() after kmalloc(). Gamers will call that n00b error?! lol!

Learning to write a Systhetic Filesystem on Linux

Rahul Murmuria — Fri, 20 Feb 2009 01:45:00 +0000

Week 4:-

Before I start with /net, I wanted to try implementing a sample FS. I thank lwn.net for providing that piece of documentation. I implemented the lwnfs.c in my home folder like they mentioned. NOTE:- A patch is needed on newer kernel versions.

My Makefile looks something like:

ifneq ($(KERNELRELEASE),)
obj-m   := lwnfs.o
else
KDIR    := /lib/modules/$(shell uname -r)/build
PWD     := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
endif

Remember, once the filesystem compiles, you need to mount the fs.
i.e. rahul@Glendix$ mount -t lwnfs none /lwnfs

Cant wait for the day when “mount -t net none /net” works!

Discussion on 9Fans

Rahul Murmuria — Sat, 14 Feb 2009 01:12:03 +0000

9Fans is the official Plan 9 mailing list. Most of Glendix’s discussions are done on that channel and it is our contact with the relatively small Plan 9 community.

Here’s link to my Feb 06, 2009 email announcing my /net project among other things:

http://9fans.net/archive/2009/02/168

No real comment so far on the idea itself, although project Glendix gained another member: J.R. Mauro (a.k.a. thedoctor).

NOTE:- Catch us at #glendix channel on Freenode. We at Glendix like IRC more than mailing lists.

Plan 9 Cheat Sheet

Rahul Murmuria — Wed, 11 Feb 2009 00:50:30 +0000

This post is a cheat sheet of Plan 9 shell commands, different from Linux.

Linux Command —-> Plan 9 equivalent

1) rahul@linux$ find . -name ‘cs.c’ -print —-> term% du -a . | grep cs.c
Comments: Notice the Plan 9′s philosophy of using one program for one action?
Also, try running “du -a .” on Plan 9, it is so much faster than even “slocate” on Linux, let alone “find . -name” !!

2) Ctrl+C exit signal for running scripts —-> “Delete” key on the keyboard.
Comments: Another example of how old UNIX is. They probably had no “Delete” key when UNIX was designed, so they used “Ctrl+C” which, logically speaking, makes absolutely no sense!