I won’t lie, I like Microsoft.  I think Windows is the best all-round family of operating systems available.  And I use Internet Explorer as my default browser, even though I have Firefox and Chrome installed.

But I also run a Linux server.  It is a modest beast.  It has a Sempron processor, three hard disks around 200-500GB each, and about 1GB of RAM.  It doesn’t need much, even though it acts as a mail server, a web server, and a DNS and network file server for my home LAN.  It runs Fedora 11, which is actually quite nice.

I started using Linux with no experience, and with the help of some patience, a few good Internet resources, and good old intuition, I pretty much know what I’m doing.

About a week ago, my Internet connection started to die sporadically, at unpredictable intervals, for no apparent reason.  I traced the lack of connectivity down to the DNS server not responding to requests, and this led me to realise that the Linux box would not respond to any kind of request at all: SSH, HTTP, or even ping.

So imagine my surprise, when after a lot of investigation (and I really do mean a LOT of investigation) it turned out to be Samba, the service which handles network file shares.

I had recently updated Samba from v3.3 to v3.4 using yum, assuming it to be a bug-free release which wouldn’t break anything.  How wrong I was.  From v3.4, Samba has been changed to use a different authentication mechanism by default.  It used to use smbpasswd up to v3.3, but now uses tdbsam.

Ok, no problem really – until the upgrade from v3.3 to v3.4 changes your smb.conf without leaving an smb.conf.backup or something similar sitting next to it.  So without knowing it, your Samba installation now uses tdbsam.  Now, the problem with this is that there is a known bug whereby Windows clients become unable to authenticate and connect.

It was disappointing to have to find this out the hard way – by which I mean trawling around the web.  But still, it should be easy enough to edit smb.conf and change the authentication back to smbpasswd, right?  Wrong, because it seems the upgrade deleted the file containing the smbpasswd credentials (/var/lib/samba/private/smbpasswd).

Redhat’s Bugzilla entry explains how to convert smbpasswd account credentials to tdbsam format, so presumably you can do the reverse.  Well, technically you can – but only if the tdbsam credentials file (/var/lib/samba/private/passdb.tdb) isn’t empty!  That’s right, the upgrade failed (or didn’t even try) to convert the smbpasswd account credentials to the tdb format.

The solution is to run smbpasswd -a username for every Samba account you lost.  I’m glad I only had two.

This is why I think Linux is a long long way from being ready to take over the world.  I don’t think the Linux community appreciates how much help the average desktop user needs when they encounter problems.  This is demonstrated by the fact that Linux gives you no help at all; even the noddy ‘How do I…’ help topics in Windows destroy the assistance Linux offers.

From the very start, Linux makes it very clear you’re on your own.

While I admit that the change to Samba was publicised in the Samba release notes, I simply do not have the time to read through the release notes of, and perform impact analysis upon, every update I install.  I want to be confident that each update is the best thing for my system – much like Windows Update.

For now, I will stick with the majority of users, and keep current with Windows updates for the advantage of not having to enter the root user’s password whenever I want to do something.

Yesterday I made the decision (read: mistake) to update my Netgear DG834G router (hardware v4, firmware v5.01.09) to firmware v5.01.14 – and, as is the way with these things, it brought trouble.  After the upgrade I couldn’t reach www.nikrivers.com from the LAN side of the router.

The problem is caused by the way the router handles traffic coming from an internal IP address and destined for the WAN (i.e. external) IP address.  In this situation it requires that the router first transfers the traffic from the internal network to the external network, and then immediately passes it back whilst applying any firewall or routing rules that are relevant to incoming external traffic.

This behaviour is called ‘NAT loopback’, and it seems the vast majority of routers built for the home market have this ability turned off, or do not have the ability at all.  It can be a big problem if you host a website and wish to access that same website using its domain name.  The domain name will resolve to the WAN IP address of your router, and any traffic headed there (such as an HTTP GET request on port 80) from the internal network will be ignored by the router.

There are a few ways to solve this, but none of them are ideal.

1. Use the server name instead of the domain name to access your website
2. Modify the list of known network hosts on each client to point your domain name straight to the server in question.
3. Run your own DNS server, using a view to return the server’s local IP address to requests for your domain name originating from your network.

Of course, the situation gets more complex if you’re also using your router to send TCP traffic on port 80 to your webserver and UDP traffic on port 8668 to a game server.

The solution is to get NAT loopback working on your router.  With some routers, such as the Touchspeed 535 as provided by Be Broadband, this feature can be enabled using a simple CLI command.  For other routers, such as the Netgear DG834G, it’s not quite so easy.

For the purposes of this post I’ll assume the internal network is on the 192.168.0.x range, the router is 192.168.0.254, and the web server is 192.168.0.1.  You will need to modify these IP addresses according to your own network setup.

The first thing to do is to enable debug mode on the router.  Simply go to http://192.168.0.254/setup.cgi?todo=debug and you’ll be rewarded with an appropriate message, “Debug Enable!”.  Nice.

Now connect to the router with ‘telnet 192.168.0.254′ to gain access to the router’s cut-down installation of Linux.  All that is required is to add one additional entry to the router’s iptables (which is a standard Linux feature; Google it or more info).  Type the following, amending any IP addresses according to the network setup:

iptables -t nat -A POSTROUTING -d 192.168.0.1 -s 192.168.0.0/24 -p tcp –dport 80 -j SNAT –to 192.168.0.254

This adds a rule to the POSTROUTING chain on the nat table which applies to all TCP traffic on port 80 (HTTP) coming from the private network and headed to the router.  The rule redirects the traffic to the server, and then processing jumps to the SNAT chain.

If the server is more than just a simple web server, such as an NTP server or mail server as well, the above step needs to be performed (changing the -p and –dport parameters accordingly) for each port and protocol combination you require.  Alternatively, those parameters could be omitted altogether, which will allow all traffic types on all ports through:

iptables -t nat -A POSTROUTING -d 192.168.0.1 -s 192.168.0.0/24 -j SNAT –to 192.168.0.254

If you do this, I recommend you run a firewall on your server, with only the appropriate ports opened.

There is more information in section 10 of Rusty Russell’s Linux 2.4 NAT Howto.

Unfortunately, the iptables change isn’t retained when the router restarts, so it is necessary to go through the process every time – which is a pain in the backside.  Fortunately, however, the Netgear support website has a download link for previous firmware versions, so I downgraded my router back to firmware v5.01.09 and everything worked fine again–including NAT loopback–with no iptables hack required.

A robust solution with DNS

Simply put, proper DNS is the best way to get around a router’s lack of/poorly implemented NAT loopback.

If you have the resources to host a website then you most likely also have the resources to host a DNS server for your internal network.  Simply create an ACL list describing all the clients on your internal network (probably as simple as specifying the CIDR block for your network, maybe something like 192.168.1.0/24).  Then create a view whose clients match that ACL, and define that view as a master DNS server for your website domain.  You then need to create a zone file for that domain – but instead of using an external IP for your webserver, use its internal IP.

All requests for other domains will be routed to the DNS forwarders, but requests for your webserver’s domain will be handled locally, and internal IP addresses will be returned.

The benefit is that you avoid traversing your gateway router to simply come back inside your network.  It doesn’t make sense that you rely on your gateway router to access a website within your own network.  In addition, the firewall on your router can be hardened to a much greater degree: for example, you needn’t leave FTP ports open on your router if you’re only connecting locally.  Or, to put it another way, you’re likely going to want to give yourself more access to your server than you want to give to the outside world; configuring a router’s firewall rules for this kind of conditional logic is simply asking for trouble.