mcr at sandelman.ca
Tue, 20 Oct 2009
Split-DNS (or Split-horizon DNS) is a technique that has been around for nearly 20 years of firewalls. It used to be an absolutely required mechanism of deployment. When I designed the Milkyway Networks Blackhole back in 1994, specific support was put into place to support and encourage easy configuration of split-DNS.
Here are some more links to what it is:
http://www.isaserver.org/tutorials/You_Need_to_Create_a_Split_DNS.html http://en.wikipedia.org/wiki/Split-horizon_DNS http://wiki.zimbra.com/index.php?title=Split_dns
And while split-DNS worked great in 1995, when all sites had a single upstream firewall, and there was no remote users, or mobile laptops that could move in out, it is no longer such a great thing.
As soon as you have multiple sites (with or without VPN), then you have to make sure that ALL the sites all use the same internal DNS. That means that every site has a recursive internal name server. That's not always easy when you have a lot of sites, or if some of them are small, or have not got much in the way of infrastructure.
Then you need to realize that everyone's laptop may well be a site, if it can move around. And sometimes laptops move to another organizations' Intranet, and then they need two (or more) internal views!!! Which one to pick?
Split-DNS can be made to work if you set up recursive DNS server on every laptop as a stealth secondary for the internal zone, and always use 127.0.0.1 in "/etc/resolv.conf". All of this is easy to do on a *NIX laptop, but how can you do it on a windows laptop? Yeah, you can install the NT version of bind9, and this isn't a bad idea for a lot of other reasons.
What if you need have a VPN/RemoteAccess system on the laptop that won't bring up the VPN until packets flow... so you can't resolve a name until the VPN is up, but you won't bring the DNS up until the name is resolved... Having the names locally also helps for that as well.
With the rise of virtual corporations, where in fact there is no office, just a lot of nomadic laptops with a few central servers, and maybe a cabinet (bay) of servers with VPNs to access things, it can become a major pain to mainitain.
But, it's worth stepping and asking yourself: is it really worth it? What did split-DNS really buy you?
Split-DNS was a way to keep private IP addresses out of the public DNS. To have privacy, and to avoid confusion, because someone else has 10.1.2.3 as their internal mail server too!!!
A subdomain (i.e. 'intra.example.com') mostly works just as well to avoid the confusion. Put "intra.example.com" into people's domain search order, and most of the issues go away.
And the reason for the privacy was because it was belt-and-suspenders on the proper operation of the firewall. Today, firewalls are irrelevant --- they mostly work, and the real concern is compromised Windows PCs... those PCs, if inside, already have access to the internal DNS.... so why make it harder to manage by having split-DNS (or no internal DNS), when that front has already been lost?
And then there is IPv6, coming VERY SOON. (Many of us already use it daily) IPv6 has no RFC1918, so you'll never have confusion. But, where do you put the AAAA records? You have hosts which are "inside", and you may have IPv6 firewalls that prevent most access to these hosts. Part of the whole point of IPv6 is that you can now directly address those hosts, and you can selectively permit access to them via firewall or access control lists. (Remember the client machines are now also directly addressable, and so ACLs work very well now).
Do you put these AAAA records in the internal DNS, or the external one?
Now, add DNSSEC to the equation --- how will you trust the internal zone? It does not have any link to the outside world. Once your external zone is secured, your internal "trusted" zone will look insecure!!! That's another reason to actually make your internal zone (even if you implement it with split-DNS), a sub-zone of your external zone.
posted at: 15:31 | path: /colo | permanent link to this entry
Wed, 17 Jun 2009
Preferring open standards (SIP) to closed systems (Skype), I have been trying the Gizmo Project "Gizmo5" SIP client. It didn't work for along time due to some bugs in the Pulse/libao that were fixed upstream, but debian never considered a bug to release a patch for etch. It started working again in lenny.
I haven't figured out how to properly set my audio device in kphone to be my USB headset. aplay -L says:
marajade-[~] mcr 1102 %aplay -L
front:CARD=I82801DBICH4,DEV=0
Intel 82801DB-ICH4, Intel 82801DB-ICH4
Front speakers
surround40:CARD=I82801DBICH4,DEV=0
Intel 82801DB-ICH4, Intel 82801DB-ICH4
4.0 Surround output to Front and Rear speakers
surround41:CARD=I82801DBICH4,DEV=0
Intel 82801DB-ICH4, Intel 82801DB-ICH4
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=I82801DBICH4,DEV=0
Intel 82801DB-ICH4, Intel 82801DB-ICH4
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=I82801DBICH4,DEV=0
Intel 82801DB-ICH4, Intel 82801DB-ICH4
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
null
Discard all samples (playback) or generate zero samples (capture)
front:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
Front speakers
surround40:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
4.0 Surround output to Front and Rear speakers
surround41:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=default,DEV=0
C-Media USB Headphone Set , USB Audio
IEC958 (S/PDIF) Digital Audio Output
but I'm unclear what to put into the kphone box, and it does not give me a list of available devices like Gizmo does.
Well, gizmo lets me register to my office's Asterisk PBX, and it does give me a nice list, but I just discovered two problems:
* I can not get DTMF ("Touch Tones") to work through the SIP interface. No setting (inband) or outband (RFC2823) seem to work when I tried to call Porter Airlines, or one of the free conference call systems.
* I started to investigate, and was surprised to see that the registration for my extension did not come from my desktop's IP address. Rather, it came from the GIZMO project! Specically I was registered from:
cirrus*CLI> sip show peer 403 .. DTMFmode : rfc2833 Addr->IP : 198.65.166.131 Port 5060 Useragent : LinGizmo/1.7.07 (Gizmo-s2n1) Reg. Contact : sip:17471318555@proxy01.sipphone.com:5060 .. marajade-[~] mcr 1103 %host 198.65.166.131 131.166.65.198.in-addr.arpa domain name pointer northamerica.sipphone.com.
This really surprised me, and may well also explain the DTMF problems. I tried to call Porter Airlines via the GIZMO project, but the call did not complete, so I couldn't tell who was at fault.
This also concerned me because it means that GizmoProject can potentially listen in on my company calls, as they are a registered proxy. They also may get to see (and record?) my SIP passwords which otherwise should not be passing in the clear. (Of course, they could do this anyway as they provided the program I am running, but this is a new tack that I didn't realize)
I shall be building kphone from source, to see if perhaps I can figure out how to properly set the audio I/O device properly.
posted at: 17:14 | path: /colo | permanent link to this entry
Mon, 08 Jun 2009
LeftHand networks - not a useful answer
I generally prefer freely available (in the sense of beer and speech) open source solutions that I can tinker with, but I recognize that sometimes a complete, well packaged and well supported solution is a win. I find it especially nice if I know it is really well packaged open source. As a friend says, "I do not do my own dentistry"
I came across LeftHand Networks <http://lefthandnetworks.com/> last fall, and was excited that they were doing a sales presentation in Ottawa in March. What they offer is a virtualized SAN. You run a special virtual machine on each of you diskful physical machines, give them the bulk of your local disk space and these Virtual Storage Appliances (VSA) communicate together than present an iSCSI target. The contents of the disks are replicated (RAID'ed) between machines, so even if one machine goes down, then disk contents continue to be available.
You then point your virtualization infrastructure at this iSCSI target and spin up more virtual machines as guests. This solves an annoyance about the various "live migration" or "vMotion" facilities: you need to have a SAN to make it possible, and the SAN is now a single point of failure, and can be really expensive.
LeftHand supports ESX(i) at present, but my guess is that you can run qemu-img on the vmdk files and boot the thing under XEN. It's clearly a Linux system inside, and they might even have para-virtualization support in (2.6.26+ kernels have that available), so it might "just work" under XEN, even without HVM (VT/Pacifica).
You can download the appliance and run them, and then you tell two of them to join the same group, and they can replicate disks, and you get 30 days of demo license to do this... I'm at day 15, and it was time to find out what the solution will cost.
I got a quote: $5839CDN. Wow. You can buy physical SANs for that price. You can hire consultants to setup Openfiler for you for that price.
I think LeftHand/HP has missed the boat here. I expected to pay $500 to $900 per system and/or site. (I can see many licensing options here).
posted at: 17:20 | path: /colo | permanent link to this entry
Fri, 20 Jun 2008
When moving a physical machine into a VM, you'll likely have to add the RIGHT SCSI interface, if you are using SCSI in the VM (it works better than IDE).
I did the copy by installing a minimal debian system into my new VM, and then doing a dump/restore copy of my physical system.
(The system in question is a firewall. No loadable modules. Static kernel only)
I had a hell of of a time getting the right SCSI interface, however.
It's not an item under "Device-Drivers" SCSI. It's the Fusion MPT Device Support!
posted at: 19:11 | path: /colo | permanent link to this entry
Tue, 06 May 2008
I installed VMware server 2.0 beta (p-84186) on a machine. It doesn't use the vmware-server-console application to talk to it.
It uses a web interface.. I found the web interface uninteresting. I tried to open an existing VM that I had built on it and it failed.
Do I really want to run a VNC client in my web browser to access the consoles? No. Do I think that that they will drop Linux support and become IE7 only? Yes.
Alas, this machine doesn't have VT extensions (or at least, doesn't have a BIOS that supports that for the XEONs inside), so my original plan to run XEN with HVM will fail. Unfortunately, I need to support some native kernels due to customized patches to the kernels, so I can't use paravirtualization, as much as I'd prefer to.
Summary: I don't like VMware server 2.
posted at: 18:01 | path: /colo | permanent link to this entry
Fri, 18 May 2007
how to activate a volume group
Using the magic of the "lt_hotswap" module, I can remove the cdrom in my T42's ultrabay. It gets removed from the IDE sub-system, and I can then insert my spare hard disk.
Laptop ultrabay hotswap driver version 0.3.1
lt_hotswap: '\_SB.PCI0.LPC.EC.BAT1' found (Hot-Swappable)
lt_hotswap: '\_SB.PCI0.IDE0.SCND.MSTR' found (Hot-Swappable)
lt_hotswap: '\_SB.PCI0.PCI1.DOCK.IDE1.PRIM.MSTR' found (Non-Swappable)
lt_hotswap: '\_SB.PCI0.LPC.FDC.FDD0' found (Hot-Swappable)
lt_hotswap: '\_SB.PCI0.PCI1.DOCK' found (Hot-Swappable)
lt_hotswap: Requesting IDE eject!
lt_hotswap: Attempting to eject
PM: Removing info for ide:1.0
PM: Removing info for No Bus:ide1
lt_hotswap: Attempting to eject
ide1: BM-DMA at 0x1868-0x186f, BIOS settings: hdc:DMA, hdd:pio
Probing IDE interface ide1...
hdc: TOSHIBA MK4018GAP, ATA DISK drive
PM: Adding info for No Bus:ide1
ide1 at 0x170-0x177,0x376 on irq 15
PM: Adding info for ide:1.0
hdc: max request size: 128KiB
hdc: 78140160 sectors (40007 MB), CHS=65535/16/63, UDMA(33)
hdc: cache flushes supported
hdc: hdc2 < hdc5 hdc6 hdc7 hdc8 hdc9 hdc10 >
So, I ran my normal script that mounts partitions by LABEL= and it complained that my /dev/MaraUltraBay/* files weren't there. Oops. Where is the volume group?
After some futzing, I learned that I did to run:
marajade-[~] root 179 #vgchange -a y MaraUltraBay 11 logical volume(s) in volume group "MaraUltraBay" now active marajade-[~] root 181 #lvs LV VG Attr LSize Origin Snap% Move Log Copy% CACHE MaraMainDisk -wi-ao 3.00G DistrosKernel MaraMainDisk -wi-ao 4.00G ProjectOpenswan MaraMainDisk -wi-ao 3.00G ProjectXelerance MaraMainDisk -wi-ao 2.00G ProjectsHifn MaraMainDisk -wi-ao 2.00G UMLROOT MaraMainDisk -wi-ao 2.00G BackupEtc MaraUltraBay -wi-a- 500.00M ProjectCobbix MaraUltraBay -wi-a- 1000.00M ProjectCroquet MaraUltraBay -wi-a- 2.00G ProjectStudioX MaraUltraBay -wi-a- 4.00G ProjectThintropy MaraUltraBay -wi-a- 3.00G ProjectVpnx MaraUltraBay -wi-a- 2.00G ProjectsFreeradius MaraUltraBay -wi-a- 1.00G ProjectsHifn2 MaraUltraBay -wi-a- 6.00G ProjectsMisc MaraUltraBay -wi-a- 2.00G ProjectsOpenswan2 MaraUltraBay -wi-a- 4.00G UMLROOT2 MaraUltraBay -wi-a- 2.00G marajade-[~] root 183 #/root/etc/sswboot start
Now, really, this needs to be driven by the hotplug system. I wish that it wasn't quite as many twisty shell scripts... all alike.
posted at: 01:17 | path: /colo | permanent link to this entry
Mon, 09 Apr 2007
I discovered that the default /dev/MAKEDEV does not create the xbd2 device entries, so a NetBSD domU does with three disks does not get initialized properly by default.
Normally, you run "cd /dev && ./MAKEDEV xbd2" and you are done.
But, now that NetBSD has an mfs based /dev, the results are not saved.
The problem is that if you edit /dev/MAKEDEV with vi, it renames the file, which means that it creates a new file in the mfs. (It is mounted with union option). Really, I think that the mfs /dev should be mounted below the real /dev, and also created elsewhere, so that one can decide of the node should be permanent or not.
I could have mounted the / on another virtual machine (one nice thing about virtual machines), but NFS exporting / to another NetBSD machine and mounting it (which it gets the underlying /dev), and running MAKEDEV was easier.
posted at: 12:34 | path: /colo | permanent link to this entry
Mon, 29 Jan 2007
How to get Asterisk working under a XenU
Asterisk works in basic SIP mode, but you can't use any of the conference facilities, and or any of the announcements without a source of 1khz interrupts.
Normally, this comes from the zaptel driver in the kernel, if you have a stock FXO card. You can also use ztdummy on 2.4 kernels using the USB interface for timing, and on 2.6 kernels you can get 1khz timing from the RTC driver.
But, there is no RTC on Xen.
The solution is to use the ztdummy driver, but to REBUILD your XenU kernel with HZ=1000, so that it can use the stock timer interrupts.
In theory, the in-kernel HPET interface should be used instead, and I hope to adapt the code. I also expect to reduce the number of .h files, and most of the dependancies.
(Note, you need the CCITT CRC routines available in your kernel as well)
I have placed my tgz file of drivers/char/zaptel at:
note that it overwrites drivers/char/Makefile and drivers/char/Kconfig. I will produce patch-o-matic and git trees soon.
I put a kernel that I'm using at:
If you use this, and it works, I'll clean it up a bit more.
The config items you need to get HZ=1000 are:
gimli-[xen/xen/xen-3.0.3_0-src/linux-2.6.16.29-xen] mcr 1062 %grep HZ .config # CONFIG_HZ_100 is not set # CONFIG_HZ_250 is not set CONFIG_HZ_1000=y CONFIG_HZ=1000 CONFIG_MACHZ_WDT=m CONFIG_NO_IDLE_HZ=y
This is under the "Processor Type and Features" item in menuconfig, if you are looking that way. Near the bottom: "Timer frequency (100 HZ)" is the default.
1000HZ settings make the kernel a bit more intensive, and it might be that you have to also adjust the Xen scheduler at bit. I haven't done this yet, but seem to have good results.
With these changes, I can now deploy Asterisk into XenUs at a colo.
posted at: 03:51 | path: /colo | permanent link to this entry
Mon, 15 Jan 2007
some notes on IBM OpenPower LPARs and VIO channels
I thought I'd share something I just learned about VIO disks by a process of experimentation, based upon some comments I found in a wiki, but couldn't confirm anyplace in documentation.
You can run:
mkvdev -vdev lv_cayenne_home -vadapter vhost9 -dev dev_cayenne_h mkvdev -vdev lv_athabasca_sl -vadapter vhost9 -dev dev_athab_sl
I.e. set two virtual disks to be on the same host/server adapter. Linux, sees, properly:
root@cayenne:~ # dmesg | grep 'Attached scsi' Attached scsi disk sda at scsi0, channel 0, id 1, lun 0 Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0 Attached scsi disk sdc at scsi2, channel 0, id 1, lun 0 Attached scsi disk sdd at scsi3, channel 0, id 1, lun 0 Attached scsi disk sde at scsi3, channel 0, id 2, lun 0
This is a nice thing, because you can then add disks to running LPARs, and get the SCSI bus rescanned by doing:
root@cayenne:~ # echo >/sys/bus/scsi/devices/3:0:1:0/rescan
that means you can also do things like perform rescue operations on a sick LPAR's root disk by mounting it on another LPAR, and you can do this without rebooting.
In particular, if you have provisioned your VIO server with enough spare SCSI server adapters, you can easily add LPARs without restarting the VIO server.
The instructions we read suggesting creating a new VIO channel per disk, which seemed dumb. We much prefer to have a seperate LV on the VIO server per partition, because ideally we can resize them as well.
posted at: 21:44 | path: /colo | permanent link to this entry
Sat, 10 Jun 2006
Getting cable service from cybersurf/3web
I have a 3MB/s bridged ethernet DSL service from http://www.travel-net.com/. They are mostly a good company, with some minor connectivity issues. Specifically traffic to/from Rogers Cable can be slow, and they have not reconnected to http://www.ottix.net/ yet.
One of the partners of http://www.xelerance.com/ lives on the edge of town, and is lucky to have Rogers Cable. We have VPN tunnels, etc. and it would be nice if we could get more bandwidth between us. Our Oakville office now has a Cogeco cable as well as DSL, with the tunnels running over that connection, and it seems to work well. Trust the cable cable company to be connected to itself, right?
I abhore Rogers. I hate them. As soon as GSM phones get portable numbers (come ON CRTC. Get a clue on this. There is no competition in mobile phones until there is number portability.), then I leave Fido/Rogers. So, I'm not signing up for Rogers Cable. (We use Starchoice satellite TV).
I contacted cybersurf/3web, the Calgary company that purchased igs.net, cyberus.ca, and the customer list of istop.com. They have a way to offer cable internet service. This is partly because, I'm told, they have an investment from Sprint, who also invested in Rogers. Rogers has been in trouble with the CRTC for not permitting third parties to offer service over their wires. They have given lots of stupid pseudo-technical excuses, which the CRTC, in their ineptitude have accepted. Still, permitting 3web to sell service over their wires makes them appear more compliant.
I had an initial install date of May 9. A guy from Spectrum Cable arrived to install things at around 5:15pm. He soon realized that there was no cable to the house, no demark, and that Rogers had disconnected the cable back in 2001, up at the pole. He didn't feel like putting his ladder up on my side of the fence, and didn't feel like getting on the roof of my neighbour's shed without their permission (they weren't home), so he packed up and went.
I contacted 3web, arranged for another install date, got a letter from my neighbour giving permission to access the pole by climbing on the shed. The new install date was May 16. Nobody showed.
I contacted 3web again. Days passed. I tried phoning --- their phone system was in a major loop. Can't call them.
I noticed that my credit card had been billed already. A bit of a problem, given that I have no service yet.
I am now in email contact with "Shannon", HD agent 1200, and she can't seem to understand that I want to give them my money.
posted at: 14:47 | path: /colo | permanent link to this entry
Tue, 02 May 2006
A big problem for anything that manages many systems is keeping the systems working. A company recently put out something that I think is basically a GPL'ed syslogd for windows. http://www.loglogic.com/logforge/ It looks like hey are thinking about syslog, and mention "TCP syslog".
(A joke... you can't use the same port. syslog is UDP port 514, while TCP port 514 is... rlogin! That was a surprise one day in firewall land, a decade ago)
TCP syslog is not perfect --- the problem is that you want reliability, but you need to not slow the machines down due to network congestion. That calls, really, for SCTP.
For now, I stick to UDP, and use IPsec to keep it private, if available.
I've long pointed the syslogd on my Unix machines to one machine, usually my desktop, and arrange for my desktop to show them on my screen. Okay, when I'm not plugged in at that IP, I don't see suff.
It used to be that you could run xconsole or xterm -C, and you'd get the /dev/console stolen to that pty. This doesn't work as well anymore, but one can now use:
xconsole -file /dev/xconsole -geometry 1000x30+0+1 -font 5x7 -fg green -bg black -name Console
along with the default entries in /etc/syslog.conf:
# The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # daemon.*;mail.*;\ news.crit;news.err;news.notice;\ *.=debug;*.=info;\ *.=notice;*.=warn |/dev/xconsole
Note that xconsole's geometry is in pixels, vs xterm's which is in characters.
Don't forget to edit /etc/init.d/sysklogd to set
SYSLOGD="-r"
You'd expect to see an /etc/default/syslog file on Debian, but I don't.
The challenge now, is actually to get a better xconsole program-- one with a menu of some kind, and a way to interactively set filters, and have it show me logs with a given pattern, from a given host, etc.
posted at: 13:42 | path: /colo | permanent link to this entry
Thu, 27 Apr 2006
For some time I have run an open source friendly co-location facility in downtown Ottawa. We depended upon SDSL lines to connect us to local ISPs that also have facilities in the downtown area.
We have decided for a number of reasons not to continue as a group. There is a business case for continuing, but we are unable to commit the time required to make it work.
There are still some customers, including ourselves, that we need to deal with, and want space. We were about to move equipment into a 19" rack at Cooperix, and so I started investigating renting a cabinet somewhere.
I have decided that I will rent some cabinet space from Storm, in their Courtwood facility. http://ox.ca/eu
It's not far from my house, and we seem to have a good relationship with Storm. They have made it through some growing pains that they had two years ago, and seem far more stable than they were before. I will have 10Mb/s (likely burstable to 100Mb/s) service, and there will be a 30GB/month cap (at which point, we will have to discuss things).
I will have the following things in the cabinet:
router/firewall
primary and secondary DNS server
mailing list server/mail relay
serial console server
VPN/IPsec/OE gateway
Prices will be as follows:
$125/month for 1U server, including serial console.
$150/month for 2U server, including serial console.
$175/month for 3U server, including serial console.
Existing customers will be grandfathered at their current rate and server size for 1 year. For a very short period (until June 30), tower cases from existing customers will be transported to the new location, if you can not arrange to switch them to a rack-mount case in time.
The same IP address space will be used.
Customers who do not want to move will be credited on a pro-rated based upon April 1 termination.
As storm is also an ISP at Cooper Street, we will actually be able to transition easily, adding host routes as each system is moved. I expect the first date on which we can move things to be May 15, and the last date to be June 16.
posted at: 19:24 | path: /colo | permanent link to this entry
