March 13, 2009
I’ve been very impressed several times in the past few months when I’ve discovered awesome new top-like utilities. I’m probably being slow on the uptake and everyone else but me knows about these, but in case its not just me thats been stuck in the ’70s:
- A much-needed refresh of oldschool top, this still works on your beloved console but gives you visual bar-graphs of CPU, RAM and swap, lets you scroll through the processes and deliver signals/renicing without having to copy the PID off the moving target. Its like the future!
- One of those things I use so often now I have no idea how I even survived without it. Why is this server lagging, who’s hogging the wireless/DSL, which VM is chewing all of the upstream bandwidth? iftop shows you at a glance how much traffic is being used by which host pairs on a given interface, and you can toggle port numbers on and off with simple key-presses. Absolutely indispensable.
- Does this box feel slow to anyone else? Is it swapping, or is it the database server chewing all the IO? Why does my drive keep seeking? It’s amazing… top for IO bandwidth usage!
A passing mention is deserved for apachetop too, which is pretty neat, but when a server is being hammered it’s not something I found too hard to get a feel for just by tailing the log for a while, so it’s not been as life-changing as the others. Maybe that just means my servers don’t see enough traffic.
March 6, 2009
“… all programs that interact with e-mail are broken in one way or another. Please be careful.” – Lars Wirzenius
I seem to have a cunning knack of finding problems with configuring server software, particularly involving e-mail, where a) I can’t find answers in Google, b) most people I go and ask for help say they’d usually ask me such things, and c) if I go onto IRC or mailing lists I end up helping other people and not getting any help with my problem. It’s quite likely this is just because I’m something of a perfectionist, so the ridiculous crappy hacks people come up with and seem content to entrust their mail to are unacceptable to me for one reason or another. Anyway, in my ongoing quest for the perfect mail system, I’ve painted myself into a corner again.
(I’m currently running with postfix, postgrey, clamav-milter, dspam, dovecot using LDA, managesieve and the cmusieve & antispam plugins. If I can get the current incarnation working, I’ve had enough requests to write up a full HOWTO, and seen enough around with pretty questionable content, that I’ll probably do it before too long.)
I’ve got postfix’s local transport configured to hand mail to dspam over LMTP, using mailbox_transport = lmtp:unix:/dspam/dspam.sock. dspam is configured to listen here, add X-DSPAM-Result and signature headers, and then deliver the mail with dovecot’s deliver LDA (which I’ve set to 4750 root:dspam). From dspam.conf:
ServerParameters "--deliver=innocent,spam -d %u"
My dovecot configuration is pretty standard, using PAM for both passdb and userdb, and provides the auth-master socket that deliver needs. The problem I have is that postfix’s local transport is qualifying the local username with the FQDN of the machine before delivering it to dspam with LMTP (the local mail transfer protocol), even for locally-originated mail which was only addressed with a bare username! dspam doesn’t mangle it or care if the user is local or not, and then cheerfully invokes deliver -d firstname.lastname@example.org, which returns EX_NOUSER (addressee unknown) because my username is just robot101. From mail robot101:
Mar 6 02:25:40 omega postfix/pickup: DAA4942F41F: uid=1000 from=<robot101>
Mar 6 02:25:40 omega postfix/cleanup: DAA4942F41F: message-id=<20090306022540.DAA4942F41F@omega.example.co.uk>
Mar 6 02:25:40 omega postfix/qmgr: DAA4942F41F: from=<email@example.com>, size=339, nrcpt=1 (queue active)
Mar 6 02:25:40 omega dovecot: auth(default): passwd(firstname.lastname@example.org): unknown user
Mar 6 02:25:40 omega dspam: Delivery agent returned exit code 67: /usr/lib/dovecot/deliver -d email@example.com
Mar 6 02:25:40 omega postfix/lmtp: DAA4942F41F: to=<firstname.lastname@example.org>, orig_to=<robot101>, relay=omega.example.co.uk[/dspam/dspam.sock], delay=0.08, delays=0.05/0.01/0.01/0.03, dsn=4.3.0, status=deferred (host omega.example.co.uk[/dspam/dspam.sock] said: 421 4.3.0 <email@example.com> Delivered (in reply to end of DATA command))
So, no e-mail for me. Dearest lazyweb, which of the three components is behaving wrongly, and how can I fix it?
(And no, I’m not just going to switch to GMail. I store my data on hard drives, which are sometimes in servers, not “in the cloud”. Until about a month ago, most people I knew spoke about clouds which were made of particles of water in the sky, rather than as a data storage media. What if it rains? 😉
Update: The problem is fixed! Even though arguably the problem is dspam’s for not knowing which users are local or not, it’s fixable in dovecot 1.1.x using the auth_username_format = %n option. Thanks so much to Angel Marin for helping me out.
Update 2: There’s also a patch for dspam floating around which adds a StripRcptDomain option, which makes the LMTP server truncate the e-mail address at the @, so essentially assumes everyone to be a local user. The problem with both of these fixes is that they’re both blunt instruments which will break virtual users on the same host. I think the real fix would be something more like a LocalDomains option in dspam, to choose which domains should be considered local and truncated from the e-mail addresses for delivery purposes.
January 23, 2009
It’s the last day of the most awesome linux.conf.au 2009 conference in Hobart, Tasmania. I’ve just witnessed the a room full of 500 people sit with baited breath as Linus wielded a set of clippers to shave Bdale Garbee‘s beard, followed by Bdale (with a razor with 3 more blades than last time he shaved, a tiny bowl of water and a hand-mirror) trying to make it look neater. The LCA twitter feed was up on the projector, and someone rightly observed this whole event was actually pretty weird. There are already pictures on flickr too. However, well done to Bdale for being such a good sport, but it looks like his wife Karen will accompany him next year to make sure he doesn’t agree to anything else like this, and supervise the waxing of Rusty‘s chest… 🙂
What’s this all in aid of? After the incredible auction for this beautiful picture from Karen, and generous donations at the Penguin Dinner on Wednesday night, the conference has now raised between AU$ 35k and 40k towards the Save the Tasmanian Devil appeal. Around AU$ 1.3k of the nonsensical winning consortium’s AU $10.6k bid came from the Collabora folks who were at the dinner, and AU$ 1.2k from Collabora and Collabora Multimedia directly. We were all set to place a winning AU$ 3k bid but then Matthew and Daniel came up with the Bdale shaving scheme, and then things really picked up. I’m glad we took part – the lead scientist from the project was really grateful, and I hope the money can make a real difference to their great work.
On more mundane matters, I also gave my talk this morning, and my slides (Telepathy slides v2.0 thanks to Marco) are online. I also made a few demos of new awesome stuff you can do with Telepathy (most of the patches are already merged upstream or well on the way):
- Geolocation support (XEP-0080) support in the XMPP backend and Empathy, using GeoClue to find your location and the libchamplain Clutter & Open Streetmap widget to display where your contacts are. Thanks to Pierre-Luc, Alban and Daf for their work here – more details on Pierre-Luc’s blog.
- Support for launching file transfers over link-local XMPP from Nautilus using the Empathy plugin for nautilus-sendto. This is already merged upstream but needs a patch to work with trunk Empathy. Thanks to Marco, Jonny and Guillaume for their work on this.
- Alban also made a neat hack to Rhythmbox which allows exporting your DAAP music server to one of your contants over a Telepathy Stream Tube. Thanks also to recent work from Marco, these tubes now go over XMPP’s SOCKS5 Bytestreams, giving much better throughput than the earlier in-band implementation, network permitting. The next step is unleashing the full might of our libnice NAT traversal library, signalling tubes with Jingle, and therefore making connections work peer to peer in up to 95% of the cases. However, this won’t affect the APIs, stuff will just go faster! Isn’t Telepathy wonderful?
- Olivier stepped up to show off the demo from his talk about Farsight, which shows his branch using the new telepathy-farsight library to allow recording Telepathy video calls directly into the PiTiVi video editor. His network was screwed up so it didn’t work, but I did see it work in his talk yesterday! Awesome stuff, hopefully Edward and friends can pick it up and merge it in before too long.
- Unfortunately we ran out of time for Will to show off Guillaume’s recent work on Telepathy-enabled Abiword on the desktop (rather than just Sugar’s Write activity), but I expect he’ll blog about it soon!
On that note, these were just the five that I picked to try and fit into my talk. There are a load more demos in the pipeline from the other guys in Collabora of doing stuff with Telepathy, so keep a close look on Planet Collabora for the next cool thing.
January 21, 2009
(Apologies in advance for the shameless plug.) It’s possibly a little late to talk about new year’s resolutions, but if anyone is a Facebook user and has given up smoking (or plans to), you might be interested in Quitclock.
My brother Alastair is a health editor for the Bupa private health insurance company, and thought of this neat project which was implemented by my housemate Martin Kleppman, and announced at new year. It helps you keep track of how long since you’ve smoked, how much money you’ve saved, lets you and your friends know how you’re doing, and has health tips about the improvements that you can expect from your new-found abstinence. I’m not a smoker but hopefully someone will find it useful.
January 21, 2009
You know it’s time to call it a day and write your talk tomorrow when…
I just upgraded Gtk+, Cairo and Pango to the versions in Debian experimental while I was upgrading some Telepathy packages, and got this the next time I loaded OO.o. Magic. But seriously, anyone got any ideas what’s going on?
Update: I switched my Debian mirror to .au and downloaded OpenOffice.org 3.0.1~rc2, and installed the Gtk+ and GNOME stuff too, and not only did the fonts came back, but it no longer looks like the 80s. Score! Thanks for the tips. Back to my talk…
July 10, 2008
For everyone at GUADEC, there’s a boat party sponsored by Collabora this evening. Our sponsorship was confirmed after the program was printed so it’s listed anonymously, but it’s mostly sponsored by Collabora, of course with huge thanks to Baris and his team for the organisation, and local sponsors for food and discounted beer. The boat leaves from Kabatas at 9pm today (Thursday). It’s actually pretty close to the 3rd anniversary of Collabora’s incorporation, so I think we could call this our birthday party. 😀
I gave my presentation earlier today about using Telepathy to make collaborative applications. My slides are available, as well as the apps I demoed, which are the python VNC demo based on telepathy-python’s examples, Elliot’s Tic Tac Tube game, and Guillaume and Alban‘s quick hack to Empathy and Vinagre to share your desktop over a stream tube. I was followed up by Senko‘s presentation (slides also available) which had an overview of the libraries and APIs currently available to embed Telepathy functionality in applications on the desktop. The talks were all recorded so hopefully some videos will turn up on-line too.
Speaking about collaborative stuff with Telepathy, we’re really keen to hear from the authors of applications like Abiword, Inkscape, Vinagre, Jokosher, Tomboy, Gobby, etc, to find out how we can help you integrate with Telepathy, what features you need before you can get started, or just try and convince you that it’s a good idea. 🙂 If you’re at GUADEC, come and grab one of the Collaborans or drop by on #telepathy on FreeNode and tell us what you’d like to do and how we can help.
Yesterday after the talks on Soylent and the People project, Travis and I met up with the guys behind the People project (Ali Sabil and Johann Prieur) and some of the Online Desktop team (Owen Taylor and Marina Zhurakhinskaya). We sketched out a way of plugging together Telepathy, People, Soylent and the Online Desktop to deal a lot better with meta-contacts on the desktop, providing “first-class” people objects. It all looks pretty promising and hopefully we’ll all find some time to make moves towards our vision.
This afternoon at 3:30pm, Olivier is also presenting our work on Farsight 2, which is really cool stuff and should include some exciting demos of the multi-person video conferencing stuff we’ve been doing.
More generally, GUADEC is awesome. I’m having a great time in Istanbul (despite it being pretty hot for a pale-faced Brit like me), and enjoying the sights and sounds of such an interesting city and culture. I’ve caught up with many of the usual suspects (like Lennart, who never turns down the chance to turn up at a conference and sample the local bars and clubs), and had some great discussions.
June 9, 2008
I’ve got a load of boxes running Debian etch with Xen 3.0.3 with routed networking (rather than bridged, so I can do iptables and reverse path filtering etc in dom0). Since upgrading from Xen 2.x many moons ago, I’ve not known how to configure one virtual interface to have more than one IP. In the meantime, I’ve ended up doing nonsense like providing a VM with two interfaces just to give it two IPs. However, this interacts really badly with reverse path filtering unless you do a bunch of source-routing rocket science in the domU to send out through the right vif.
So, I looked at the vif-route script and it seems to support iterating through a space-separated list of IPs, but I was totally unable to find any documentation or mailing list posts explaining how to format the IPs within the formerly-Python key/value Xen domain config file syntax. After a while playing with the parser and various levels of quoting, I found that actually, the correct amount of quoting is none at all, and also uncovered a bug in another script which prevents it from working correctly. In the hope that this might help others using Google and trying to achieve the same as me, here is my recipe for configuring Xen vif devices to have multiple IPs (note that I think this might be specific to Xen 3.0.x, as I believe 3.2.x introduces config files in the S-expression format which is what xenstore uses internally):
- Configure your VM using this surprisingly obvious, but somewhat dubious syntax (including a second argument just to prove that yes, it really does work like that):
vif=['ip=18.104.22.168 22.214.171.124, mac=00:16:3e:01:23:45']
When parsed into SXP by xm create, this sets the ip value correctly as a space separated list as the scripts expect:
(device (vif (ip '126.96.36.199 188.8.131.52') (mac 00:16:3e:01:23:45)))
- Fix the bug in /etc/xen/scripts/vif-common.sh:
--- /etc/xen/scripts/vif-common.sh~ 2008-06-09 01:14:23.065065119 +0100
+++ /etc/xen/scripts/vif-common.sh 2008-06-09 01:11:06.599986274 +0100
@@ -103,7 +103,7 @@
if [ "$ip" != "" ]
- for addr in "$ip"
+ for addr in $ip
frob_iptable -s "$addr"
- Set up multi-homed or aliased interfaces as normal in the domU (depending if you’re a ip or an ifconfig kinda guy).
June 5, 2008
In the office, Christian has just unwrapped a very carefully packed box containing a bulging tin of Surströmming. Apparently due to the “unique” smell you have to open it outside, and he’s invited people to come to his house to try it. Now, I’m not keen on various types of fish even at the best of times, so add a year or so of fermentation, and I am very, very scared. In unrelated news, some people in the office have decided to take up vegetarianism.
April 18, 2008
Last night I thought of a few gadgets which I’d like to have, and although I’m pretty sure you should be able to get hold of them, I had trouble finding anything that looked quite right:
- Alarm clock which makes coffee: I can’t be the only one who finds it hard to bootstrap my days because I have to get out of bed and make the first coffee of the day before I’ve had any coffee. My parents had a machine which was an alarm clock which made tea (very noisily) at the appropriate time in the morning. Surely there should be a similar device which can display the time and make (at least passable) coffee instead? Be it an alarm clock with a sideline in making coffee, or a coffee machine with a built in timer. I’ve noticed some of Gaggia’s bean-to-cup machines claim to have 24-hour clocks, but does that mean they have a timer function? We just don’t know.
- Decent watch with USB storage: I found some watches online last night which had USB storage built in, some with a little USB connector that folded out, some with a mini/micro USB connector on the side, with the idea I could store (maybe parts of) my GPG and SSH keys on it, and maybe a bootable Debian installer/rescue system. The thing is, I have a reasonably nice Timex Expedition watch at the moment which I quite like: it has an electro-luminescent analog display for the middle of the night, and a digital bit for the date, alarms and multiple time-zones. The USB-enabled watches I saw didn’t look that great as watches, but I might be wrong. Does anyone have a watch that features USB storage that doesn’t compromise too much on the watch functionality? Maybe I should just give up on this one and go for the rugged USB stick on the keyring approach.
- Video output over USB: I have a reasonably new HP 2510P laptop which I also use as my main machine at work with a docking station, TFT, keyboard, mouse, etc. However, as a machine for watching DVDs or other videos on at home, it’s a bit on the small side. I have an olde-worlde big flatscreen TV at home (which is not as good as Christian‘s flat-screen blueray surround sound movie set-up, but I think I retain the moral high ground on taste in films), but my laptop doesn’t have any video out. Is there a USB 2.0 widget which produces composite or S-Video output which I can feed to my TV, that will work with Linux, or should I just get a scan converter of some sort so I can use the VGA output?
So, answers on a postcard…
May 11, 2007
Daf blogged earlier about some of the work we’ve done thus far for the One Laptop Per Child project. Tubes (although the picture looks more like snakes if you ask me :D) are a really cool technology which should let the OLPC activity authors just work on their activity, and use D-Bus and Telepathy to take care of the communications.
At the moment our implementation for Gabble (Telepathy’s XMPP backend) is pretty rudimentary and sends all data via the server, but this already lets us layer multi-user tubes over XMPP multi-user chat rooms and have it act like a bus where each member of the room is also a D-Bus endpoint. You can export objects, call methods and emit signals just as normal.
Next up we’re going to implement them in Salut (the link-local XMPP backend, which we’ll use for communications over OLPC’s mesh networking) using good old TCP for the one-to-one connections, and some of Sjoerd’s more exciting link-local multicast stuff for multi-user tubes. To make tubes work for desktop clients we’re going to go on and look at more advanced Jingle-based ICE NAT traversal stuff.
Maybe one of our next ports of call should be raw stream tubes for existing TCP protocols, then we can make a reality from X over Jabber (or whatever other protocol) that Matthew Allum was wondering about. 🙂
I’ve also just stolen Planet Collabora from Daf’s home directory and put it on its own subdomain, so you can add it to your feed readers and keep track of what we’re up to with Telepathy, Farsight and friends.
- August 2019
- October 2018
- July 2017
- May 2010
- October 2009
- August 2009
- July 2009
- March 2009
- January 2009
- July 2008
- June 2008
- April 2008
- May 2007
- January 2007
- December 2006
- June 2006
- April 2006
- March 2006
- November 2005
- October 2005
- September 2005
- August 2005
- July 2005
- May 2005
- April 2005
- March 2005