|
This is a blog of mcr at sandelman.ca |
Tue, 30 Dec 2008I learnt this morning while listening to Jian Ghomeshi interview Gordon Ramsey, that most (all?) of the interviews are on webcam, and posted to Youtube. This is brilliant! Cool content that I already paid for, and why not video tape it... http://www.youtube.com/user/Qtv posted at: 17:32 | path: /netneutrality | permanent link to this entry Converting DVD into small chunks for Youtube A friend dropped a multiple-hour DVD at my house, transfered from an 8mm tape. "Can you upload to Youtube?" I mounted it: mount /cdrom cd /cdrom/video_ts and then I found some space to put the results (an NFS server), and I proceeded to run: ffmpeg -i vts_01_1.vob -r 16 -ss 0 -t 300 /ssw/morevideos/tim/part1.mp4 ffmpeg -i vts_01_1.vob -r 16 -ss 300 -t 300 /ssw/morevideos/tim/part2.mp4 ffmpeg -i vts_01_1.vob -r 16 -ss 600 -t 300 /ssw/morevideos/tim/part3.mp4 ffmpeg -i vts_01_1.vob -r 16 -ss 9000 -t 300 /ssw/morevideos/tim/part4.mp4 I could write a shell script to do this for me, but I typed it out. ffmpeg said to me: Input #0, mpeg, from 'vts_01_1.vob': Duration: 00:14:34.8, start: 0.227022, bitrate: 9741 kb/s Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480, 9300 kb/s, 29.97 fps(r) Stream #0.1[0x80]: Audio: ac3, 48000 Hz, stereo, 256 kb/s Output #0, mp4, to '/ssw/morevideos/tim/part4.mp4': Stream #0.0: Video: mpeg4, yuv420p, 720x480, q=2-31, 200 kb/s, 16.00 fps(c) Stream #0.1: Audio: aac, 48000 Hz, stereo, 64 kb/s and the part4.mp4 was actually empty, because the _1.vob file only had about 15 minutes of video in it. I then went on to the _2.vob file. The results are at: http://www.youtube.com/watch?v=FT1-caLBJ9c posted at: 22:32 | path: /howto | permanent link to this entry I am travelling to Basking Ridge, NJ today. I am going VIA to Montreal/Dorval, Delta to JFK, and then Long-Island RailRoad (LIRR) and NJ-Transit to Basking Ridge. This route vs in/out of EWR, because there are no (inexpensive) fares that would let me stay to a reasonable lateness in NJ, while still getting home on "Thursday". The VIA 12:45pm departure gets me to Dorval for 2:15, in time for my 4pm flight. The train is a VIA rennaissance coach: I will upload a picture of it. The seats are 2 x 1. So much roomier than the LRC. They are elevated above the ground, and there is an angled space under one's seat that accomodates a medium sized wheelie bag easily. The overhead compartment is miniscule. The view out the window is very nice. Complaints: it's not clear that a bag will actually fit under the seat. Some better pictures/diagrams showing this would be good. The trays pull out from the seat behind you. The seats, do recline, but the seat in front of you really can't get in your way too much... to recline, the seat slides forward, rather than the back going down. I like that the trays there, it is WAY better than the fold out from the armrest. That also means that the armrest for the side-by-side seats can move up, very nice if two people are trying to sleep snuggled. But, the tray is NOT strong enough for laptop. It folds out in two sections, and it it does not hold itself straight. So, to get your laptop in the right position, either it is sitting mostly on the second half, bouncing, or it's pushed back, and the keyboard is kinda elevated. If I had a eeePC or an XO, I guess I could have not folded out the second half at all. The trays are not well labelled either: a grandmother next to me was very unclear how to get it open at all. There are footrests (optional fortunately). My legs are too long for them. The floor is curved (to make room for the seat in front of mine's luggage slide), so it comes up to meet your feet, which for my leg length is okay, but it doesn't give me much space to adjust my feet. Bending my knee at 90 degrees lifts my leg off the seat. There is AC on the train, all classes now have it. (That was the major reason I took VIA-1 to Toronto. I have much less reason to do that now that power is everywhere. Wifi is still too expensive for using to Montreal) Suggestions: Move the AC plug to be higher. Give me a little shelf above the tray area for my drink: that way I can move it up when I need to stand up. Make the tray stronger: just put a piece underneath that can be turned to provide extra support. It's nice to be higher. I would dispense with the above-head spaces completely, and if I were rebuilding, I'd make it even higher! The seats are nicely lined up with the windows, which they are not on the LRC. Conclusion: nice ideas, but not built for my size. Not enough instruction on how to use things: pictures would be good. posted at: 22:32 | path: /travel | permanent link to this entry Tue, 11 Nov 2008For some weeks my son Liam has been asking when I leave for the SIMtone office, "Please go too!" He had been to my office once or twice on errands, and we spent a few hours on a holiday monday afternoon with me on a call with some US workers who didn't have that day off. He remembered eating shreddies and playing with his trains. This morning I took Liam with me on the bus to the office, and he was used to do performance testing of our virtual desktop service: his favorite tools are www.thomasandfriends.com, and www.cbc.ca/kidscbc, which are both flash game sites. Shortly after 10 my mother (Rona) and my wife (Meaghan) arrived at the office and we left by bus to go downtown for the 11am ceremony. I don't think I've been there for some years, maybe 1992, or maybe not since I was in elementary school. There was a lot of people, and despite some strategically placed TV monitors, we couldn't see. And then Liam had to pee. And then we couldn't find Meaghan and Rona. And then Liam talked through the 2 minutes of silence. He's only 3, but he seemed to understand that people died. ("that's bad!") And he concluded that war wasn't a good thing. It was good to be there, even if we couldn't see anything. When I got home tonight, I caught up on my blog feeds, and came across Monbiot:
George Monbiot is a bit older than me (8 years), so he has perhaps some minor first hand experiences of the Vietnam war that I (born in 1971) do not. By the time I could understand what was what, the US was leaving Vietnam with it's tail between it's legs, and it was the high-inflation late 1970s, where towns as described by Springsteen's Born in the USA were typical. America the great was a thing of the past. The USA's reputation didn't get better in the 1980s, nor the 1990s, and I do not need to speak of Dubya's legacy. (Remember, I was involved in the crypto-(policy) wars of the 1990s, so I may view the US NSA, CIA, DoJ and executive branch with more cynicism than average) It wasn't until one winter when I received two books at the same time for Xmas: Thomas Reed's At the Abyss (http://www.amazon.ca/At-Abyss-Insiders-History-Cold/dp/0891418377/ref=sr_1_1?ie=UTF8&s=books&qid=1226445513&sr=8-1) and Michael Moore's Dude, Where's my Country (http://www.amazon.ca/Dude-Wheres-Country-Michael-Moore/dp/0446532231/ref=sr_1_2?ie=UTF8&s=books&qid=1226445524&sr=8-2) that I completely understood my disconnect. Like Monbiot with World War I (which I think of being certainly being the War to End All Wars. He was the last time we had a war that was quite clearly just about me vs you. i.e. a gentleman's war. It wasn't about anything. It wasn't idological, it was just about resources and money), I just couldn't understand Vietnam: why did the Americans go there, why couldn't they just leave, and why didn't they understand where the war really was. Thomas Reed's book tells the glass-half-full version: it is amazing that we were smart enough not set off a nuclear war. The military knew what to do in Vietnam, but wasn't allowed to do so by the politicians who were afraid of WWIII. Yet, as far as I can tell, the military were given carte blanch in Iraq and Afghanistan, and the problem can't be solved with soldiers. Reading Thomas Reed and Michael Moore at the same time (I keep books in different rooms) was very much a difficult, but educationally experience. Thomas, the patriot born too early and too late to fight in any war gave me a glimpse of the America he knew: America the Great. Moore, a man of my generation, and a patriot, remembers just enough of that America to ask, "where did it go?" Conclusions... blog entries don't need conclusions, do they? posted at: 23:47 | path: /politics | permanent link to this entry Wed, 05 Nov 2008Google Chromium from X-over office I tried the codeweavers (authors of the X-over office) version of chromium on my debian etch machine. It is at:
and I installed it with dpkg -i. It showed up in my GNOME-panel, and it appears to run {{{ /opt/cxchromium/bin/wine c:/chromium/chrome.exe --no-sandbox --in-process-plugins }}} I'm guessing that they did not recompile it under wine with libraries. I wish that they had. There are instructions for installing chrome with wine, but they don't work on debian etch (you need a newer wine, and some newer libraries). The biggest problem seems to be that many of the drop-down menus do not wait for the cursor to move --- that is, if you are in google maps, and you go to the "By car/Transit/Walking" menu, that it doesn't wait for you to pick the item you want. You have to hold you mouse down to change the combo box. The other problems are that the window can not be resized, and the bits all kinda look a bit squashed. I wish I had time to work on the Linux port myself. posted at: 19:05 | path: /oss | permanent link to this entry Robert Weissman writes in corp-focus that:
http://lists.essential.org/pipermail/corp-focus/2008/000303.html I felt it too. I watched bits and pieces (some of it at Doug Yuill's house with a bunch of progress/ogwifi people), and some later on at home. I felt myself breath easier. Robert Weissman quotes Robert Kuttner
I sure hope that the "infrastructure" focus is on sustainable infrastructure. I know that Robert Weissman gets that, but will the US Senate? Joe Bidden is a regular Amtrak user. What is the official residence of the VP? Does he have one? Will he continue to travel from Delaware by Amtrak? Wouldn't that be cool... "Air Force 2" --- is a train. posted at: 16:29 | path: /politics | permanent link to this entry Thu, 04 Sep 2008Not obvious to me at first is that Foxy Fixtures in RoR, where you can specify a foreign key relationship by name, as in:
### in pirates.yml
reginald:
name: Reginald the Pirate
monkey: george
### in monkeys.yml
george:
name: George the Monkey
pirate: reginald
Depends upon the fact that you didn't include id: in the fixtures. That is, RoR is not looking into the pirates.yml file to find the "reginald" fixture, and then inserting the "id" from it in. Rather, it's applying a hash of the string "reginald" to get the id, and so the id: of reginald had better be derived in the same way. posted at: 15:19 | path: /ruby-on-rails | permanent link to this entry Fri, 08 Aug 2008Last weekend was the 35th Blue Skies Music Festival. We went on Saturday for the day. I've been to this event since I was about 16 (yes, with my mom!). I'm 37 now. It's a great place, and I was very excited for Liam, my 3 year old, to visit. It's his 3rd trip (we didn't go when he was 2 months old), and this time was a bit familliar for him. He'll remember it. It was very rainy. We arrived at noon, just before the torrential rain. Oh well. That's life. It cleared up between 3pm and 10pm, enough to enjoy ourselves. What I do not enjoy is driving to Blue Skies. As a non-car owner, we have either rented cars, or gone with people who had them. This time we took a vrtucar. We stayed in Perth at a B&B on Saturday night, which involves 30 minutes of driving before we can sleep. You can camp there, but it can be hard to get camping tickets. Camping involves bringing more stuff, which, alas, really means more/larger vehicles. This year, they parked more of the weekend people on the road, and the result was a 2km long row of cars parked on the side of the road. Some people parked on the right-hand side as well, which was really bad idea. Walking down the road, on the cloudy night at 10pm, to fetch the car, was a bit scary: dark (except for my "shaker" flashlight, and my freeswan keychain), except when a car came at you, at which point one was blind. But, I kept wondering: why do I have to drive at all. Why do we have to have so many cars HERE? It is time to organize a bus to Blue Skies. Years ago, I wanted to cycle all the way there (strong cyclists can do it in a day from Ottawa, no problem. Go Friday, come back Monday). I wanted someone to take my stuff, however. A bus might be able to take people, stuff, and stuff for cyclists. Best would be to have several bus loads of people. If the buses are available, do not drive them back, but rather, park them there. The last bus could return, however, and take the drivers of the other buses back home. posted at: 00:57 | path: /music | permanent link to this entry Thu, 07 Aug 2008Bill C-61 and tcpdump -- my concerns I am concerned that Bill C-61 will make possession of tcpdump, (i.e. having it installed on your computer), illegal. Here is the thing: despite ample evidence that the TV "scramblers" were easily defeated, satellite TV operators have never actually deployed much security other than security-by-obscurity. Even the "modern" digital systems, where you need to use a phone line to get pay-per-view, which could TRIVIALLY use public key cryptography to provide security, they do not use such systems. Instead, they have relied upon ligitation to prevent "theft" look at: http://en.wikipedia.org/wiki/FTA_Receiver
But, if they are sending the encryption keys inline, then there is no real security. It might as well just be encoded in a complex way The Radiocommunications act says:
OFFENCES AND PUNISHMENT
Prohibitions
9. (1) No person shall
(a) knowingly send, transmit or cause to be sent or transmitted any
false or fraudulent distress signal, message, call or radiogram of
any kind;
(b) without lawful excuse, interfere with or obstruct any
radiocommunication;
(c) decode an encrypted subscription programming signal or encrypted
network feed otherwise than under and in accordance with an
authorization from the lawful distributor of the signal or feed;
(d) operate a radio apparatus so as to receive an encrypted subscription
programming signal or encrypted network feed that has been decoded
in contravention of paragraph (c); or
(e) retransmit to the public an encrypted subscription programming
signal or encrypted network feed that has been decoded in
contravention of paragraph (c).
but the act does not define encryption. As the Supreme Court found that satellite systems were in fact "encrypted", and therefore protected (cf: http://scc.lexum.umontreal.ca/en/2002/2002scc42/2002scc42.html) that tells me, that if I decode (not just "decrypt" as cryptographers would think) a signal, then I may be enfringing copyright. That's okay so far, as it's been the act of infringing that was illegal, so as long as I do not "decrypt" the wrong signals, then I'm okay. But, C-61 makes possession of such tools illegal. Note that tcpdump/wireshark not only decodes dozens of protocols (including some which have never had published specifications), but it also, provided with the keys, will decrypt IPsec ESP (VPN) packets. I even wrote the ESP code --- because I needed it to debug VPN code. It's still very secure, because I have to provide the keys "out-of-band", but there are dozens of protocols which is not secure. For instance, all of the emails that you send, web pages that are communicated between my server and your computer (including this one) have an implicit copyright. If I look at them transitting the internet, I may be violating your copyright! (Am I violating your privacy? Did you have an expectation of privacy? I'm not sure.) references: http://en.wikipedia.org/wiki/Pirate_decryption http://en.wikipedia.org/wiki/FTA_Receiver http://www3.sympatico.ca/dylan.reid/satellitetv.html http://scc.lexum.umontreal.ca/en/2002/2002scc42/2002scc42.html http://laws.justice.gc.ca/en/ShowFullDoc/cs/R-2///en posted at: 18:53 | path: /legal | permanent link to this entry Hey Big-Daddy's: Blackened Catfish should be black On Sunday night, we went out for dinner. We were divided between Indian and Cajun. Liam decided he wanted catfish, so to Big Daddy's Crab Shack we went. We don't visit Fisher/Baseline much, as it's a pain, but we still had our Vrtucar from the Blue Skies trip. Liam mis-behaved, and went to sleep on me instead of eating, but that's a different storey. My fish arrived. It was white. At first, I didn't pay attention, too concerned about keeping Liam awake. I cut it up with my fork, and then was about to eat it when I looked carefully and noticed... it's not black. It doesn't even look like it's been near a pan! I finally called our waiter over, and complained. He took it back. I couldn't see the kitchen, but Meaghan could, and she said the cook came out to look at us. My fish was returned to me, with some red-spice on it. Allspice or something. We should have gotten up and left at that point, but Liam was fast asleep and I was hungry, so I ate it. I question whether this fish was cooked, or just defrosted. But, we certainly will not be returning to Big Daddy's Crab Shack. posted at: 18:41 | path: /food | permanent link to this entry Fri, 01 Aug 2008Strollers on buses (letter to CBC all-in-a-day) From: Michael Richardson <mcr@sandelman.ottawa.on.ca> To: allinday@cbc.ca Subject: strollers on buses X-Mailer: MH-E 7.82; nmh 1.1; XEmacs 21.4 (patch 19) Date: Thu, 31 Jul 2008 16:52:04 -0400 Sender: mcr@marajade.sandelman.ca I am the father of a 3 year-old. I depend upon transit. I expect to have more children. I have definitely seen strollers that are too big, but I have also seen far too many patrons, including older people, who will not move to get out of the way. I think is the because most of the bus operators do not give the older people (or the people with strollers) enough time to sit down. I have both large (running stroller, yes a Chariot Cheetah) and smaller ones, including an umbrella stroller that we bought for $10, and has over 20,000km of travel on it (much of it in a luggage hold) With a three-year old I do not need a stroller anymore, fortunately. I am definitely in favour of limiting the size of strollers on buses. I definitely would like that. It would make my life easier, as it would make more room for more babies! I am also in favour of limiting who can bring their walker on the bus, and where they can sit, and how big the walkers can be. The fold-down triple seats need to be replaced: they should have a a two-seat and one-seat fold down, so that I can fold up two seats and sit next to my smaller stroller. Otherwise, I take up FOUR seats. There is a vertical bar on many buses which keep the motorized wheel chairs from being able to get off the bus easily, causing those patrons to RUN OVER other patrons feet. Many of the people who have to "move" to keep their feet safe are themselves older people. Finally, on articulated buses, there are now seats in the articulation. In rush hour, we do not need 4 useless seats, we need standing room. Outside of rush hour, we have lots of seats. They need to be removed so that there is space for luggage and people to stand, and yes. STROLLERS. posted at: 15:42 | path: /otrain | permanent link to this entry Mon, 23 Jun 2008Many people (including me), get:
This is because you've (correctly) started your mongrel with mongrel_cluster, and it does not create the PIDs with the same name. To fix this, change your mongrel_cluster.yaml file to say:
posted at: 20:53 | path: /ruby-on-rails | permanent link to this entry Fri, 20 Jun 2008When moving a physical machine into a VM, you'll likely have to add the RIGHT SCSI interface, if you are using SCSI in the VM (it works better than IDE). I did the copy by installing a minimal debian system into my new VM, and then doing a dump/restore copy of my physical system. (The system in question is a firewall. No loadable modules. Static kernel only) I had a hell of of a time getting the right SCSI interface, however. It's not an item under "Device-Drivers" SCSI. It's the Fusion MPT Device Support! posted at: 19:11 | path: /colo | permanent link to this entry Thu, 08 May 2008Unit testing with model security As explained in: http://www.embracingchaos.com/2007/05/model_security_.html the problem with model security is that it requires that you have a controller for your unit tests to work. We solved this with a mock object we called "AlwaysAdmin"
class AlwaysAdmin
# this routine is most useful when using script/console!
def self.fake_out_login
Thread.current[:session] = Hash.new
User.current=AlwaysAdmin.new
end
def logged_in_as_admin?
true
end
def logged_in_as_role1?
true
end
def logged_in?
true
end
def admin?
true
end
end
In our unit tests where we don't care at all about model security, we do:
class MyTest < Test::Unit::TestCase
fixtures :myfixtures
def setup
AlwaysAdmin.fake_out_login
end
end
Bruce Perens says:
posted at: 23:11 | path: /ruby-on-rails | permanent link to this entry Today, I converted some MOV (Quicktime) files from the railscasts.com webcast site to mp4, which my Neuros OSD can read. I couldn't do this with ffmpeg from debian stable, because it doesn't have the aac codec. I did it with ffmpeg from http://debian-multimedia.org/ I used:
gimli-[/ssw/cadillac/movies/railscasts] mcr 1075 %ffmpeg -i 004_move_find_into_model.mov 004_move_find_into_model.mp4
FFmpeg version SVN-rUNKNOWN, Copyright (c) 2000-2007 Fabrice Bellard, et al.
configuration: --prefix=/usr --libdir=${prefix}/lib --shlibdir=${prefix}/lib --incdir=${prefix}/include/ffmpeg --enable-shared --enable-libmp3lame --enable-gpl --enable-libfaad --mandir=${prefix}/share/man --enable-libvorbis --enable-pthreads --enable-libfaac --enable-xvid --enable-libdts --enable-amr_nb --enable-amr_wb --enable-pp --enable-libogg --enable-libgsm --enable-x264 --enable-liba52 --enable-libtheora --extra-cflags=-Wall -g -fPIC -DPIC --cc=ccache cc --enable-swscaler
libavutil version: 49.4.0
libavcodec version: 51.40.2
libavformat version: 51.11.0
built on Feb 4 2008 14:45:57, gcc: 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '004_move_find_into_model.mov':
Duration: 00:02:02.8, start: 0.000000, bitrate: 328 kb/s
Stream #0.0(eng): Audio: aac, 44100 Hz, stereo
Stream #0.1(eng): Video: qtrle, rgb24, 800x600, 29.97 fps(r)
File '004_move_find_into_model.mp4' already exists. Overwrite ? [y/N] y
Output #0, mp4, to '004_move_find_into_model.mp4':
Stream #0.0: Video: mpeg4, yuv420p, 800x600, q=2-31, 200 kb/s, 29.97 fps(c)
Stream #0.1: Audio: aac, 44100 Hz, stereo, 64 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Stream #0.0 -> #0.1
posted at: 21:17 | path: /oss | permanent link to this entry Tue, 06 May 2008I installed VMware server 2.0 beta (p-84186) on a machine. It doesn't use the vmware-server-console application to talk to it. It uses a web interface.. I found the web interface uninteresting. I tried to open an existing VM that I had built on it and it failed. Do I really want to run a VNC client in my web browser to access the consoles? No. Do I think that that they will drop Linux support and become IE7 only? Yes. Alas, this machine doesn't have VT extensions (or at least, doesn't have a BIOS that supports that for the XEONs inside), so my original plan to run XEN with HVM will fail. Unfortunately, I need to support some native kernels due to customized patches to the kernels, so I can't use paravirtualization, as much as I'd prefer to. Summary: I don't like VMware server 2. posted at: 18:01 | path: /colo | permanent link to this entry Thu, 01 May 2008testing with authenticated resources So, you add new code to authenticate various access, and your functional tests stop working.. You need to authenticate! Add this to your test_helper.rb:
def user1_login_details
'Basic ' + Base64.encode64('user1'+':'+'user1pw')
end
def user1_login
@request.env['HTTP_AUTHORIZATION'] = user1_login_details
end
(the details are split out because we used them in other places too) then add a call to "user1_login" to your functional tests, or if all of them need it, to the "def setup". Remember to write tests for not being logged in, and for situations where the logged in user should not have access to some protected resource. posted at: 03:49 | path: /ruby-on-rails | permanent link to this entry embrace and extend of php/mysql application We had a prototype for a user-facing service written in PHP. After some discussion, we decided that we really wanted the Test Driven Development that is so easy in RoR, and even thought we could do it in PHP, as our PHP developers had no experience using any kind of object-oriented or template driven PHP development (the code was totally raw PHP), we decided that we would gradually/incrementally replace the code with Ruby-on-Rails. The first task was to create an empty RoR application. We set it all up with Capistrano, SVN, etc. and deployed it to a test application server. We setup Apache and Mongrel as normal, and made sure to enable PHP in Apache. We then copied all of the existing web site into Rails' public directory, and checked it into SVN. We then deployed again, and took a look at the web server, and lo-and-behold, there was our legacy PHP application. On our laptops, we want to develop, so we need Apache to get the PHP code running. As described in: http://www.sandelman.ca/mcr/blog/2008/04/30#starting_local_mysql_database we had some code that started Apache+Postgresql, and this time we run just the Apache code. I decided that test/cluster/etc was needlessly deep, and I installed things into test/etc instead: See http://www.sandelman.ca/mcr/ruby-files/embrace/ for files: apache2.conf.in, php.ini.in (a snippet), portnum.sh, runweb.sh.in, shutit.sh.in, and apache.rake. Place apache.rake into lib/tasks, it has:
namespace :apache do
desc "Start web server and local server"
task :start do
puts "Starting up Apache"
system "test/etc/runweb.sh"
system "sh -c 'script/server -p 9000 &'"
end
desc "Setup web server"
task :setup do
puts "Setting up Apache"
system "test/etc/fixup.sh"
end
end
Start out (after a raw checkout) with: rake apache:setup This will process the .in files into proper config files for apache. It will find the right modules directories for apache, as this sometimes varies between distros. Then run: rake apache:start This will start apache on port 8000 + your numerical userid. This picks a consistent port, but lets' multiple users run simultaneously if they happen to be logging into a common work machine. Point your browser at http://localhost:8xxx/ , and you should see your application running. A copy of mongrel can be started under the normal load balancer, setup to run in development mode. We commented that line out as actually, we prefer to have it in the foreground in a window, in case we want to do debugging. posted at: 03:36 | path: /ruby-on-rails | permanent link to this entry Using ssh+svn URLs for capistrano We use SVN (we'd like to use git, but we haven't convinced TracGitPlugin to work for us yet) with SSH access to deploy our application. Our application servers have SSH access to our master repository, but they must login with public keys only, and we prefer that they do so with the user id of the person doing the deploying. So we have in our config/deploy.rb: set :repository, "svn+ssh://"+ENV['USER']+"@svn.example.com/path/to/proj/repo//trunk" Each of our application servers have a special login accessed by SSH public key authentication, which holds our source code. We don't use sudo, but we probably should. Eventually, with git, we would expect to do a git push to our application server, and then it could do a local checkout. That would permit our application servers to operate with access to the SVN master system. To get the public key authentication from application server to SVN master we needed to have SSH authentication agent forwarding. With Rails 1.2.3, we had to hack the file: /var/lib/gems/1.8/gems/capistrano-2.0.0/lib/capistrano/ssh.rbto add ":forward_agent > true".
With 2.0.2, we can just add to config/deploy.rb:
<pre class"example">
set :ssh_options, { :forward_agent => true }
and every works the way we wanted. posted at: 03:12 | path: /ruby-on-rails | permanent link to this entry strange complains about class being obsolete We have some ActiveRecord (and ActiveResource) models that are subclassed using the "type" field. We would attempt to make multiple selection boxes using collection_select for them, and we would get errors about the #type method being obsolete, or worse, we'd get complaints that we couldn't turn a class into a string. The problem is that Rails 1.2.3 (at least, we haven't seen this with 2.0.2) uses the 'type' field for subclassing, but ruby's Object used to have a .type method, now renamed .class. Since the ActiveRecord would synthesize the attribute accessor methods, the type method wasn't made until runtime, and since it already exists in Object, one wasn't synthesized. To get around this, we added the following to our models (to the parent class):
def type
self[:type]
end
I tried removing this with rails 2.0.2 to generate the precise error, but it appears to be fixed. I may update this entry with the exact error by checking out some older code that uses 1.2.3. posted at: 03:11 | path: /ruby-on-rails | permanent link to this entry We transitioned from using sqlite on devel laptops (with inherent very low hassle) to having to run mysql everywhere to as our migrations and some of our complex database operations just didn't work with sqlite3. Setting up all the database junk in mysql in annoying. Some developers have laptops with other mysql programs running on them too, and some development machines are shared by multiple projects and multiple developers. It's much nicer to just run a "local" mysql. This is relatively easy to do with postgresql, but we are locked into mysql due to corporate desire for someone to buy support from. Building local clusters for mysql is annoying, but not ultimately that hard. We use several shell scripts, but we think that these could be turned into rake tasks easily, and we intend to do that soon. These scripts were inheirited from a PHP/Postgresql project call "ITERation" (a TBS.gc.ca project). There is a Makefile in test/cluster/Makefile
SRCDIR=$(shell cd ../..; pwd)
...
%.sh: %.sh.in Makefile
sed \
-e 's,@SRCDIR@,'${CortlandSRCDIR}',' \
$< >$@
chmod +x $@
dbflush: etc/initdb.sh etc/rundb.sh etc/shutdb.sh
-[ -f run/mysqld/mysqld.pid ] && etc/shutdb.sh
rm -rf ${SRCDIR}/db/mysql
etc/initdb.sh
etc/rundb.sh
dbinit: etc/initdb.sh
etc/initdb.sh
dbstart: etc/rundb.sh
etc/rundb.sh
The first is test/cluster/etc/initdb.sh.in.
#!/bin/sh
SRCDIR=@SRCDIR@
USER=${USER}
RUNDIR=${SRCDIR}/test/cluster/run
LOGDIR=${SRCDIR}/test/cluster/log
PIDFILE=${RUNDIR}/mysqld/mysqld.pid
ROOTPW=therootpw
mkdir -p ${RUNDIR}/mysqld
mkdir -p ${SRCDIR}/db/mysql
mkdir -p ${LOGDIR}/mysql
/usr/bin/mysql_install_db --basedir=/usr --datadir=${SRCDIR}/db/mysql --pid-file=${PIDFILE} --skip-external-locking --socket=${RUNDIR}/mysqld/mysqld.sock --log_bin=${LOGDIR}/mysql/mysql-bin.log
# now start the DB.
# have to start up mysql with TCP networking enabled!
/usr/sbin/mysqld --basedir=/usr --datadir=${SRCDIR}/db/mysql --pid-file=${RUNDIR}/mysqld/mysqld.pid --skip-external-locking --socket=${RUNDIR}/mysqld/mysqld.sock --port=3307 --log_bin=${LOGDIR}/mysql/mysql-bin.log &
sleep 10
/usr/bin/mysqladmin -h 127.0.0.1 --port=3307 -u root password $ROOTPW
echo "update user set host='%' where host='localhost';" | mysql -h 127.0.0.1 -u root --password=$ROOTPW mysql
(
echo "CREATE DATABASE application_test;"
echo "CREATE DATABASE application_development;"
echo "CREATE USER application;"
echo "SET PASSWORD FOR application = PASSWORD('nonprivpw');"
echo "GRANT SELECT,INDEX,INSERT,UPDATE,DELETE,ALTER,CREATE,DROP ON application_test.* TO 'application'@'%';"
echo "GRANT SELECT,INDEX,INSERT,UPDATE,DELETE ON application_development.* TO 'application'@'%';"
echo "CREATE USER webuser;"
echo "SET PASSWORD FOR webuser = PASSWORD('phppw');"
echo "FLUSH PRIVILEGES;"
) | mysql --protocol=socket --socket=${RUNDIR}/mysqld/mysqld.sock -u root --password=$ROOTPW mysql
/usr/bin/mysqladmin --protocol=socket --socket=${RUNDIR}/mysqld/mysqld.sock -u root --password=$ROOTPW shutdown
And the script which starts things up for normal things:
#!/bin/sh
SRCDIR=@SRCDIR@
USER=${USER}
RUNDIR=${SRCDIR}/test/cluster/run
LOGDIR=${SRCDIR}/test/cluster/log
SOCKET=${RUNDIR}/mysqld/mysqld.sock
mkdir -p ${RUNDIR}/mysqld
mkdir -p ${SRCDIR}/db/mysql
mkdir -p ${LOGDIR}/mysql
ln -s ${SOCKET} ${SRCDIR}/../application.sock
/usr/sbin/mysqld --basedir=/usr --datadir=${SRCDIR}/db/mysql --pid-file=${RUNDIR}/mysqld/mysqld.pid --skip-external-locking --socket=${SOCKET} --skip-networking --log_bin=${LOGDIR}/mysql/mysql-bin.log &
The symlink is placed in the dir above because in our case, we have multiple RoR applications that want to read from that database. Slowly we are converting them to RESTful/ActiveResource mechanism. To access the symlink, the database.yml looks like: development: adapter: mysql database: application_development username: application password: nonprivpw host: localhost socket: <%= RAILS_ROOT %>/test/cluster/run/mysqld/mysqld.sock posted at: 03:11 | path: /ruby-on-rails | permanent link to this entry Running migrations with a different database ID As we developed our virtual desktop provisioning application, we had to integrate against some PHP code that wanted direct access to the database. (See next tip about that) We started to maintain the properties of that access including low-priviledge database login, and associated limited access views for it. To do this, we initially gave our normal RoR database login the right permissions. We got less and less enamored of this:
We couldn't do a lot about the second point because RoR doesn't abstract some of things we wanted to do, and sqlite doesn't even support them. Initially, we just avoiding running those migrations that edited views, or GRANTed permissions by wrapping the code itself in:
if ENV['RAILS_ENV'] == 'production'
execute "alter view webview_table as select uuid,firstname,lastname,username from logins;"
execute "GRANT SELECT (`id`, `filesrv_ip`, `title`, `code`, `baynum`) ON `locations` TO 'webuser'@'localhost';"
end
What we decided to do was to make a new database stanza in database.yml: development_migration: adapter: mysql database: application_development username: root password: mypasswd host: localhost socket: <%= RAILS_ROOT %>/test/cluster/run/mysqld/mysqld.sock We run this with: RAILS_ENV=development_migration rake db:migrate We are looking for a way to make that the default for db:migrate, but our rake-fu isn't high enough yet. To make this work during capistrano deployments, we add to config/deploy.rb: set :rails_env, "production_migration" posted at: 03:10 | path: /ruby-on-rails | permanent link to this entry default schemas for active resource A problem with the RESTful ActiveResource class is that it doesn't get any kind of schema from the server. For GET/Retrieve of CRUD, it's not a problem. For POST/CREATE it is. We put code like this in our ActiveResource method's initialize:
class MyRestfulResourcet < ActiveResource::Base
self.site = "http://mysite/"
self.collection_name = "myrecords"
self.element_name = "myrecord"
Schema = [ :name, :type, :firstboot, :subscriber_id ]
def self.find_single(x, options)
return nil if x.nil?
return nil if x.to_i < 1
super(x, options)
end
def initialize(attributes = {})
super(attributes)
if @attributes["type"].nil?
@attributes["type"] = "TheSubclass"
end
Schema.each { |attr|
attr = attr.to_s
if @attributes[attr].nil?
@attributes[attr] = nil
end
}
end
end
This initializes the @attributes hash to contain the things that we need, and so appropriate attribute accessor methods will be created. Maybe there is a better way. posted at: 03:06 | path: /ruby-on-rails | permanent link to this entry Tue, 01 Apr 2008Submission from David P. James Summary: Overall, the options that have been presented are very disappointing and are a sort of "two steps forward, one step back" improvement on the current TMP. They all represent a very limited vision for the transit network in Ottawa in the next 20+ years, despite spending a minor fortune. The 'Options': Options #1 and #2 are not even viable since a tunnel operating with the volumes of buses anticipated is not even feasible. 14-16 second headways are unrealistic when station dwell times will exceed that time by a considerable margin, especially if the effective number of stations is reduced (Bay, Kent, Bank, Metcalfe, McKenzie King, Laurier & Campus (7) reduced to Lyon/Kent, Bank, Metcalfe, Union/Rideau & Campus (5)), thus putting more pressure on the remaining stations. Options #3 and #4 might represent a starting point towards an improved network, but as a vision of what the network should look like in 2031 they are inexcusably limited - they are a closer approximation of what it should look like by 2013, not 2031. Therefore, no option is particularly good, with options #3 and #4 "winning" by dint of being less bad. General criticisms and suggestions We should be looking at a network where light rail extends to all four of the extra-Greenbelt suburbs well before 2031. Baseline to Blair is a good starting point, and it is commendable that Transitway conversion is finally on the table as a serious option, if about a decade late. But it's only a starting point. Extending light rail to Bayshore in the west and through the Smyth Road hospital corridor in the east should be the first priority once Blair to Baseline is complete. Many of the bus transitways listed should not be built as such but rather as light rail transitways. For example, there seems to be very little reason to build the Cumberland Transitway as anything other than a light rail transitway. Indeed, there is compelling case to build the Cumberland Transitway as light rail all the way to Trim Rd before converting the East Transitway to Blair since doing so will relieve much of the pressure on the latter and facilitate its conversion. I also notice that many of the secondary or "urban transit" corridors that are part of the current TMP have been reduced to grey arrows denoting "Future Urban Transit" - i.e. post 2031! There is no good reason that most of these corridors couldn't be in service by 2031 and their transit mode should be decided (as in the current TMP) now. In particular, the Carling Avenue and Rideau Street corridors are of concern since they go through downtown, a fact that can influence the decision of whether to go with a tunnel, and, if so, when. Indeed, the entire tunnel question has been treated as a foregone conclusion. Calgary still has plenty of life left in its surface LRT system now that they are lengthening the stations for 4-car trains, and that is for a system that already carries more riders daily than does the Transitway. I don't dispute that we may one day require a tunnel, and it would be prudent and wise (and good planning) to plan for such an eventuality in the immediate future, but whether we need one right away is an altogether different question - a question that was indirectly answered in the section on surface options:
Surface Options
An LRT-only option would require: For example, it may well be the case that 3-4 car trains operating on the surface would be sufficient for 15 years and would be available for use much sooner than a tunnel. Of course, that leads to an obvious question: what to do with the surface LRT once a tunnel is in place? There seems to be a one-or-the-other thought process with respect to surface vs subsurface LRT downtown. Well I refer back to my earlier point about the unmentioned "Urban Transit" corridors, particularly Carling-Rideau. This corridor would go downtown but it may not make sense to insert it into a tunnel. It might well make more sense for it to go on the surface downtown. We would then have the genesis of a solution to our tunnel vs surface question: do both but only do each when it makes sense to do so. The implementation could look something like the following: 1. Construct surface LRT from Bayview to Hurdman, and implement diesel light rail extensions south (Leitrim, then Airport and Riverside South), southwest (Barrhaven) and west (Kanata) so as to relieve pressure on the Transitway during conversion while releasing more buses immediately for more frequent suburban feeder service. 2. Convert Transitway from Baseline to Blair, or better yet, Baseline to Bayview and construct Cumberland (light rail) Transitway to Trim Rd. 3. Complete conversion of Transitway from Bayshore to Lincoln Fields, and from Hurdman to Blair 4. Begin planning/design work for the tunnel and for the Carling-Rideau tram line 5. Carry out further extensions of light rail transitway to Kanata, Orleans, Riverside South, etc. 6. Build downtown tunnel and Carling-Rideau tram line 7. Once the tunnel exists, switch Transitway trains into it and the Carling-Rideau tram can use the surface tracks. The interconnections on either side of the tunnel between the two sets of tracks can be left in place for use in emergencies or for periodic tunnel shutdowns, etc. I have plenty of other concerns with the options that have been presented, especially the vehicle costs (based on my knowledge of Calgary's CTrain network, only about $300M of the $1.3B or so set aside for vehicles is needed to buy sufficient light rail vehicles) but even some of the bus transitway costs that have been quoted in the Citizen seem excessive (i.e. $400-500M for the Cumberland Transitway between Blair Station and Trim - how does one manage to spend so much on a practically empty corridor? digging an entire new trench to rival Scott Street perhaps?). I also have concerns with the entire process in that so little has been examined. The options don't even convert the entire Transitway within the Greenbelt, never mind extending light rail across it. There is nothing in the way of phasing or what options are available for immediate implementation. In particular, there has been no discussion or apparent consideration of the use of diesel or other autonomous light rail vehicles on existing tracks to serve people outside the Greenbelt. Citizens have not been asked if the requirement to transfer at Bayview is a reasonable trade-off for having an improved ride for most of the journey and to deliver immediate benefits rather than having nothing much for (apparently) a generation. What of looking into the acquisition of dual-mode vehicles, as recommended by the Mayor's Task Force, once a tunnel is in place? These options or entire areas of discussion have been simply excluded and the result is an unsatisfactory set of limited options to choose from. Regards, — David P James Ottawa, Ontario </pre> posted at: 13:02 | path: /otrain | permanent link to this entry Submission to City's "4" transit "plans" From: David Jeanes To: friends-of-the-otrain@lists.ox.org Subject: Comments on the City's 4 transit plans. Submitted at 15:59. Only spellchecked visually. Some errors missed. Receipt of feedback was confirmed. David Jeanes === There has to be a "None of the Above" option, for this to be a valid process. All of these options assume first that there MUST be a tunnel, before we have even determined that a tunnel is necessary, feasible, and affordable. (That is what the Tunnel EA will do, and council has directed that that study team include world class transit tunneling experts). This proposal is supposed to be how to deal with downtown transit issues, but all options involve over 60 km of bus transitway construction. This is about 50% more than the total in the 2003 TMP. The total rail component in Option 3 is only about 32 km (including the existing O-Train), which is only about 25% of the LRT network in the Transportation Master Plan of 2003. There is no indication how the conversion of the Transitway to rail, which has supposedly been in the City's plans since the 1980's, would be staged to minimized distruption and maximize total system capacity at each stage. It does appear that the extimated costs for conversion are far higher than determined by previous studies. Why? The two options with bus only east west are clearly unacceptable and should have been screened out of the study early to reduce wasted effort. The public has made it clear that East-West transit MUST be rail-based. The capacity of an all bus system has been found unacceptable. The operating cost has been found unacceptable. The environmental issues have been found unacceptable. There is no potential to address interprovincial transit needs. And finally the capacity is unacceptable, (and in fact has been since the downtown reached transit capacity failure in 2004). So why are these options still being studied? The complete exclusion of surface rail solutions, avoiding re-use of our existing rail corridors, track, bridges, stations, even existing train services, is unacceptable for a rational transportation plan for this millennium. It completely ignores the Mayor's task force recommendations, and the multifaceted strategies that are being appplied in cities such as Toronto, Montreal, and Vancouver. The continued talk of Ottawa's "renowned" Transitway and the reference to transfers as "taboo" by Nancy Schepers in her presentation, goes against the evidence of the City's own public consultation. 85 percent of the people who voted at City Hall said a hub and spoke system with effective transfers was preferred. Failure to invest in transit infrastructure downtown may have seemed like a good near-tern strategy in the 1980's, as stated by John Bonsall, but a quarter Century later it has become a disaster. The system is running at its limits and has no capacity for growth, or reliable operation under any abnormal circumstances, including winter weath\er. Furthermore an all-bus system offers no economies of scale on labour costs as ridership increases, unlike multi-vehicle single-operator LRT or commuter trains. The claims that a transfer-free ride downtown is still a viable option for planning the system is ludicrous. OC Tanspo does not operate the kind of all-destinations network that Toronto is building with its new Transit City LRT plan. Instead the focus remains on the downtown which is already well-served but only represents the morning destination for 23% of trips by all modes and 31% of cross-river trips from Gatineau. The proposal to use hybrid buses in battery mode in the tunnel at the volumes proposed is a totally unproven concept. Seattle's recent experience does not scale to Ottawa's needs. This is also a very bad use for hybrid buses as ALL transitway and express buses to downtown would have to be the very expensive articulated hybrids, but would not generate any significant fuel savings or emission reductions, due to the high proportion of their route that would be on Transitway rather than start-stop. These buses could also not be used on most express routes through suburban neighbourhoods. Study a tunnel quickly (with a 6-month EA), but only for LRT and only in comparison to an LRT-only surface option, which has never been properly studied. The open house panels indicated that it had been determined that surface LRT was "practical" with 3-car trains. The powerpoint presentation was contradictory in this respect. Extend the existing O-Train immediately to the Leitrim Park and Ride and to Terrasses de La Chaudiere, with further planned extensions to interface to Gatineau's Rapibus at the Casino and to serve both an airport spur and an extension to Artmstong Road, when it is rebuilt. Work with VIA Rail and Ottawa Central Railway immediately to provide cross region commuting, effectively linked with the O-Train and Transitway at existing stations. Make sure that VIA's current and future track and signalling investments near Ottawa support these service expansions. Investigate new options for combined intercity freight and light rail service on the same tracks, as is being done in Europe and the United States. Lay out a network plan that is phased to show how the downtown transit solution will be progressively extended to the suburns, including all major siburban park and rides. Show how transit capacity requirements will be met at each stage of the expansion. Remember that the provious process was required by the federal and provincial governments to be open and transparent. It was not, and City councillors recognized this in 2006. Make sure that this process does not fail for the same reasons. posted at: 12:59 | path: /otrain | permanent link to this entry Mon, 28 Jan 2008http://blag.xkcd.com/2008/01/28/obama/ Writes:
posted at: 18:16 | path: /netneutrality | permanent link to this entry Wed, 16 Jan 2008After playing with Russell McOrmond's Neuros on Sunday, I ordered one on Monday from ThinkGeek. ThinkGeek was the only company that had stock and would ship to Canada. I elected for the UPS option, since DHL almost always screws up. UPS delivered without even collecting duty, and then an automated recording called me to bore me. The UPS fee for brockage was going to be $0.00, so I hope they will just invoice me. I wanted to NFS mount my file server, where for some time I've had a 225G partition waiting for me to setup the hauppage card in my wife's old desktop. (That machine is now a Pluto@Home, and it's mounted on a wall in basement. I might actually use the security system in it as well as the PVR stuff. We have a second Satellite/Starchoice receiver in the basement) After telling my dhcp server to give the neuros a constant IP, and putting that IP in DNS, and letting DNSSEC resign my zone (it needs randomness and I don't know why. It should be using RSA...), I added the new name to /etc/exports on the file server, and did a mount. The mount took awhile on the neuros. I had telnet'ed in as root, and the Neuros doesn't have a lockd running. I went into IRC, and learnt that I should add "-o nolock" to my mount line. I wondered how to make it mount each time it booted, and learnt that it runs /mnt/OSD/rc.userif the shell script exists. Mine is: #!/bin/sh # mount stuff from aragorn. sh /mnt/OSD/ssw.sh and /mnt/OSD/ssw.sh #!/bin/sh mkdir /mnt/tmpfs/media/aragorn mkdir /mnt/tmpfs/media/aragorn/cadillac mkdir /mnt/tmpfs/media/cadillac mkdir /mnt/tmpfs/media/new mount -o nolock aragorn:/cadillac /mnt/tmpfs/media/cadillac mount -o nolock aragorn:/cadillac/movies/Den /mnt/tmpfs/media/new The second mount was necessary so that it would be willing to accept that location as the default recording location. It didn't seem to like being told to record to a sub-subdirectory. It recorded some ads from TV to "osd.mp4" in that directory. I then wondered what happens if I hit the red record button again. It felt that it hasn't been configured as to a location to record to, so I told it the same location, and told it to default to 30 minutes. It now recorded to: aragorn-[/cadillac/movies/Den] root 35 #ls -l total 226320 ---------- 1 root 1002 49152 Jan 16 23:35 .rec_SCA_1466957263_852.bin ---------- 1 root 1002 49152 Jan 16 23:35 .rec_SCV_1466957263_852.bin ---------- 1 root 1002 0 Jan 16 23:26 .rec_SS_1466957263_852.bin ---------- 1 root 1002 0 Jan 16 23:26 .rec_TSA_1466957263_852.bin ---------- 1 root 1002 98304 Jan 16 23:35 .rec_TSV_1466957263_852.bin -rw-r--r-- 1 root 1002 34338611 Jan 16 23:09 osd.mp4 -rw-r--r-- 1 root 1002 197132288 Jan 16 23:36 osd_06.26@1607_01.mp4 (aragorn is a NetBSD XEN guest file server) I was puzzled by the new file name, until I ran date on the OSD: /mnt/OSD $ date Sun Jun 26 16:10:37 UTC 2016 /mnt/OSD $ rdate istari Wed Jan 16 23:21:45 2008 /mnt/OSD $ date Wed Jan 16 23:21:46 UTC 2008 An rdate cleared that up. I stuck the rdate into the rc.user just for luck. So far, we are very happy. Liam likes it too. Next step is to transfer all his Thomas videos to soft-copy before they VHS tapes completely wear out. posted at: 23:33 | path: /oss | permanent link to this entry
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||