let’s do this!!
Woo! We shipped an implementation of libcloud, check it out over on libcloud.org. It is a pure python library for working with a variety of cloud server providers. It is still a work in progress, but would love to get others involved!
Today has been an amazing week — and it is only half over! On Monday, we launched our product Cloudkick. Cloudkick is a system to manage cloud servers, like those provided by Amazon EC2 or Slicehost. We have been building this over the past three months as part of the y-combinator program. It was very exciting to get it out the door!
So far the results have been awesome.
We broke 2000 registered servers tonight! Note, the x-axis is about 1.5 days, we nearly doubled in size!
The press has been great too:
Today we pitched as part of Y-Combinator’s demo day. It went very well. We said all the things we wanted to say, and had very strong interest from investors. I can’t name names or anything — but things are looking good, even though we are in a down economy!
We are still are working some kinks with the system, and quickly working on adding new features, but so far everything is going well. I’m excited to see what the rest of the week has in store.
Put together a little hack … thought I would share … it allows you to do automatic failover on EC2, using the AWS “elastic ip” feature. Not perfect, but very cool!
Posted over on the AWS forums:
http://developer.amazonwebservices.com/connect/thread.jspa?messageID=112816
Tonight I got myself pretty excited about an EC2 hack. Essentially I was able to create arbitrarily sized root filesystems, when the limit was supposed to be 10GB… or so I thought.
A little background. Amazon AWS allows you to make custom machine images to boot their “elastic compute” (EC2) nodes. Essentially they are a giant file with the entire OS in it. The file gets slapped on their virtual servers and booted. You can create these images with tools that AWS gives you. The images are limited to 10GB, which means your root (/) filesystem can only be 10GB in size.
Normal Example:
root@domU:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 1.4G 8.0G 15% /
I was investigating how the tool that created the images worked. After poking around with the build script, I noticed that the max size was hard coded. “Hmm, I wonder what happens if I just increase this…” I figured if the file size protection would be else where in the stack, like in the upload part, or when you actually register the image with AWS.
Anyway, the image built without a problem … then it uploaded, no problem … and then it registered with AWS! At this point I was pretty excited, but then I remembered I still had to boot the thing. Guess what?
It booted!
root@domU:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 50G 521M 47G 2% /
That’s a 50GB root file system! “I’m such a hax0r.” Well, at least that’s what I thought — up until I wrote out an 11GB file and the filesystem went read-only then crashed the box. Looks like AWS did the filesystem size limit in the kernel and/or the hypervisor. Good idea AWS … “bummer, not a hax0r, just ignorant.”
Lessons learned:
The always awesome Paul Querna came up with a cool trick to log cache hits/misses. In your virtual host set the environment variable SetEnv CACHE_MISS 1, then log it using your CustomLog with %{CACHE_MISS}e. It will log as “1″ when it is a miss, or as “-” when it gets hit.
How does it work? To quote Paul:
20:36 < @pquerna> as the setenv command runs in fixup hooks
20:36 < @pquerna> which is after the Quick Handler
20:36 < @pquerna> that mod_cache uses
Mostly just noting here so I don’t forget in the future … Thanks Paul!
Keeping and eye on the police has never been so much fun! Recently, my friend John lent me his USRP, allowing me to play with GNURadio. From wikipedia:
The Universal Software Radio Peripheral (USRP) is a high-speed USB-based board for making software radios.
GNURadio is a free software, software radio. Essentially, the USRP allows you to tune or transmit any radio signal, given the correct tuner/transmitter attachment. Currently it is anything in the 0-5.9Ghz range. The USRP John lent me came with a FM/TV tuner — meaning I can listen to anything between 80Mhz and 800Mhz.
What exists between 80Mhz and 800Mhz? Lots of stuff! For example, normal FM radio is between ~87Mhz – 108Mhz, all of which can be demodulated using GNURadio. Also, analog TV transmits in that range, allowing you to watch or listen to any of that on your computer.
But! It gets even more interesting. For instance, San Jose International Flight Control exists around 124Mhz (although, I have not heard much there…). Or, you can even listen to Police Central at 460.4Mhz. Listening to the police has been pretty fun. So far, Sam and I heard about a stabbing at a local restaurant, and plenty of phone number / license plate checks.
Anyway, so far I have not done much hacking on the device — there are a lot of example applications to do the FM demodulation. The one thing I did figure out was now to remove the FM static when the channel is not in use. It took me awhile to figure out, mainly because I did not know the correct terminology. Turns out, the term is “squelch“, and gnuradio implements a block to do this. All I had to do was create a squelch handling block, put it as part of the rest of the processing chain. Looks something like this (inside the usrp_wfm_rcv.py example program):
squelch = gr.simple_squelch_cc(0, 5e-3)
Then add the squelch to the rest of the connect chain:
self.connect(self.u, squelch, chan_filt...
This thing is super cool — and I’m excited to see what else I can make it do.
Just a quick note to point out my, now open sourced, videostrip generator.
My dream goal would be able to generate these using a webapp and then let the site grow. I originally implemented a flash recorder, but now my flash trial has expired and I am unable to continue messing with it. Guess that means it is time to start hacking on a firefox-addon/ogv/icecast stack to replace the existing flash-plugin/flv/flash-server stack! It’s 2008, why is flash still the only solution for streaming video out to the web??
Last weekend I went to Austin, TX, for the Makers Faire. There, Pat, David, and I make a crap load of custom ice cream scoops. The process involved filling a dixie cup with whatever ingredients you wanted, including fun stuff like chayanne pepper and BBQ sauce (some people made normal ice cream, too), and then instantly freezing it with liquid nitrogen. The kids loved it!
Just as a note, this was a promotion for Pat’s just published book, the Hungry Scientist Handbook. The book is meant for foodie/geeky types — I’d definitely check it out if you self identify with either of those groups.
Anyway, I made a timelapse of the setup. Here, you can see us getting ready and serving for about half a day (then my battery died).
After discovering CHDK, I’ve started messing with timelapses. Here is how I created the above timelapse:
mencoder "mf://*.JPG" -vf scale=640:480 -o timelapse.avi -of lavf -ovc lavc -lavcopts vcodec=mjpeg -lavfopts format=aviAnd tada! You now have timelapse.avi.
I’ve been casually hacking on a videobooth project that will generate videostrips (analogous to photobooth and printing photo strips). Here are the full results of the free culture conference, where I got to test the system out last weekend. Check it out at, videobooth.me/fc2008/.
Yesterday I demoed this to the team at Eye-fi, and they said I was the first person to show them video sent over wireless from a consumer camera (+eye-fi). One of the first implementations of wireless video on consumer cameras, I’m pretty stoked! It means you can use your cheap Canon camera to auto post data to youtube, or some other video sharing service.
If you are not familiar with Eye-fi, it is a special SD card that allows you to upload photos over wireless as you take them. It does not, however, upload videos as you take them. This is because it only looks for valid JPGs. I was able to achieve video uploading by creating a spoofed JPG with CHDK that embedded the video. Eye-fi uploaded the JPG, then the AVI was decoded client side.
To use this, you will need to make sure your camera is supported by CHDK (see table on the homepage). I’ve posted the patch over on the CHDK forums. You can get a built firmware for my camera (SD700IS) here.
