*warning: plot spoilers below*. Imagine yourself in the trenches of WWI. The next chapter you're in Ancient Egypt, then in Troy. A little while later you've gone through the land of the bugs, dark/twisted versions of the Worlds of Oz and Wonderland, and many more. Of course this is before you arrive at the heart of the universe itself, where the basic rules of reality no longer apply.
The nice thing about Tad William's Otherland series, which takes place in the Otherland virtual reality network, is that he takes great liberties in his artistic vision. Otherland doesn't conform to the rules of many other artificial realities portrayed in fiction that try to preserve a consistency in their environments, rather is was built by a secret cabal of the most powerful people in the world to serve their nefarious purposes, each of which has a completely different vision of their utopia.

At first I was a little apprehensive about the last bit, in general I'm not a big fan of stories with a huge super-secret organization that runs the show, but progressing through the series it began to grow on me. Most of the main protagonists in the story start off as simple peons in the big picture, stuck in the simulation network, unable to leave, narrating the chaotic events unfolding around them. As the story progresses, the roles of the protagonists and antagonists become more apparent, but these mostly arise as artifacts of their characters, it is just who they are and often the random luck of their circumstances that results in their impact on the story.

There are exceptions to this, one of the most interesting characters is the dark serial killer knows as Dread, who is employed by the leader of the Grail Brotherhood (those that built and are running the network) to act as an assassin in both the real and virtual worlds to meet their goals. Of course things aren't as simple as they seem as Dread has perverse ambitions of his own, and seeks to control the very thing which his masters built to rule the world. Furthermore Felix Jongleur, the worlds richest man and leader of the grail brotherhood, proves to be quite interesting as well. While a bit on the cliché side his ruthless quest for immortality at any cost sets up and drives the events of the series and as they unfold around him, whether to his liking or not, and serve as many of the focal points of the epic. If you do start the series, make sure to commit yourself to reading to the end (it consists of four rather lengthy novels) as while it starts off slow, the ultimate fate of these two characters alone are so vivid that I suspect they will linger in memory for a while.

At the heart of the series, the network, and namesake of the books is the Other itself, the semi-sentient AI serving as the Operating System driving the virtual reality. The story of the Other when fully revealed is a very sad one, and there is no happy conclusion for this being, though there is some vindication with the fate of Nemesis, an ALife entity which the Other creates to quash anomalies in the network. I feel that in itself is enough reason to read the series, while many of the protagonists manage to escape the network and achieve their goals, it's no happy-hollywood ending. Rather there are many deaths and fates far-far worse, and as events unfold no character manages to escape the network unscathed, having to bear their experiences for the rest of their life. For some characters its worse than others, those with abilities which permit them some control over the network quickly find those abilities turn on them and they become more of a curse than a blessing. It is a bit of a tale in irony as nothing is ever what it seems, and while Williams often reveals the basic facts that setup for the awesome conclusion as the reader goes along, it isn't until the very end that all the pieces fall into place and the final puzzle is solved.

The series is an epic in every sense of the word, and in some senses it can be seen as a meta-epic, as other previously established epics such as the Odyssey and the Lord of the Rings are folded into the mix as characters go through those worlds, though often with a dark twist. I very much recommend the series to any sci-fi fan, the last book is so compelling I could not put it down (read the 900 page novel in a little under a week!) and I consider it to be one of my top three favourites of all time.

Until next time, happy dreams!

As a follow up to the GSOC, I've been mentoring several talented high school students from around the world who have been adding many cool new features and functionality to the isitfedoraruby webapp as part of the GCI contest. They have been doing a phenominal job on the site and an update sharing their work is long over due!

• Nico (from Argentina) has deployed the site to openshift, wrote scripts to syncronize the db on a daily basis, automated the production push process (whenever we merge our commits into the 'stable' branch on github, the site automatically gets updated), is working on i18n internationalization support, and much more
• Kendhia (from Algeria) has been working on features involving 'gamification', eg importing the contributors that maintain fedora packages, and displaying that data in visually appealing manners (including badges!)
• Mark (from Honolulu, Hawaii) has been working on adding more visualizations to the site, most notably graphically representing the packages a contributor maintains using d3.js here. Mark also designed a logo for the fedora-ruby sig, which can be seen above.
• Animesh (from New Delhi, India) implemented a timeline of contributions, so we can see a summary of all updates, both on the rubygems side and those in Fedora. He also wrote up and made a screencast of a great guide on how to get a simple hello world rails app up and running on the Fedora/Ruby stack.

We aim to keep driving the site forward, now that we've automated the push and syncronization process, getting new features into production should be very trivial. I'd like to thank the talented students for all their hard work and really look forward to continuing to work with them in the future!

Friday I flew out to Mountain View, CA to attend the Google Summer of Code Mentors Summit hosted at the Googleplex, Google's worldwide headquarters. Every year, two mentors from every organization are sponsored by google to attend the summit and I was offered the extra slot by Buddhike, the Fedora summer of code admin.

After the inevitable hubub w/ delayed flights and such (surprisingly my stow-away bag actually made it!), I arrived at the hotel on Friday night intime for the pre-summit dinner. On Saturday shuttles were available to transport us from the hotel to the Googleplex which I have to say in quite impressive. From architecture of the buildings, to the modern artwork scattered around the complex, to the meals/beverages/snacks, and even random bicyles available for any employee to just pickup and use, the complex lives up to the legend! (hover over images for descriptions, click for full sizes)

The summit was very much developer centric which I very much enjoyed, as most of the other conferences I attended included a mix of sys-admin and developer tracks. Obviously the driving event was a summer of _code_, so many of the attendees were software engineers themselves, mentoring students doing the development. Some projects were bigger than others, some well established, some not, but it was a great mix of contributors from across the industry working on many cool things.

Among the best of the presentations I attended included roundtable discussions on the latest languages and language features being used in the industry, different collaboration tools that the projects are using and their thoughts on them, and one on developing tools to quantify and measure community metrics and individual contributions. During the lightning talks (one minute a piece), I showed off Zuhao's isitfedoraruby project, and extensively promoted his work, as well as Sammy's and Nitesh's contributions to Aeolus throughout the entire conference.

I also held an impromptu session on the JSON-RPC protocol during the afternoon which was a last minute thing, but was able to leverage my presentation and demo of rjr in Brno from a few months back. This led to an good discussion with some interesting ideas emerging from it and even a couple adopters!

Saturday night was the Summit Dinner Party at the hotel, italian food, good mingling, and even a dip in the pool and hottub! This morning I checked out of the hotel and attended more sessions before wrapping up the trip (I am currently blogging this from SFO prior to catching my red-eye home).

I have to say this might just be the best conference I've been to so far. The talks and presenters were phenomenal. Even the attendees were constituted of some of the brightest of the industry, so many conversations lead to insightful perspectives and different conclusions than I hadn't come upon so far. I hope to be able to participate in the GSoC and maybe attend the summit again next year. Until next time, happy coding!

RJR uses eventmachine on the backend to process incoming messages over tcp/amqp/http/websockets/etc and dispatch json-rpc requests to registered handlers. It also employs a built in thread pool to hand off requests to the dispatcher so that the reactor isn't blocked. What results from combining the reactor pattern with a thread pool is a highly reliable concurrent event processing system.

The reactor central to eventmachine and the design pattern itself processes events one after another in a serial manner. Events are executed in the order that they arrive and no event executes until the one before it completes. Events can come from multiple sources and the reactor will take care of the serialization necessary to ensure data integrity.

A developer initializes the necessary connections and receives events by registering callbacks to be triggered on their invocation. Since the reactor blocks on any operation, the developer needs to ensure that his/her callbacks execute quickly and return control back to the reactor to continue processing events. Threads can be used to accomplish this, but spinning up a thread is an intensive operation, so a managed thread pool can be used as a nice solution that allows the developer to execute callbacks quickly and asyncronously without blocking the reactor.

Of course any resources leveraged by the callbacks will need to be protected from concurrent access but the nice thing is that if care is taken, the reactor itself can be used to handle this. In the callbacks launched by the thread pool, the developer can leverage the event machine reactor to schedule additional work that will be executed serially with the rest of the work already there. What results is an elegant / simple solution to schedule and process work concurrently with a built in mechanism to protect shared resources. Optimally the thread pool will incorporate a timeout mechanism that is able to kill jobs that exceed the timeout and restart their worker threads to keep things moving smoothly (as RJR does).

I would give this combined design pattern a name (perhaps Reactor Pool?) but I'm sure someone out there probably has already thought about this... and blogged about it... and uploaded an article to wikipedia... etc...

Be sure to see the first three parts in this series which contains the complete overview of my demo for the conference. I also have a minimal set of slides which I've uploaded here (original showoff source and commands).

And thats about it! I head out in a short bit for my 7hr drive there (stopping in Buffalo) and back on Sunday. On Friday, I'm scheduled to present in the first slot at the 'Build A Cloud' Day, and then on saturday my OLF talk is at 3pm in room C110-112. Everyone who can make it is welcome to attend, and I'll try to take some photos and post from the conference but there is always a million things going on so no promises! :-)

Take care!

See my first article in the series to set up cloud providers, and my second article on using the core Aeolus components.

Web interface

To setup the full Aeolus suite, simply:

  yum install aeolus-all
  aeolus-configure

Open a web browser and navigate to http://localhost/conductor to access the conductor web interface.

Login w/ the default credentials ('admin' / 'password').

To manage cloud provider, click 'admin' and 'providers':

Add a new provider and provider account, specifying the type and location
of the provider, access credentials, and uri of the deltacloud instance
through which to connect to it.

Manage cloud hardware and realms:

Manage pools of providers and environment which instances run in:

Manager users and rights:

Build images and start instances themselves on providers according
to any number of metrics:

CLI

To connect via the command line, setup a ~/.aeolus-cli file containing the credentials which to use to connect to conductor:

   :conductor:
     :url: https://localhost/conductor/api
     :username: admin
     :password: password

Yum install the command line utility:

   # yum install aeolus-cli

All operations available via the web interface are also available from the command line:

   $ aeolus --help
 
   $  aeolus hwp list
 
   $  aeolus hwp create --name demo
 
   $  aeolus image --help

</presentation>

That about wraps up the presentation content that I had prepared, besides a small / minimal set of slides that I will be posting shortly. I have also downloaded a few screencasts off our youtube channel as a backup plan incase things go awry with one of my machines (there is a nice / short one on the Audrey config server that I'll probably end up showing anyways to demo that component).

Stay tuned for my final post in this series!

To setup a few cloud providers to use against Aeolus, see my first blog post in this series.

deltacloud

To install, the core Aeolus component, deltacloud and the drivers for oVirt (RHEV) and OpenStack:

  # yum install deltacloud-core deltacloud-core-rhevm deltacloud-core-openstack

Deltacloud comes with a RESTful server, a command line client, and various programatic interfaces including C and Ruby. We'll start a few instances of deltacloud to serve requests to the various providers and connect to them via IRB / Ruby. A single deltacloud server can dispatch requests to as many cloud providers as desired, provided the correct HTTP headers are set such that deltacloud can resolve the remote provider entry point. But for this demo, to make it especially clear what is going on, two deltacloud instances are used:

  # API_PROVIDER='https://ovirt:8443/api'  /usr/bin/ruby /usr/bin/deltacloudd -i rhevm -e production --port 3004 --host localhost -u nobody -g nobody
  # API_PROVIDER='https://openstack:5000/v2.0'  /usr/bin/ruby /usr/bin/deltacloudd -i openstack  -e production --port 3005 --host localhost -u nobody -g nobody

Pop open an irb session and connect oVirt

  $ irb
  > require 'deltacloud'
  > c = DeltaCloud.new('admin@internal', 'cloudpass', 'http://localhost:3004/api')
  => <DeltaCloud::API:0x00000002d75968 @password="cloudpass", @username="admin@internal",
                                        @api_uri=#<URI::HTTP:0x00000002d75580 URL:http://localhost:3004/api>,
                                        @entry_points={:instances=>"http://localhost:3004/api/instances",
                                                       :images=>"http://localhost:3004/api/images",
                                                       :drivers=>"http://localhost:3004/api/drivers",
                                                       :instance_states=>"http://localhost:3004/api/instance_states",
                                                       :storage_volumes=>"http://localhost:3004/api/storage_volumes",
                                                       :realms=>"http://localhost:3004/api/realms",
                                                       :hardware_profiles=>"http://localhost:3004/api/hardware_profiles"},
                                        @features={:instances=>[:user_name, :user_data]},
                                        @verbose=false, @driver_name="rhevm", @api_version="1.0.0">
 
  > c.instances.collect { |i| [i.id, i.name] }
  => [["8b347be3-0a76-4c84-aa79-c017557540f5", "demo6"],        ["894fb094-c75d-49a8-b49d-5523346b46a3", "demo7"],
      ["98511559-46a3-4eb2-aba2-4c81c4e0d5b5", "i-1347974040"], ["29c4d6ae-87ab-46cd-8424-af05b96b1d91", "i-1348008963"],
      ["c1844032-45d4-4b17-9405-224b4942ace7", "i-1348425727"], ["12f8c82e-2837-49e0-8d41-745aa208021a", "i-1348516821"],
      ["9f3e115f-e097-4fdc-bfa3-5d634141c38a", "i-1348586396"], ["76b5bc43-6bc5-4c06-a92a-c86595bfc303", "inst1"]]
 
  > c.images.collect { |i| [i.id, i.name] }
  => [["00000000-0000-0000-0000-000000000000", "Blank"], ["d0a8df2e-1563-470e-bfd7-8384a400380f", "e3280dab-e719-4e6e-b954-55578acfe4b7"]]
 
  > i = c.create_instance("00000000-0000-0000-0000-000000000000")
  => <DeltaCloud::API::Stateful::Instance:0x000000025ddb60 @base_name="instance", @client=<DeltaCloud::API:0x00000002d75968 ...>,
                                                            @url="http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4",
                                                            @id="d637ee52-787d-48de-a6c4-a92b854677b4",
                                                            @objects=[{:method_name=>"stopped?", :type=>:state, :state=>"STOPPED"},
                                                                      {:method_name=>"running?", :type=>:state, :state=>"RUNNING"},
                                                                      {:method_name=>"pending?", :type=>:state, :state=>"PENDING"},
                                                                      {:method_name=>"shutting_down?", :type=>:state, :state=>"SHUTTING_DOWN"},
                                                                      {:type=>:text, :method_name=>"name", :value=>"i-1348597714"},
                                                                      {:type=>:text, :method_name=>"owner_id", :value=>"admin@internal"},
                                                                      {:type=>:link, :method_name=>"image", :id=>"00000000-0000-0000-0000-000000000000"},
                                                                      {:type=>:text, :method_name=>"image_id", :value=>"00000000-0000-0000-0000-000000000000"},
                                                                      {:type=>:link, :method_name=>"realm", :id=>"99408929-82cf-4dc7-a532-9d998063fa95"},
                                                                      {:type=>:text, :method_name=>"realm_id", :value=>"99408929-82cf-4dc7-a532-9d998063fa95"},
                                                                      {:type=>:text, :method_name=>"state", :value=>"STOPPED"},
                                                                      {:type=>:link, :method_name=>"hardware_profile", :id=>"DESKTOP"},
                                                                      {:type=>:text, :method_name=>"hardware_profile_id", :value=>"DESKTOP"},
                                                                      {:type=>:action_link, :method_name=>"start!",
                                                                       :id=>"d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :href=>"http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start",
                                                                       :rel=>"start", :method=>"post"},
                                                                      {:type=>:action_link, :method_name=>"destroy!",
                                                                       :id=>"d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :href=>"http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :rel=>"destroy", :method=>"delete"},
                                                                      {:type=>:action_link, :method_name=>"create_image!",
                                                                       :id=>"d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :href=>"http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                       :rel=>"create_image", :method=>"post"},
                                                                      {:type=>:collection, :method_name=>"public_addresses",
                                                                       :values=>[{:address=>"127.0.0.1", :type=>"vnc"}]},
                                                                                 {:type=>:collection, :method_name=>"private_addresses", :values=>[]},
                                                                                 {:type=>:text, :method_name=>"storage_volumes", :value=>""}],
                                                            @action_urls=["http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start",
                                                                          "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4",
                                                                          "http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4"],
                                                            @actions=[["start", "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start"],
                                                                      ["destroy", "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4"],
                                                                      ["create_image", "http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4"]],
                                                            @state="STOPPED">
 
  > i.actions
  => [["start",        "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4/start"],
      ["destroy",      "http://localhost:3004/api/instances/d637ee52-787d-48de-a6c4-a92b854677b4"],
      ["create_image", "http://localhost:3004/api/create_image;instance_id=d637ee52-787d-48de-a6c4-a92b854677b4"]]
 
  > i.state
  => "STOPPED"
 
  > i.start!
  > i.state
  => "PENDING"
 
  # and after a little while
  > i.state
  => "RUNNING"

Now you can use the same API to connect to and control openstack:

  $ irb
  > require 'deltacloud'
 
  > c = DeltaCloud.new('admin+admin', 'cloudpass', 'http://localhost:3005/api')
  => <DeltaCloud::API:0x0000000149fc90 @password="cloudpass", @username="admin+admin",
                                        @api_uri=#<URI::HTTP:0x000000014a7530 URL:http://localhost:3005/api>,
                                        @entry_points={:instances=>"http://localhost:3005/api/instances",
                                                       :buckets=>"http://localhost:3005/api/buckets",
                                                       :images=>"http://localhost:3005/api/images",
                                                       :drivers=>"http://localhost:3005/api/drivers",
                                                       :instance_states=>"http://localhost:3005/api/instance_states",
                                                       :realms=>"http://localhost:3005/api/realms",
                                                       :hardware_profiles=>"http://localhost:3005/api/hardware_profiles"},
                                        @features={:instances=>[:user_name, :user_files, :authentication_password]},
                                        @verbose=false, @driver_name="openstack", @api_version="1.0.0">
 
  > c.instances.collect { |i| [i.id, i.name] }
  => [["f6915123-c3de-4a3c-bb0e-e88143ea838f", "2012-09-25 11:32:43 -0400"]]
 
  > c.images.collect { |i| [i.id, i.name] }
  => [["a4180f79-c4bf-4d61-a801-b5990b590fb0", "f16-jeos"]]
 
  > i = c.create_instance('a4180f79-c4bf-4d61-a801-b5990b590fb0')
  => <DeltaCloud::API::Stateful::Instance:0x000000014fe1a0 @base_name="instance", @client=#<DeltaCloud::API:0x0000000149fc90 ...>,
                                                            @url="http://localhost:3005/api/instances/c32688e4-eeb0-468f-a08f-cdf047d36b45",
                                                            @id="c32688e4-eeb0-468f-a08f-cdf047d36b45",
                                                            @objects=[{:method_name=>"stopped?", :type=>:state, :state=>"STOPPED"},
                                                                      {:method_name=>"running?", :type=>:state, :state=>"RUNNING"},
                                                                      {:method_name=>"pending?", :type=>:state, :state=>"PENDING"},
                                                                      {:method_name=>"shutting_down?", :type=>:state, :state=>"SHUTTING_DOWN"},
                                                                      {:type=>:text, :method_name=>"name", :value=>"2012-09-25 14:39:31 -0400"},
                                                                      {:type=>:text, :method_name=>"owner_id", :value=>"admin"},
                                                                      {:type=>:link, :method_name=>"image", :id=>"a4180f79-c4bf-4d61-a801-b5990b590fb0"},
                                                                      {:type=>:text, :method_name=>"image_id", :value=>"a4180f79-c4bf-4d61-a801-b5990b590fb0"},
                                                                      {:type=>:link, :method_name=>"realm", :id=>"default"},
                                                                      {:type=>:text, :method_name=>"realm_id", :value=>"default"},
                                                                      {:type=>:text, :method_name=>"state", :value=>"PENDING"},
                                                                      {:type=>:link, :method_name=>"hardware_profile", :id=>"1"},
                                                                      {:type=>:text, :method_name=>"hardware_profile_id", :value=>"1"},
                                                                      {:type=>:collection, :method_name=>"public_addresses", :values=>[]},
                                                                      {:type=>:collection, :method_name=>"private_addresses", :values=>[]},
                                                                      {:type=>:text, :method_name=>"storage_volumes", :value=>""},
                                                                      {:type=>:collection, :method_name=>"authentication", :values=>{:username=>"removed"}}],
                                                            @action_urls=[],
                                                            @actions=[],
                                                            @state="PENDING">
  irb(main):006:0> i.state
  => "PENDING"
 
  > c.instances.collect { |i| [i.id, i.name] }
  => [["c32688e4-eeb0-468f-a08f-cdf047d36b45", "2012-09-25 14:39:31 -0400"],
      ["f6915123-c3de-4a3c-bb0e-e88143ea838f", "2012-09-25 11:32:43 -0400"]]
 
  # openstack instances autostart
  > c.instances.first.state
  => "RUNNING"

The same API can be used to control ec2, vsphere, gogrid and many others.

imagefactory

To install imagefactory, the core component used to build and push cloud images, simply:

  # yum install imagefactory

You may want to checkout / build the latest imagefactory from source to get all the latest updates:

  # yum install git
  $ git clone git://github.com/aeolusproject/imagefactory.git
  $ cd imagefactory
  $ make rpms
  $ cd imagefactory-plugins
  $ make rpms
  $ rpm -ivh <imagefactory.rpm> <imagefactory-plugins.rpm>

The latest imagefactory also leverages the oVirt SDK to build images:

  $ rpmbuild --rebuild http://repos.fedorapeople.org/repos/aeolus/imagefactory/testing/latest_bindings/ovirt-engine-sdk-3.1.0.3-1.fc14.src.rpm
  $ rpm -ivh <ovirt-engine-sdk.rpm>

To push images to oVirt, mount the NFS export domain you had previous setup.

  $ mount ovirt:/ext/ovirt31export /mnt/ovirt

Two configuration files are read in by imagefactory, one describing the provider which to build images for and the other the credentials to use to push an image to it. For example the rhev provider json & credentials xml files would look like:

  $ cat rhevm.json 
      {"nfs-dir": "/mnt/ovirt", "nfs-path": "/ext/ovirt31export",
       "nfs-host": "ovirt", "cluster": "_any_",
       "api-url": "http://ovirt/api", "timeout": 1800 }
 
  $ cat rhevm.xml 
      <provider_credentials><rhevm_credentials><username>admin@internal</username><password>cloudpass</password></rhevm_credentials></provider_credentials>

And the openstack config files:

  $ cat openstack.json 
    {"glance-host": "openstack", "glance-port": 9292 }
 
  $ cat openstack.xml 
    <provider_credentials><openstack_credentials><username>admin</username><tenant>admin</tenant><password>cloudpass</password><strategy>keystone</strategy><auth_url>http://openstack:5000/v2.0</auth_url></openstack_credentials></provider_credentials>

Setup a template containing the description of the image you would like to build. You may download many useful ones from the Aeolus Templates Repo. There are ones to setup httpd, mysqld and more.

Import the template into imagefactory like so:

  $ sudo imagefactory base_image template.xml

Build the image for a particular cloud provider like so (target image is from last step):

  $ sudo imagefactory --debug target_image --id b55b42f1-aa2d-418f-b635-252e5f343209 rhevm

Push the image to the specified provider like so (image id from last step):

  $ sudo imagefactory --debug provider_image --id "2492c8c2-ada4-4325-a783-da3c37871ba7" --target rhevm rhevm.json rhevm.xml

other components

Aeolus consists of many components which can be used individually or together for a seamless cross-cloud experience. Imagefactory uses one of the most popular components Oz behind the scenes, to install a bare-bones operating system with Just-Enough-OS (JEOS) to bootstrap further installation mechanisms using the tooling native to the local environment. This allows us to generate minimalistic cloud images that are as portable as possible. The Audrey config server is used to orchestrate cross-cloud services and configurations and I have previously posted about snap which uses native tooling to take portable snapshots of cloud instances. Ontop of everything the Conductor RESTful API and Web application is used to manage all this, and configure various components to work with each other with users and an extensive permission schema. Infront of this is a the aeolus-cli which provides a rich command line cloud management interface.

Stay tuned for my next post in the series which will be on the Conductor Web UI and aeolus-cli.

This weekend I head out to Columbus, OH for the Ohio LinuxFest where I will be presenting Aeolus. This is one of the largest community run Linux conferences in the states, and should be my biggest presentation audience yet, so I'm really looking forward to going. The presentation leans heavily on a live demonstration of the software, with a minimal introduction to the topic and architecture at the beginning. This week a few of us in the Aeolus community are going to be blogging / tweeting / etc to try and drive a bit of buzz around our recent work which includes support for more cloud providers (prominently openstack!), various command line utilities and migration tooling, and our community efforts.

The demo consists of two machines, my desktop workstation acting as an 'external' cloud provider, and my laptop running a vm w/ the Aeolus suite on it. Network resources are always unreliable at conferences, so I'm not planning on deploying to ec2 or similar, but rather I have setup OpenStack and oVirt (RHEV) on my desktop, each bonded to their own management network interface, through which they will be controlled via my laptop. The exact same commands and setup I will be demoing works with any cloud provider supported by the Aeolus suite (which is the most comprehensive IaaS cloud management suite to date).

The demo will be run on freshly installed Fedora 17 systems with components from the Fedora package repositories. In retrospect the hardest part of setting everything up was setting up the cloud providers themselves, after which Aeolus was a cinch. It is no surprise through, a Cloud-based computational environment is a very complicated thing, and involves orchestrating many disparate components, each of which may be complicated in its own right. This reinforces the need for a technology such as Aeolus, as even with open frameworks such as oVirt and OpenStack, the end user is often adopting one particular way of doing things, which entails certain benefits and drawbacks. By adopting open standards 100% centred around portability, one is freeing themselves from being tied to a specific implementation and may choose at any point to adopt alternate solutions with minimal effort.

Attached below is my guide on setting up oVirt and OpenStack on a fresh F17 box (after the jump). I plan on posting an Aeolus overview later to demo interaction with the providers setup below.

Recently our Sensai sent out an email to the members of our dojo requesting submissions for articles to include in the Fall Newsletter. I've been meaning to write a bit about the sport I love and figured this would be the perfect opportunity and venue to do so.

To step back a bit, Aikido is not only a sport, but also a way of life. Aikido literally means the unification (ai) of the spirit (ki, energy) with the way or path (do). Based on principles of simple physics and mechanics to redirect and use an attacker's force against them, Aikido is the culmination of millenia of human warfare refined to the point where the a practitioner (called an Aikido-ka) learns the optimal placement of one's body, mind, and spirit so as to diffuse an attack and neutralize the enemy. As part of this, one of the prominent lessons learnt by an Aikido-Ka is self control. One must be aware of and in control on one's body and mind at all times to anticipate and affect the course of actions. Only through this continual awareness and relaxation, one can free the body and mind to the point which actions and reactions can occur instantly and with optimal efficiency.

As in the dojo one must be aware at all times to fully appreciate and confront life. While the traditional Japanese saying goes "Cry in the Dojo, Laugh on the Battlefield", many of us are fortunately to live in an age and part of the world where we do not have to face an enemy in combat. Regardless, we as humans still have to content with a continuous battle against our own selves. Our own psyches play against us on a continual basis, fuelling the emotions and feelings that drive the human experience, both the ups and downs, the good times and times of sorrow and anger. It is often too easy to get wrapped up in all this and feel lost / without control, some people living their whole lives this way.

Fortunately the exact same mental training practised by an Aikido-ka to control one's mind and body can be applied to one's self in any situation. By being constantly aware of who we are and what we are feeling, one can continuously examine one's self and alter his/her mental state to the point they desire at any time. With enough practice this becomes completely natural, a hair trigger reaction, whenever one starts feeling anxious or sad, one can train him/herself to completely relax be aware of their current state of being and not let the trivialities of our lives get in the way of the magnificence of existence. After all the human experience is just that, perceived mental interpretations of limited sensory inputs in a universe that is incomprehensible by our standards. Only by freeing the mind can one grasp the true potential of human existence.

Another central concept to master while studying the Art of Aikido, is that of infinity. If such a concept can even be grasped at all! When an attack is launched or received, whether deflected, countered, or neutralized, the Aikido-ka must let his or her energy flow from their center to infinity. If one tries to refrain and hold back or stop the reaction and the natural movement, even for a split second, the attacker can and will gain the advantage. When practiced correctly, there will be no possibility of a counter or resistance. The technique will proceed along the most efficient mathematical path through which all the opponents energy will be redirected to where they have no support and balance, using gravity, leverage, and the Aikido-ka's core / center of strength to drive the counter.

Likewise outside the dojo one must try to continuously expand one's conciousness beyond our limited human perceptions towards infinity.At any given moment we as humans are experiencing only an infestisimle amount of what there is to experience. Furthermore, many people are so lost in their thoughts that they don't take the time to experience every moment in its fullest, taking in the vast amount of energy and signals which we are experiencing at any given time. Even at this very moment you are absorbing light, subconsciously interpreting the many audio clues from the environment around you, even every-continuously adjusting your position due to gravity and friction in minute ways. All forms of energy interact with each other no matter how big or small, and after all all matter is energy! (just moving very slow, E=MC^2).

This brings us back to the the Ki (energy) in Aikido. By unifying one's energy with one's path, one can harmonize and balance their life and by mental training one can begin to detach themselves from the little/trivial things that consume so many peoples lives. That is not to say to not enjoy life pleasures. But everything should be viewed in the scope of what it really is and one should not get too wrapped up over anything. I very much enjoy the Martial Art of Aikido, and am planning to continue training as long as I can. I encourage anyone who might be interested to find a dojo around them, there are many great schools all over the world! Until next time, onigashimas!

A few months back I gave a presentation to the Brno Ruby Users group about the JSON-RPC protocol and my implementation RJR, but didn't go into too much detail here (blog post). Recently I've been pushing many updates and improvements, including a sorely needed docs update, and figure now would be a good time to do just that.

The goal was to develop a rpc mechanism that was as extensible and pluggable as possible with the implementation being transport agnostic, eg the developer would be able to satisfy and invoke rpc requests over a variety of transport mechanisms, such as tcp, http, websockets, amqp, and more. This would provide the most outreach for developers, allowing their methods to be invoked in a wide variety of infrastructures and existing systems. It was also important that the handlers be able to determine which transport a request came in on, so as to be able to alter flow-control if desired. To accomplish this, RJR sets a variety of instances variables in the scope of the invoked method handler, things like @rjr_node_type will contain the transport which the request came in on, and other things like @rjr_callback allows the server to send json-rpc request methods back to the client, so long as the transport mechanism remains intact (eg the tcp or websocket remains open, the amqp queue is still valid, etc).

# example server from the RJR documentation:
# define a rpc method called 'hello' which takes
# one argument and returns it in upper case
RJR::Dispatcher.add_handler("hello") { |arg|
  arg.upcase
}
 
# listen for this method via amqp, websockets, http calls
amqp_node  = RJR::AMQPNode.new  :node_id => 'server', :broker => 'localhost'
ws_node    = RJR::WSNode.new    :node_id => 'server', :host   => 'localhost', :port => 8080
www_node   = RJR::WebNode.new   :node_id => 'server', :host   => 'localhost', :port => 8888
 
# start the server and block
multi_node = RJR::MultiNode.new :nodes => [amqp_node, ws_node, www_node]
multi_node.listen
multi_node.join

Since JSON-RPC is a very simple protocol I also wanted to add a mechanism to allow developers to extend the protocol easily, even if this meant that these customizations would only work against nodes running RJR. To do this RJR allows developers to set arbitrary headers to be written to the json-rpc request, so that method handlers and their invokers may process this additional metadata and do what they will with this. For example, a node being used as a server can take method arguments, authenticate them against any backend, and set a 'session-id' header on all subsequent messages. All subsequent client requests will contain this header which is available to the handlers that can authorize the user. (obviously the end user would want to use a secure transport mechanism incorporating ssl to prevent session-hijacking)

# example clients from the documentation
# invoke the method over amqp
amqp_node = RJR::AMQPNode.new :node_id => 'client', :broker => 'localhost'
puts amqp_node.invoke_request('server-queue', 'hello', 'world')
 
# invoke the method over http using rjr
client = RJR::WebNode.new :node_id => 'client'
puts client.invoke_request('http://localhost:8888', 'hello', 'mo')
 
# Invoking json-rpc requests over http using curl
# $ curl -X POST http://localhost:8888 -d '{"jsonrpc":"2.0","method":"hello","params" ["mo"],"id":"123"}'
# > {"jsonrpc":"2.0","id":"123","result":"Hello mo!"}

As far as next steps, flushing out the UDP transport mechanism and continuing to optimize performance are high on my list. At some point I would love to do a complete rewrite in a lower-level language such as C and simply write wrappers / adapters so that methods implemented in higher level languages can be invoked simultanously. But for the time being, RJR serves my purposes and will continue developing that for now.