Codemarc's Blog

The CodeMarc of Excellence

A Fresh Start?

“Spring is the time of plans and projects.” - Leo Tolstoy (Anna Karenina)

myoldell
As a software developer with a background in hardware it is always interesting and often painful to setup a new work environment. As it happens to be a sunny spring day (good friday almost easter) it seem that the theme of renewal is here. Coincidentally my old Dell precision workstation (circa 2004, perhaps the first 64-bit machine i ever used) has finally died.

Over the past few years I have bee running Ubuntu as my primary desktop working environment. This is probably the reason I to so much milage out of the old system. Each time someone else’s Dell precision workstation died, I would cannibalize the good parts and add them to my system. Net result is that for a grand total of $65 (for a galaxy force 6500 video card, an old but awesome card appropriate for the old system)  I had a great 2 dual core cpu’s (4 total), 24G memory and about 1 terabyte of disk space dual monitor developer workstation. So what happened? One of the three old fans died probably about a month ago but ubuntu didn’t care, it kept on ticking. eventually I needed to use the dvd drive for something and noticed it was dead, so eventually I powered down the system to check it out. And that was all she wrote. Funny thing is that although it was obvious that some of the insides had issues, the hard drives (2 beautiful SATA drives) were just fine. Oh well.

optiplex7010So I removed the good parts and scrapped the old system and ordered a new one. You know what, sometimes new is really better. I ordered up a Dell Optiplex 7010. with 3rd Gen Intel Core I7 (quad core, bug cache, real fast) so basically a 8 cpu machine with 16G of memory, 2TB disk, no monitors no fluff  for about $800 bucks.  It came in today so I ware ready to go to work on it.

On boot I got a brand new super clean version of window 7 running. I quickly configured the dual monitors and that I thought about how to configure the machine for real. After think about it for a while I decided there was no real reason torero what was done and as long as I am vigilant in the way I use the host (in this case windows 7) it should service as a great platform for development. So I painfully located all my windows application license codes for the basic stuff. For me this means Google Chrome, Dropbox.  Office 2010 (64-bit), Adobe Acrobat and virtualization software (I typically use virtual box). A few hours I finish installing the latest UBUNTU as a guest and start using it. AND it sucks. What—–

It just doesn’t feel right to me. Several discussions later and I realize it. Linux or OS X on the bottom is always the way to go. So now what. I guess I can use something to suck up my windows drive and virtualize it. Or may be I should go right ahead and boot Linux from a different physical drive. Then I can use the actual physical drive (with windows as the guest) and maybe that won’t suck at all. By the way I’ve also had a great deal of success using the latest wine stuff, so maybe I can look at that as a another option.

Funny thing about the story is that while I sit here waiting for all the file transfer/shuffling to finish (I needed to rearrange some of the stuff in disk) I am typing on my mac using its notes app. I really just use my mac and don’t ever really think about it.  So thats it for now I wait:

“Expect to have hope rekindled. Expect your prayers to be answered in wondrous ways. The dry seasons in life do not last. The spring rains will come again.” - Sarah Ban Breathnach

What do you do?

I am often asked, what do you do? I typically respond, I am in computers. I have no idea why this is an appropriate response but in the beginning it was quite literally true

dc_la120

I learned beginners programming (BASIC) in 1977, where the Dec-Writer was king and the ribbon was always broken. You literally got your hands dirty if you wanted to get anything done. I then took an after school job at Oliver Office Equipment Company, an original typewriter repair company that sold and delivered office supplies.
The ugly part of fixing typewriters was crawling around the basement of the store to find a replacement part from the hundreds of broken buybacks. I was just beginning to learn how to repair the IBM Selectric (with its million or so unique parts) when the first shipment of Kaypro II computers arrived. As an employee, I got the 50% discount and my first real box for about $850.00. I still have this computer on my window sill and it still works (although half of the 360K, 5 1/4″, single sided, single density are de-maged). Taking that bad boy apart and putting it back together was alway fun.

Then there was my work study job at Syracuse University with the campus computer services department. Day 1, I was given a cool briefcase full of tools and handed a bunch of trouble tickets and sent out to fix terminals. About half of the tickets were to replace ribbons on old Dec-Writers around campus and the other half was reseating video cards in volker-craig terminals. Then there was the day where a friend of mine tried to replace a card on terminal where he had inappropriately grounded the static discharge. And calling the ambulance after he flew across the room convinced me it was probably time to get out of being in computers.

So what do I do? I read, I write, I hack, I invent, I think and I teach. Creativity is a part of who I am. As an mathematical artist, I find beauty in music. I love its patterns and symmetry. I am interested in the layers and complexity that I hear and feel effortlessly. The math of technology is harder for me. I love simple logic, truths, repeatable certainty. I am fearful of complexity in that it can be a huge distraction. I am a fan of good field position, keeping your eye on the goal, and getting it done and while I dispise the notion of “good enough”, as an adult I understand it. The essence of Zen is on not identifying with one thought or its opposite, it is about getting to the awareness that is behind the thought.

I leave you with this thought: A distraught man approached his master. “Please, Master, I feel lost, desperate. I don’t know who I am. Please, show me my true self!” But the teacher just looked away without responding. The man began to plead and beg, but still the master gave no reply. Finally giving up in frustration, the man turned to leave. At that moment the master called out to him by name. “Yes!” the man said as he spun back around. “There it is!” exclaimed the master.

Big Data Floating in the Clouds

For the last few months I have been building a prototype on top of an Apache Hadoop 1.0.4 cluster that I  built from scratch out of six virtual machines running  Ubuntu Server 12.04.2 LTS. It has been an interesting experience. Simply put, this is the actual learning process that every hacker goes through on every new project whether its a programming language, platform or technology. So now that I got a handle on the basics and I can take an earnest look at other peoples packaging.

Today I am checking out the current offering from Cloudera. I found the download named Clouder Manager 4.5 Free Edition, and proceeded with the installation. Of course I need to install it on a few nodes so I am back to setting up some more servers.

Cluster up

This time I decide to use my mac pro server configured with virtual box. I planned on running a three server cluster (cloud1,cloud2,cloud3) so I set it up and run into a few networking problems. I get my ops dept to fix my port to allow for multiple mac addresses. Here are some of the issues and solutions I encountered when setting up the environment:

For each cloned virtual server I needed to change (persistently) its host name and mac address. The tools ( virtual box in this case ) should have properly handled this. It did NOT. So I did the following  hand job on each machine.

  1. sudo vi /etc/hosts
  2. sudo vi /etc/hostname
    (remove cloud 127.0.0.1 definition from each)
  3. sudo vi /etc/dhcp/dhclient.conf
  4. sudo rm /etc/udev/rules.d/70-persistent-net.rules
    sudo mkdir /etc/udev/rules.d/70-persisitent-net.rule
    (thank you Peter Mount)

Install Cloudera Manager (Free Edition)

So my first installation was from my remote desktop linux to my cluster and it failed. I then decided to allocate another local instance (cloud0) and try again. The installer runs ok and i point my web browser at http://cloud0:7180, login as admin/admin and away we go:

This installer will deploy the following services on your cluster:

  • Apache Hadoop (MapReduce, HDFS, Common)
  • Apache HBase
  • Apache ZooKeeper
  • Apache Oozie
  • Apache Hive
  • Hue (Apache licensed)
  • Apache Flume NG
  • Cloudera Impala (Apache licensed)

You are using Cloudera Manager (Free Edition) to install and configure your system.

I specify cloud[1-3] and get the following results:

Expanded Query Hostname (FQDN) IP Address Currently Managed Result
cloud1 cloud1.ibi.com 172.30.240.110 No Host ready: 9 ms response time.
cloud2 cloud2.ibi.com 172.30.240.111 No Host ready: 7 ms response time.
cloud3 cloud3.ibi.com 172.30.240.112 No Host ready: 16 ms response time.

While it took a few tries I finally got the following:

cm3

So now It asks me decide which CDH4 services I should install. I pick core hadoop for my first attempt withan embedded PostgreSQL database setup:

Database Host Name: Database Type: Database Name : Username: Password:
cloud0:7432 PostgreSQL hive hive aflhU8ZThz

and all defaults for the rest. 13 steps later  and viola:

cm8

Now What

cm Ok so its installed, and we can see. I guess I have to spend some time installing my parts and working with this version to see what happens and how it behaves. But thats for another day.

Thoughts on a long and healthy battery life

So today, I woke up as usual, ran thru the morning rituals, ate breakfast, grabbed the keys and phone and out the door I went, A very typical day. I jumped on the train, found a seat and then started my work day (I get 1 hour each way, 5 days a week, 4 weeks a month. So I basically work an additional 40 hour week on the train).

First I check my phone and notice it is off. I try to boot it a few times and realize, no power, way lo battery as it flickers off. Who knows why, maybe the charger wasn’t really plugged in or maybe some-one switch batteries on me (again!). Like I said a typical day. So I pull out the laptop, as this is Ubuntu 12.04 LTS release time, now is a great time to play with it. As I open the lid, I notice that it is powered off. Like the phone, I try to boot and cant get passed the bios check. Like the phone, another device is dead again. Next up I the iPad and it’s batter meter is red with 5% left. I try using it a but but shortly device #3 is doa. So I pull out my workout iPod nano, at least I can tune out the trip. Guess what, another fail.

Holy moly, did we have an EMP burst? So here I sit on the electric train with no electronics to use. Perhaps its time to read. I check my bag and no books, no mags, no papers except an old paycheck envelope. So I grab a pen and begin to write, Old school. Why does the simple act of writing with a pen seem like such a luxury in 2012. Here I sit on a quiet train, traveling with my thoughts and the time to compose and record them by hand (which is beginning to hurt since its been a long time since I’ve written more then a signature). As I look around I notice I am the only one with a pen, in hand. Everyone is glaring at the tiny devices; tablets, phones, laptops, etc. I am surrounded by energy and yet I have none of my own. I guess this is the real reason why, I don’t like Mondays. Ive always know that I had no real power. It’s a good thing we don’t need to remember to recharge ourselves.

Live Long and Prosper

The new York scrum

Last week I spent three days attending a Learning Tree course entitled “Agile Project Management With Scum” Course 918.
I took this course because I was looking for some validation on what my scrum team had been doing and I figured that this was an easy way to achieve it. And anyway the company had some education vouchers so it was pretty easy to arrange.

Getting up real early and riding the subway downtown had some real surprises. First of all, I had forgotten that if you leave the house real early, then parking at the station is easy as there are many many spots. Also the subway downtown pre-rush hour is pleasant, although the extra 15-20 mins each way kinda sucks. But the biggest thing that I had forgotten is how simply wonderful lower Manhattan is in the morning.

The facility is located at One New York Plaza-31st Floor. This spot is across the street from South Ferry, and has a great view of the waterfront including Governors Island, the Statue of Liberty, and the rest of the harbour. I have not really spent much time downtown since the summer before 9-11, so this was a pleasure. I roamed around quite a bit and kept thinking about the “Gangs of New York“.

The course was lead be a guy named Maurice Hagar who was reasonably well trained as a “professional teacher”. These classes always feel like they are a little heavy in the marketing aspects of “professional education” but I guess everyone needs to make a buck. There were at least 25 attendees and while the information disseminated can be learned from books there is nothing like being forced into participation with a group of strangers. I had initially thought that I was going to hate this, but it turned out to be really ok. And the whole experience was worthwile.

Sometime agile is referred to as fragile. In the end the all we need to do to tighten up our current agile process is to get a little stricter with out adherence to the principles and maybe do a little less scrum but.

iPad – DellLaptop + iPod – Blackberry + iPhone + MacBookPro = iPerson

From my trip to San Francisco, Dreamforce 2010

Monday December 6th, 2010

So for this trip I took both my iPad and my laptop. I finally signed up and powered on the AT&T wireless cellular plan (the cheapest one) as I am sure 30,000 people on wifi may not really work all the time. I have every intention of trying to not really use the laptop (if I can) we shall see.

I am not loving the WordPress app on my iPad. Sitting here, 7 miles high tapping away, sharing my thoughts on a westbound AA flight to the San Francisco high-tech homeland to attend a “global gathering” feels so familiar, and yet so different. A simple finger brush off of the writing surface and my unsaved local draft has vanished (for the second time). Oh well, more time to kill while I retype it in. I guess it must be a feature of “airplane mode”.

On another note, i got this free book on my iPad and have been reading about quantum physics and reality, awesome stuff. No wonder I often feel lost in space.

Tuesday December 7th, 2010

So the conference is off In full swing and today I decide to carry both devices. Of corse, my bag is now extra heavy. Good thing I did a full workout this morning. All morning I use my iPad, writing, tweeting, chatting answering emails, using apps like clipboard (my new best friend). Connectivity is somewhat spotty. Sometimes I can get a signal, sometimes it fades and I switch to cellular (which also works 50/50). The only missing element is a camera. Ok I still carry a blackberry and it has one so a quick email and were in business. I wonder if I can pair the two using Bluetooth. Have to try that later.

Now it’s afternoon and I am listening to a session on what’s coming in the next version of chatter. The dude speaking is the primary developer so at this point I whip out my laptop and load up my dev environment so that I can make noted and code changes right here. Sorta glad I had the laptop cause typing code snippets on the iPad is a little painful. It is evident that if I am going to use this as mobile device I probably need to invest in some sorta keyboard.

Wednesday December 8th, 2010

Today I decide to leave the laptop in the hotel. This causes me issues on many fronts. First, I am paranoid and am always concerned about theft. So I feebly hide my laptop in some dirty clothes in my suitcase cause it make me feel better and I am off to the races. I get a good seat at the keynote and establish connectivity early. The keynote starts about 15 minutes late and guess what happens. People are tweeting about things that have not yet happened. Talk about great marketing, poorly executed. Not too many people really notice or if they do they don’t think its real important.

Later in the day, as Stevie Wonder and Bill Clinton speak having the iPad to watch the feeds in real time is awesome.

By the way as, even though I am a republican, I will always be a friend of Bill C.

Wednesday December 9th, 2010

Mornings uneventful, conference ends and I am off to the airport. Once again even though there is free wi-fi, it is not as available and clean as it is in NY. So here I sit in the plane home, tapping out this entry. Of corse since I eat with my hands the screen is getting a bit salty. All in all I am convinced, if your not writing code you can probably leave the laptop at home.

If any one has thought on this I would love to hear about it. In the meantime I think I’ll go back to reading “butterflies are free to fly” by Stephen Davis.

Peace :)

Thanksgiving 2010

So it’s thanksgiving day 2010 and we are all still here, plugged into the matrix, living out our lives, growing more dependent on technology then ever before. As I sit on the long island railroad headed east, I peck out this post on my IPad, listening to Coldplay (the escapist) on my iPod, surrounded by other iPeople and their devices. Communicating constantly, tapping or talking, looking for instant amusement. I carry no less then 5 separate devices with global connectivity capability and yet I feel more alone and isolated then ever. And I am sure it will only get worse. I remember reading the Arthur C. Clarke novel 2010, as a boy and thinking how far away that was. This year my father turned 80 and he told me when he looked in the mirror he was happy to see his father smiling back. Anyway, I remember thinking about the line, all these worlds are yours and how I never considered the concept as applied to inner space. Now I simply think well maybe something is going to happen, something wonderful. Its good to be a dad.

So I am thankful, for my family, for health and happiness and hockey and grapes. Gobble-Gobble!

M ;)

Ps I stumbled across the greatest secret in the world, so I thought I’d share…

Magic not!

So this summer I finally broke down and purchased some new backyard furniture. My wife and I went to the Macy*s Outdoor Furniture shop and bought a very nice, “fairly priced” set. Hayley opened a new account for this purchase that included special financing. Of course Macy*s promised speedy delivery. Of course they failed on that. So to placate us Macy*s offered and then sent a $200 Gift Card as an I’m sorry.

I never shop, even though I work in Manhattan (across from Macy*s). Hayley, my wife insisted that I take the gift card and buy myself some clothes. So, I plopped the card into my wallet and forgot about it till this morning when my she made a little remark on how the bottoms of my pants were ripped.

So today at lunchtime I made my way into Macy*s. I picked out 4 pairs of pants. I tried them all on. I had plans to buy more but it had already been 45 mins and I had enough of the “shopping” experience. I go to a register to check out. And like magic 2 pairs of the pants were on sale, how random, how lucky. I go to pay with the gift card and guess what, the clerk says: Sir, this card has $2.46 cents on on it. After a fair amount of discussion (loud angry discussion), I decide to just pay. The clerk then informs me to get the sale price I must use my Macy*s credit card which I do not have in my pocket. He tries to use the automatic lookup system but it fails. So they call someone and tell me the card is in my wife’s name and I am not authorized. At this point I get angry, I have had enough I am done. But after a small rant I apologize to the clerk and ask for a manager. The clerk politely explains I need customer service to help me straighten this out and I should see the girl upstairs at the desk. I find the girl by the front door and explain it all to her. She tell me I should go up to the 9th Floor executive offices and find someone up there to straighten it out.

I decided that I am too hungry to continue with those retail nonsense. So now Hayley is on the phone with Macy*s credit. I cant wait to see how this turns out. Well see who wears the pants ;)

Turns out the gift card was used before it was sent to us in topanga, California. A place I’ve never been. More fraud, more fun. Still even though they will send me another card, I have no pants. Oh well!

Keeping Clean, Still Searching…

I am always amazed at how much effort is required to setup and maintain a clean computing environment. I think back on my earliest days of computing on my Kaypro II, with its dual 5 1.4 inch floppy drives and its CP/M operating system. The majority of the Console Command Processor (CCP) was dedicated to managing the disk to maintain a “clean” environment with available space. During those days the ultimate way to create a clean environment was to get a new floppy disk and run the sysgen command to install a clean copy of CP/M on it. Then boot the computer with that disk and viola!

Forward to MS-DOS, where the situation doesn’t really change much except you now have a 10MB hard disk to store all your junk.  Of course sooner or later the drive fails and you lose anything that wasn’t on the floppies in the first place but after the first or second time it happens you get in the habit of backing up your real stuff. And with a new drive comes a new OS and a clean environment.

Then there was NetWare. Finally some real storage to play with. I remember buying a physically massive 1G drive, loading it up with tons of stuff downloaded from Compuserve, Prodigy, PCTOOLS and OS2TOOLS (of course you had to be inside IBM for the last two).  Now, I could install lots of great/marginal/bad software and really screw up my environment. I got into the habit of wiping out my environment, once a year, simply to clean it up.

Enter the Matrix, the Internet and the ability to download as many viruses as you please. Forget about a clean computing environment and worry about how to avoid real theft. Open access points, unsecured networks, unencrypted passwords, etc. Wireless, faceless and costly.

So now its 2010, and my wife just gave me an iPad for my birthday. A new "computer" and another chance to maintain a clean computing environment. I have watched my son and daughter play with their iTouch versions of is device for a while now and I could never really understand why they liked it so much. I simply assumed that it was because they are young / hip and I am a PC (kudos apple marketing).

Well maybe that’s not really so true anymore as I have evo-loved into a daily ubuntu user, fluent in Linux and the way of the web. It turns out that this device is really a debian variant and as my honor student son, enthusiastically tried to explain to me the concept of jail breaking, I finally fell in real like. It’s funny, he thinks of it as free as in beer, and I think of it as free as in liberty.

So now I have a dilemma. Do I hack away like many others and treat this device as another real computer to master and mess up. Get a shell and login as root and go to work. Or do I us this device as a true consumer. A terminal would be real nice. And maybe, just maybe, I could stop lugging around my laptop, maybe.

Another day, Another Doodie

So first things first, I want to really thank everybody for the sincere birthday wishes. Its so nice to see how many people are willing to spend a moment or two to send a happy birthday. I am sad that I am not as consistent as all of you. I sit in front of a computer all day, I have multiple cell phone numbers, office phone doodie
numbers, email accounts and other social contact points and yet I can not keep it together as I drown in my technology (some of which I created myself). Oh well, another year has passed for me and I worry that I am already on the back nine. I am not a particularly spiritual person and someone had once told me that God had promised man 120 years of life (Genesis 6) and even though there are many explanations on how this was about noah and the flood, I am going to take this as a promise to me, personally so maybe I am still a yute. Anyway, thank you one and all and if I missed any of your bd’s, please accept my belated birthday wish to you.Peace and Love To All!
Follow

Get every new post delivered to your Inbox.